Surveys are a useful tool for gathering insight to inform strategic marketing, but in the B2B space in particular, they have some shortcomings and blind spots. Our three rules of thumb for avoiding common pitfalls and getting the most out of your market research:
- Do target a pool of knowledgeable respondents for reliable technical feedback
- Don’t expect a consumer survey to provide robust feedback on technical value prop
- Do leverage survey data to strengthen your approach to customer engagement
And, of course, don’t treat survey output as the sole basis for strategic decision-making; rather, consider it one piece of the puzzle that will inform your path forward.
“If you would be willing to participate in a brief survey at the end of this call, please stay on the line.”
Many of us receive some version of this message almost every day – from our cable companies, healthcare providers, fast food purveyors, and more. To the average consumer, it’s an inconsequential part of the buying process (especially if you do not, in fact, stay on the line); but to the company administering the survey, it can be a hugely useful tool for gathering thousands of data points in practically no time at all. Designed the right way, a survey can drive major decisions about marketing, product development, and pricing strategy.
Newry has been seeing increasing demand for use of surveys in our strategy engagements as our B2B clients attempt to capitalize on market research methods that have provided such tremendous value for their B2C counterparts. In general, we have found that surveys are a useful and instructive input for market sizing, strategic analysis, and decision-making. However, we have also seen how easy it is for raw survey data to throw these efforts off-course.
Surveys are powerful but not perfect, and while there are some situations where they can deliver valuable insights, there are just as many (if not more) circumstances where their results can lead the best of us astray. In B2B contexts, a survey should only ever be treated as one piece of a much larger puzzle – never as the end-all, be-all. In this post, I’ll describe where surveys have shined in our work with technically sophisticated materials companies, where they have fallen short, and how we believe they can be used to provide the greatest possible value.
Here are just a few of the big do’s and don’ts:
DO: Target a pool of knowledgeable respondents for reliable technical feedback
When a recent client asked for assistance in optimizing the feature set for their optical fiber product, we suggested incorporating conjoint analysis, a survey technique that can be extremely powerful if you know which questions to ask and have access to the right respondents. Conjoint isn’t always the right tool – for example, it isn’t great for testing speculative features in a product category that doesn’t exist yet – but it can get at detailed feature and pricing preferences for existing products, particularly when your respondents are experienced professionals in their fields. By leveraging conjoint analysis and targeting a panel of industry participants who understood the technical tradeoffs of optical fiber properties, we were able to isolate the combination of features that would result in greatest preference share gain for our client.
It is worth noting that getting a good survey panel in a highly specialized field can be challenging and expensive. Although it paid off for our client in this case, others may find that it is more efficient to have in-depth conversations with 10-20 of the right people, rather than attempting to survey 100.
DON’T: Expect a consumer-facing survey to provide robust feedback on technical features or value prop
Unlike industry professionals, consumers often do not know enough or care enough to give you reliable feedback about your product’s technical specifications. In many cases, the features you’re offering simply don’t explicitly factor into their decision-making; in other instances, they may misinterpret product characteristics out of context, leading them to misrepresent their true preferences. This doesn’t mean there’s no value in the product – just that the end user might not be the right one to ask.
Surveys are powerful but not perfect, and their results can lead the best of us astray.
For example, Newry recently attempted to quantify the value of various properties of automotive replacement windshields through a consumer survey. However, because consumers typically don’t buy replacement windshields directly (insurance covers most or all of the cost), they have poorly defined and inconsistent perspectives on specific properties like optical clarity. Pivoting to assess respondent preferences more generally helped us triangulate how our client’s technology improvement might be perceived, but getting to specific pricing data required in-depth value chain interviews.
In a separate effort to define the value prop for a client’s material in smartphones, we surveyed consumers and found that they expressed overwhelming opposition to the client’s material – in some cases claiming that they were willing to pay over $100 for a different option! However, further analysis revealed that in the context of making an actual purchase, consumers paid little or no attention to the component our client would be providing. Furthermore, the people responsible for buying the client’s material – the lead designers at major OEMs – absolutely loved the look and feel of it, and fully intended to continue incorporating it into their phones.
DO: Leverage survey data to strengthen your approach to customer engagement
While the consumer point of view is unlikely to be useful in shaping a detailed technical value prop, it can be very valuable in other contexts, like customer negotiations. B2B companies often struggle to access end users or predict their desires, and getting customers to open up about which consumer problems they are trying to solve is a challenge. By using consumer surveys to cut out the middle man, B2B companies can gather data that they may later use as leverage. For instance, the results of a survey we conducted on an antimicrobial paint additive – which indicated very strong consumer interest and willingness to pay – helped our client establish a stronger price position with their primary customer.
We are strong believers that innovation and product development should be driven by unmet needs in the market, and surveys are a powerful tool for eliciting information about interests, desires, and requirements for customers and consumers alike. However, survey data should never form the sole basis for high-level decision-making; rather, it should be treated as an educated hypothesis that can then be tested using other methods, such as primary research across the value chain. As a standalone form of analysis, the survey has many pitfalls; as one tool among many, on the other hand, it can deeply enrich your outcomes.