DIY Online Research - 5 Things to Avoid
Recent years have seen a tremendous improvement in online survey tools and by extension, a proliferation of inhouse or Do It Yourself “DIY” surveys.
As a long-time practitioner of research and defendable insights I am always pleased to see companies incorporating more primary research into their business planning. In particular, when those insights will strengthen marketing results.
Online Survey Options
Many of the available online survey tools offer basic versions for free. I like both Qualtrics and Survey Monkey (albeit there are several I have not used). Both offer greatly improved free versions from even 5 years ago; with survey design wizards and templates structured to match the nature of the study. [e.g. Customer Satisfaction, Product Acceptance, Event Feedback, and more]
These templates include sample questions, and perhaps equally importantly proposed ranking and pre-set descriptive responses. They make it easy for novice users to avoid the worst pitfalls in survey design.
However, and there is always a however. Good survey design does not in itself prevent the shortcomings of substandard sampling or weighting of responses. For this you need to understand both your business and best practices of research.
5 Things to Avoid
1. Asking Existing Customers Only
Inviting existing customers only in itself isn’t an issue, but beware assuming these responses reflect the feedback of the broader market. This is especially true for B2B companies. The reasons that your customers gravitate to your product/service may not hold true for the rest of the market. So if you have a 5% share, you may be making critical mistakes if you build product preferences from existing customers into your overall product development plan. Be certain to include a defendable representation from those that don’t buy your service/product. They may not value the same features as those that purchase your product or service.
2. Loading up on friendly respondents
It is not uncommon, where survey responses are underwhelming, for the Sales team to be pressed to solicit their contacts to shore up the overall response rate. By the very nature of these contacts, they are typically friendly to your company. While they may not be customers, often their responses are skewed towards a positive feedback that may not be representative of the market. Much like the above, there is potential for Confirmation Bias. The friendly responses reinforce what the evaluator hopes to hear or already believes, and as such does not adequately probe the results for weaknesses.
3. Unqualified participants
In the age of social media and web based surveys, some companies look for sheer numbers of responses, and interpret this as valid in itself. However, when you make the survey available to literally anyone, a disproportionate number of these may not be qualified respondents, or worse, respondents with hidden agendas trying to sabotage your insights. This might be OK for very superficial questions, but an audit of responses is prudent as part of the evaluation. Consider eliminating any business responses that come from Hotmail, Gmail or other third party internet providers. In addition to challenges in the sampling listed in the first three pitfalls above, there are other common shortcomings.
4. Extra Long Surveys
In an age where everyone is busy and anyone can publish their own survey, 5 minutes is a long time. Two minutes might be enough to glean the most important insights. Shorter is better! The completion rate is inversely related to the length of the survey. Moreover, if all questions are required for completion, the quality of answers degrades as well. While all questions may be answered, those in the last third may be answered out of frustration to finish. If you really can’t reach a consensus on a short survey, make certain that the most valuable nuggets are gleaned early in the survey.
5. Over Relying on Answers About Pricing
“What is the maximum that you would pay for product X?” This is a question that is inherently fraught with difficulties, especially when asking Business customers. They will often say less than is the complete truth, as this question makes the survey start to feel like a form of negotiation. It is always in their interest to be honest about what features they want, but why announce how much extra you will pay? This is especially true if they are genuinely interested in the product/service. Secondly, for complex solutions, they are rarely able to fully understand the value until it is explained to them via a comprehensive demo or in-person presentation. Price point sensitivity is best explored using alternate research methods.
Mark Gallagher
Principal – Beresford Group
Mark brings over 25 years of senior experience in B2B marketing and business development. Passionate about the climate crisis, brilliant marketing, sustainable technologies and helping to build the next generation of leaders.
Feel free to contact the Beresford Group for more on best practices on insight based marketing strategy and execution.
Follow Beresford Group at LinkedIn www.linkedin.com/company/beresfordgroup