On Device Research recommends that mobile surveys should be on average 10 to 15 minutes long.
On Device Research have carried out over 30 million surveys and over this time it has been seen repeatedly how mobile surveys can produce accurate, reliable, high quality data. But it's also become clear that when it comes to the length of surveys, size isn't always everything.
It is understandable that clients desire longer surveys; they have been used to this via face-to-face, telephone and online methodologies. There is a temptation to replicate this with mobile surveys, yet what works in a traditional context doesn't always translate to the mobile environment. Keeping this in mind, along with the fact that almost a third of online surveys are now being taken 'accidentally' on mobile devices, On Device Research has carried out the most in-depth research on research yet to pose the question: how does survey length affect the quality of the data collected?
To answer this, On Device Research conducted a controlled experiment. For full details of the methodology please download our report. The data was collected in Thailand, a mobile first market, in the local language.
The results show that there is a survey length 'sweet spot': the best quality data can be assured in surveys of 10 to 15 minutes in length. The longer the survey, the greater the deterioration in the quality of the data. If the survey is too long, respondents begin to speed through questions, give inaccurate responses or worse still, drop out altogether.
KEY HIGHLIGHTS
- A key metric to highlight data quality is the number of checkbox selections for the same question at different positions in the survey. For instance, 4.7 checkboxes are selected in a 14 minute survey compared to 3.3 in a 34 minute survey- a 30% reduction. Therefore if you are asking about the awareness of shampoo brands towards the end of a 30 minutes survey, up to 2 brands per respondents will not be collected, compared to a 14 minute survey, therefore under-reporting awareness considerably for some brands.
- Respondents answer the same set of questions almost 40% quicker if they are at the end of a 34 minute survey compared to a 14 minute survey.
- Drop-out rate also increases with survey length, making it difficult to achieve lower incidence quota targets for long surveys.
It is widely reported that around 30% of online surveys are now completed on mobile devices, which means that all data collected on online surveys longer than 20 minutes will potentially contain inaccurate data. As online surveys increasingly become de facto mobile surveys then, the importance of keeping them concise is more evident than ever: On Device Research recommends that mobile surveys should be on average 10 to 15 minutes long, with no single respondent taking longer than 20 minutes. Keeping this best practice in mind, mobile surveys can provide excellent quality data with accuracy and reliability at the forefront.
<odr:wgHref class="btn btn-primary" filename="20151013173223_20151008125941_ODR-30mins-Final.pdf">download report</odr:wgHref>
