While respondents are completing surveys, a wide variety of quality assurance steps are taking place behind the scenes to ensure the highest quality data is collected. 

 

Time of completion:

Once a minimum number of surveys are completed, the average time of completion is calculated. Following this, any survey that is completed significantly more quickly than the average is flagged for further investigation. If the respondent answered in ways that appear illogical or haphazard, their survey will be removed from consideration.

 

Patterns of completion:

If a respondent’s particular survey has an identical pattern of response regardless of the question, it too will be flagged for further investigation and removed if deemed necessary.

 

Attention-tracking questions:

Attention-tracking questions help ensure that respondents are genuinely reading the questions they are asked before answering them. With this technique, at random points throughout the survey, respondents might be asked something simple like “select the word ‘blue’ from the list below". Any respondent who fails to answer these questions correctly is assumed to not be paying sufficient attention, and will be excluded from the final data set.

 

Oversampling:

We systematically obtain more completed surveys than needed (generally 5% more). This allows us to eliminate any respondents who are flagged, and yet still end up completing the sample size we originally intended to fulfill.

​

If you would like to learn more about our quality assurance processes or our quantitative methodologies in general, please contact Nathalie Martel (nathalie.martel@callosum.ca).