Survey Research Unit (SRU) - Measurement

Centre for Research on Inner City Health - Keenan Research Centre

Measurement is a core component of high quality survey research. In order to collect valid and reliable measures, our measurement strategy includes question development consultation, piloting and pre-testing, interviewer training, interviewer behaviour coding, inter-rater reliability monitoring, 10 per cent quality control call backs, and ongoing skip pattern and question consistency checking. The SRU also selects key variables for each project to be monitored for inter-rater reliability, and hosts interviewer brainstorming sessions to facilitate continuous interviewer training and reliability monitoring.

Measurement Quality & Control

'Source: SRU Methods 2010

Interviewer Training

SRU interviewers undergo general and project-specific training. Ongoing refresher and feedback trainings, both in groups and individually, are also an important part of our strategy.

Interviewer Behaviour Coding

Regular consistency checks of our interviewers are performed using standardized behaviour monitoring tools. We monitor 10 per cent of all Computer-Assisted Telephone Interviews (CATI), while five per cent of our Paper-Assisted Personal Interviews (PAPI), Computer-Assisted Personal Interviews (CAPI), and Computer-Assisted Web Interviews (CAWI).

Inter-rater Reliability

Our interviewers undergo group trainings to ensure all information is communicated consistently to all project staff. Data collected is regularly checked for consistency across interviewers to ensure inter-rater reliability.

10 per cent Quality Control Call-backs

Our research staff make 10 per cent quality control call-backs to ensure interview quality and participant satisfaction.

Skip Patterns and Question Consistency Checks

To further guarantee quality, we have built-in protocols for skip patterns and question consistency monitoring (automatic for CAPI, CATI and CAWI surveys). As part of our protocol, we randomly review 10 per cent of the cases entered into our database for quality assurance. The data entry protocol utilized by our team ensures that no surveys or participant responses are duplicated, and have skip patterns programmed into the survey to reduce data entry error.

Pre-testing

Our pilot studies and pre-testing techniques ensure that the surveys we utilize meet the social and cognitive needs of different populations of interest.

Question Development Consultation

When developing your survey, the SRU can help you optimize the questionnaire by providing consultation on the survey instrument including content, readability, length, order, and formatting.