Cognitive pretesting

Cognitive Pretesting, also referred to as Cognitive interviewing, is a field research method used primarily in pre-testing survey instruments developed in collaboration by psychologists and survey researchers. It allows survey researchers to collect verbal information regarding survey responses and is used in evaluating whether the question is measuring the construct the researcher intends. The data collected is then used to adjust problematic questions in the questionnaire before fielding the survey instrument to the full sample.

Although survey researchers do not totally agree as to what a cognitive interview entails, it in general collects the following information from participants: evaluations on how the subject constructed his or her answers; explanations on what the subject interprets the questions to mean; reporting of any difficulties the subject had in answering the questions; and anything else that reveals the circumstances to the subject’s answers.

Types of Cognitive interviews

In general, there are two methods practiced when conducting a cognitive interview.

First, the think-aloud method encourages participants to verbalize their thoughts while responding to the survey questions. This method is considered purer as it reduces the possibility of the interviewer introducing any bias into the participants’ answers. In contrast, the disadvantage to this method is that it does require training on the part of the participant on the think-aloud process which can be burdensome to the interviewee.

Second, the probing method has the interviewer ask detailed probes after a respondent answers the survey question. An example of a probe question is: “In your own words, what is this question asking?” or “How did you arrive at your answer?” Advocates for this method suggest that follow-up probes do not interfere with the actual process of responding to survey questions while requiring very little training on the part of the respondent.

Participants and recruitment

In order to conduct a cognitive interview on a survey instrument, the researcher should recruit a minimum of 10 participants to a maximum of 25 participants. To ensure the validity of the cognitive interview results, these participants recruited should reflect the diversity in the population being studied.

Use by survey researchers

Cognitive interviewing is regularly practiced by U.S. Federal Agencies, including the Census Bureau , National Center for Health Statistics and the Bureau of Labor Statistics .