Links to IIIT 2019 Istanbul Conference presentations

Presentation 1
Presentation 2


 

Author: Jehanzeb Cheema
Updated: April 20, 2019

This webpage provides a non-technical introduction to best practices and ethical considerations in data collection using in-person interview-based standardized survey instruments. A summary of this content can be downloaded in the form of a PowerPoint presentation using the download link to the right. The external links point to additional resources that can be used to enhance and complement the material presented on this webpage. In order to make the material accessible to individuals who are expected to be involved directly or indirectly in AEMS data collection but may not possess a strong background in survey research or statistics, the use of technical terms has been kept to a minimum.

Introduction

Whenever data is collected from a sample of respondents there are several potential sources of error that can negatively affect the accuracy and reliability of statistical results. While some error is inevitable in this type of data collection the guidelines discussed below can help minimize such error by improving the quality of data collected through interviews. Improvement in data quality is directly tied to improvement in the quality of statistical results based on such data.

Pre-Interview Considerations

The following issues are important to take care of before the interviewers come into contact with survey participants.

  • Interview authorization. In some locations interviewing students under the age of 18 may require parental consent. Such consent should be secured before personnel are dispatched to interview sites. In some countries formal authorization may be needed from the relevant ministry, department, or school administrator. All such required authorizations should be secured before the date of the interview.
  • Workload. It is important to ensure that interviewers are not over-burdened and have a reasonable number of interviews to conduct. Interviewer workload needs to be planned in advance and is less of a concern in paper-and-pencil based surveys where respondents themselves complete the questionnaire. In such cases the interviewer's primary job is to distribute the survey questionnaires, provide clarification for any questions, and collect the completed questionnaires. However, in cases where the interviewer needs to personally fill in the answers based on participant responses and/or physically visit multiple interview locations a large workload can lead to interviewer fatigue and contribute to undesirable variation across interviews. Careful workload management from the very start can help avoid possible disruptions in the data collection process.
  • Experience and training. The quality of an interview and response rate is generally directly proportional to the experience of the interviewer. An effort should be made to recruit interviewers with prior experience in conducting interviews. In situations where this is not possible the interviewers should receive some form of training prior to field visits. Such training can take the form of an experienced interviewer briefing the novice personnel, or reading relevant material such as that recommended in the External Links section of this webpage.
  • Appearance and other characteristics. The appearance of an interviewer is by itself sufficient to affect participant responses. It is important that the interviewer appears friendly, approachable, and ideally someone that the respondents can relate to. In some cultures having a male interviewer collect responses from female respondents, or vice versa, may not be a good idea. Similarly, in some cases survey respondents may not take interviewers seriously when the interviewers are less educated than the respondents. Care should be taken to ensure that other characteristics of the interviewer such as age, ethnicity, race, education, and nationality do not affect participant responses. Past research strongly suggests that an interviewer's behavioral characteristics such as optimism and self-confidence can improve response rates significantly.
  • Interviewer safety. It is important to ensure the safety of all field personnel. This includes interview administrators not asking interviewers to visit interview sites where there may be potential for physical harm (e.g. city locations with high crime rate or possibility of violence), being aware of daily interview schedules and locations, maintaining some form of contact with the interviewers in the field (e.g. through mobile phones), providing safe transport (e.g. cars) to interview sites, avoiding interview visits after dark and on weekends, and generally making sure that the interviewers are not subject to any harm or negative consequences as a result of their participation in the interview process. For example, in some parts of the world cultural norms and conservative values may make it infeasible to send female personnel into a rural area to conduct in-person interviews. Such issues can sometimes be resolved by asking interviewers to visit in groups or by providing them with security escorts. All recruited interviewers should carry proper identification documents clearly linking them to the survey organization.

Considerations during the Interview

Human-to-human interaction cannot be avoided when information is collected through interviews. There are several issues that are specific to this mode of data collection.

  • Voluntary nature of responses. Participants should know from the start of the interview that their participation in the survey is voluntary and that they are free not to respond to any question that they are not comfortable with. They should be made aware that there would be no negative consequences for them if they decide not to participate in the interview or decide to quit the interview before it has been completed.
  • Respondent privacy. Immediate presence of individuals other than the interviewer can result in potentially biased responses. For example, a school teacher may respond to survey questions differently when she knows that her supervisor can hear her answers. Similarly, students may respond differently and may not be forthcoming in their responses in the presence of their teachers or parents. It is the interviewer's responsibility to ensure the privacy of participant responses at the time of data collection.
  • Uniformity. In order to ensure that interviewer characteristics do not affect survey responses it is important to guard against two sources of variability:
    1. All interviews conducted by a single interviewer should be as similar to each other as possible in terms of the interview environment and content. For example, interviewing teachers at the beginning of the day when they are fresh and are likely to be not in a hurry can generate different results compared to similar interviews that are conducted at the end of the day when the teachers may be in a hurry to go home.
    2. Interviews conducted by different interviewers should be as similar to each other as possible. In other words, interviewer characteristics (such as age, gender, appearance etc.) should not affect participant responses.
  • Leading actions. The interviewer should take special care in not leading a respondent towards or away from an answer choice. This can occur subconsciously through body language (for example, when the interviewer nods or frowns when reading answer choices) or through variation in speech (for example, sounding more enthusiastic about a particular response choice as compared to the alternatives). Such actions can potentially cause a respondent to sleect an answer that he/she would have otherwise not selected.
  • Changes to the survey. Survey instruments are standardized and the order in which questions appear is carefully chosen. It is thus important for the interviewer to stick to the language and content of the survey. Practices such as altering the wording of questions, translating questions into another language, and altering the order in which questions appear in the survey questionnaire can all contribute to distortion and unintended variation in survey responses, and thus should be avoided.
  • Recording responses. In some cases the interviewer may need to record survey responses on behalf of the participant. For example, when the participant is illiterate, or is unable to complete the survey by himself/herself due to a physical condition. In such cases care should be taken to provide sufficient time for the participant to respond to each question, and to ensure adherence to survey content. The interviewer's job in this case is only to record responses and thus under no circumstances should the interviewer complete any questions that are left unanswered by the respondent. An exception in this case can be made for demographic questions (such as gender, location etc.) the answers to which can be directly observed and thus do not require interpretation or guesswork on behalf of the interviewer.
  • Nudging for information. It is acceptable for the interviewer to gently nudge the respondent back on track when the interview turns into a personal conversation, to use words of encouragement in order to keep the respondent interested and engaged, and to probe in order to minimize survey non-response. However, such nudging or probing should not be aggressive and the respondent should at no point feel that he/she is being forced to answer a question.
  • Checking for completeness. Given the anonymous nature of the interviews in most cases there will only be one interview opportunity per respondent. It is thus important that the interviewer parts with the respondent after ensuring that all required information voluntarily provided by the respondent has been collected.

Post-Interview Considerations

The following issue requires consideration after the data has been collected from survey participants.

  • Data security. Once the data has been collected from the survey respondents it is the interviewer's responsibility to keep such data secure. This can be ensured by:
    1. keeping the collected survey responses in personal custody until they have been delivered to a supervisor,
    2. not sharing data with an unauthorized person even when such person has helped with data collection (e.g. a teacher's survey responses should not be shown to her supervisor even when this supervisor helped arrange the interview in the first place),
    3. ensuring that there is no personally identifiable information that can potentially connect a survey response to the corresponding participant,
    4. not making unnecessary copies of survey responses,
    5. keeping the response files (whether physical or electronic) under lock and key (or encrypted) at all times, and
    6. destroying the source material (e.g. shredding paper documents or deleting electronic files) once such material has served its purpose and is no longer required.

 
References

  • Fowler, Jr., F. (2009). Survey research methods (4th ed.). Thousand Oaks, CA: Sage Publications, Inc.
  • Survey Research Center. (2016). Guidelines for Best Practice in Cross-Cultural Surveys. Ann Arbor, MI: Survey Research Center, Institute for Social Research, University of Michigan. Retrieved July 2, 2018, from http://www.ccsg.isr.umich.edu/
  • West, B., & Glom, A. (2017). Explaining interviewer effects: A research synthesis. Journal of Survey Statistics and Methodology, 5(1), 175211.