Statistical Surveys and Non-Response Errors

Survey Non-Response

The purpose of this post is to address the concepts and issues behind non-responses in the survey. When economists or any researchers are collecting data for an academic or statistical study, non-response is becoming a more prevalent issue. Hence data collection needs to adapt to societal changes to maintain a level of academic rigor.

There are two types of non-response

  1. Unit non-response
  2. Item non-response

Either one can be selective non-response, which is a systematic difference. Systematic non-response is a severe statistical problem because of a group of responders can effectively invalidate a study. Below I give a hypothetical scenario that shows some of the reasons for non-response comes about in research.

Survey Non-response
Surveys have statistical issues

An illustration of Non-Response – Nutritional Adequacy of Low-Income Earners in the Rural US

To illustrate better the reasons for non-response, consider a survey that involves low-income earners in rural America. For example, if a researcher is studying the nutritional adequacy of low-income households, and when the household head tends to be female, there is a higher occurrence of non-response compared to other demographic categories. This means results for female lead households could be radically different from two-parent households simply because of sampling error. Therefore, with fewer female lead household responders, the data might not be as statistically rigorous for this category, and therefore, the study as a whole. Let us look at why there might be non-response in this category more than other categories.

homesteaders in America survey
With a male at home, many rural Americans grow their own food.

A Hypothetical case study from Rural America

Safety concerns of interviewee might result in non-response

In the case of one parent female lead households, women have safety concerns to identify themselves as the only adult at home, in rural areas. Therefore, single-parent females might not reply. The result is a systematic non-response. However, the nutritional profile, in reality, is different in a household when a male is present.

Additionally, this data skewing might be exacerbated in the following case. In the USA, many low-income rural households grow food and hunt and fishing for food, themselves when the male is present. (This was similar to the situation when in Poland during communism, people in the countryside often had land or domesticated animals, and this contributed to nutritional adequacy people in the city might not have access to through the standard economic system).

Therefore in the Rural US, even though incomes are low, they have moderate to high-level nutritional adequacy because of self-sufficiency coming from the land. This is even encouraged as Supplemental Nutrition Assistance Program (SNAP) benefits for food can be used to purchase seeds, plants, and trees for a backyard food plot.

People in the rural US like Appalachia, Texas, the Deep South, and the Mountain regions, have cultural values of self-sufficiency. Homesteads grow corn, have a garden, have an orchard, hunt, farm catfish, raise a chicken, pigs, or have a cow for milk, cheese, and yogurt. (Author’s note: I live in a rural area, and I personally have a sugar cane field, potato field, a garden, and a fig orchard.) 

However, single mothers do not have the time or opportunity to engage in non-income producing primary sector activities, and they and their families have lower nutritional adequacy because of their dependence on a traditional monetary economy. Because of family responsibilities, the opportunity cost of agricultural self-sufficiency is too high to take on themselves. Therefore, they do not have the same access to food as rural households with a male present (the same could be the case with male single-family households).

The result is more inadequate nutrition for household lead by single parents. Specifically, government low-income nutritional programs for families and mothers like the Special Supplemental Nutrition Program for Women, Infants, and Children (USDA WIC) specify household food purchases on this program must be the cheapest quality food because of the program’s budget restrictions. For example, milk must be the lowest price and low-fat (which is not optimal because of the need for growing children and nursing mothers to have fat for their brain development). Also, produce needs to be inorganic as does the bread. Therefore, low-income mothers are living on hormone-treated milk, white bread, and canned produce grown in soil lacking micronutrients such as chromium or selenium. These diets might be sufficient in macronutrients like protein and carbohydrates but lacking in micronutrients, which are a catalyst for health.

However, the issue here as it relates to the researcher and study is, the female head of households will not reply in a survey as mentioned above for safety reasons or feelings of self-esteem. There is also distrust/belief in the government system, which can take their children away and put them in a foster home if the state feels the family can not provide for the children, as can be the case in the United States.

Therefore, the study will show that the overall level of nutrition for low-income earners is adequate if you polled the composition and nutritional components of a family’s daily diet of low-income earners. This is because it might underrepresent the female head of households because of non-response.

That is, the case is because female lead single-parent homes are not statistically represented in an academically rigorous way, because of non-response. However, they are arguably the most important category of the study as there is a disproportional number of single parents in poverty. Statisticians and surveyors/data collectors, therefore, must overcome or account for this non-response to make it a meaningful study.

single mother survey
Rural single mothers have a large non-response

How can non-response be mitigated in this case

Ways to overcome this non-response might be a simple as hiring local trusted and known female surveyors that can go door to do. If interviewers are known in the community and mirror the demographic, in this case, female to female and a single mother themselves, the response rate tends to be higher. Also, assure the single mothers this is anonymous, and the survey is not to be used by Child Services to determine parental adequacy. Alternatively, give monetary incentives. 

Another way to mitigate is simply over-sample. By over-sampling when statisticians collate the data, there, adjustments could be made if done correctly.

It can be noted there is also Non-selective non-response but no reason that there is a difference between respondents and non-response. 

Why non-response errors are impactful

Non-response errors affect sample size and outcomes. Sample different than the entire population. It does not represent the population in a meaningful way. Needs to represent the population and non-response sampling errors can alter this.

  1. Reduction in sample size – Statistical issues, as noted above, larger sample sizes are associated with measures closer to the whole population.
  2. Unwilling participants can skew the results. The answers from the pool of responders can be different than the pool of non-responders. Each group has a different reason. Non-response bias, a particular group, is underrepresented or not represented. 
  3. Item non-response – In some cases, a particular group or demographic might respond; however, on specific questions, they might non-respond. Not replying to specific items can create an under response, which can also distort the result.

Reasons why there are non-response errors

The reasons for non-response ultimately is based on the psychology of the individual. That is their perception of the survey in the context of their defined situation.

  1.  Formal social categories (age, gender, race, political parties, religion, education level, race)
  2. Limiting social factors ( crime, extreme poverty, health, cognitive or technology deficiencies)
  3. Survey settings (meeting in person, personal appearance, subconscious signals).
  4. Survey privacy is an issue (concerns if the survey is anonymous, and the researcher protects personal data and any connection to the individual).
  5. Lack of clarity in the question can result in less response (ambiguity can lead to non-response).
  6. The mode of data collection is significant when constructing the survey. The researcher must be cognizant that the objective is accuracy, not just convenience (technology fluent individuals compared to less tech-savvy, or door to door, for example, some people might not open their door or answer their phones as a rule).
  7. Could be seen as invasive or offensive questions
  8. Psychological reasons

How to reduce non-response

  1. Try to understand the reasons for non-response in one’s particular case
  2. The chosen interviewer could mirror the demographics of the interviewee. For example, if part of the study looks at older Hispanic Males, the interviewer could be an older Hispanic male. In contrast, younger Caucasian females might not be relatable to the interviewee.
  3. Include visual graphics to explain and simplify the purpose and questions.
  4. Oversampling
  5. Dress professional can and a congenial, professional attitude
  6. Economic incentives (such as a chance to win a Starbucks gift card if you reply).
  7. Follow up.
  8. Have a clear privacy policy, and ensures data is anonymous.
  9. Have the timing of the call or interview respondent centered rather than researcher convenient.
  10. Before the use of the questionnaire, do a user testing for clarity before the release. This way, you have feedback on how to improve the sample questionnaire for higher response. This allows researchers to improve on their mode and format and clarity of the questions.
  11. Give respondents multiple ways to respond, such as email or post.
  12. Give a clear summary of the importance of the research. That is why the study will be beneficial to society as a whole.

Related Posts




This is my Youtube Channel: EconLessons


Leave a Reply