Create Representative Samples for Stronger Survey Data

Good Survey Research Design Starts with Strong Sampling Strategy

Father and daughter drawing with markers while on the floor.
Draw a Representative Sample When Doing Surveys Research. Getty Images | David Sacks | Digital Vision

Surveys Research - Sample Selection

In a perfect world, a survey research project could study all the members of a target universe. Generally, this is neither practical nor affordable. Instead samples of the larger population (universe) are generated - the sample is the base from which assumptions are made about the target universe. Further, the sample is constructed by using techniques and strategies that contribute to a valid and reliable study.

Traditional market research is based on the idea that a sample - a representative group of respondents -- can be identified and accessed.

Representative Samples in Survey Research

In market research, the term representative sample refers to:

    • The proportions of members to whom relevant characteristics can be attributed in a sample must closely approximate the proportions of members in the targeted universe of consumers.
      • For example, if the consumer universe contains business people, college students, and senior citizens, a representative sample could not be built from agreeable students in the university bookstore on Wednesdays afternoons.
      • Access to survey participants can be difficult. This is one of the main reasons why professional panels of consumers are often used in survey initiatives.
      • Another effective strategy is to use a stratified random sampling procedure that assists a researcher to tease out data about sub-groups.

      Sample Selection in Survey Research

      Members of a sample are selected in a number of ways that are intended to reduce bias. This means the probability of generating valid research conclusions is increased, and the conclusions can be generalized to the target universe.

      Survey samples are preferably selected through a randomizing process. For instance, if sample members are selected from a database, every third member in the database listing might be selected. Occasionally, members of a sample may need to be assigned rather than randomly selected. This is not a preferred approach as, even under the best conditions, surveys are subject to sample-based inaccuracies that have everything to do with chance and nothing to do with research design. Let's look at a list of sources of error, modified from voter telephone polling issues identified by Experimental Resources. This list includes possible sources of inaccuracies across survey design, survey implementation, and analysis of survey data:

      • Incomplete information about members of a database result in important variables being left out of the sample
      • Sample members who were selected are unwilling to participate in the survey.
      • Sample members who decline to participate in the study are different with regard to an important variable in the study, than those sample members who agree to participate.
      • Survey respondents provide false or incomplete responses to survey questions.

      The items in this list, again modified from the telephone polling list by Experimental Resources, are related to survey design.

      • A randomization process was used but - by chance -- it picks up too many outliers.*
      • The questions on the survey are worded poorly and confuse the respondents.
      • The order of the questions on the survey unduly influences the responses of subsequent questions.
      • Survey responses are subjected to weighting or grouping that distorts the data.

      Once a market researcher is reasonably comfortable that a sample is representative of the target population in his survey research, attention can shift to consideration of sample size and confidence intervals.

      Experiment Resources is an interesting website created by psychology researchers who were trying to figure out how to calculate and remove outliers.