How to Increase Data Quality in Online Surveys Research

5 Things To Know When Building Your Own Respondent Panels

girl with smartphone
Establish Quality Connections With Your Panel Respondents. © Francisco Hernandez Vega |

The convenience and speed of online surveys research come with a number of important challenges that market researchers must be prepared to address. A fundamental consideration with surveys research that is conducted in digital environments is the absence of a connection (beyond the necessary electronic connection) between the market researcher and the survey respondent. While market research that is conducted face-to-face is certainly not without issues that center on data integrity, the anonymity of electronic environments can substantively increase those challenges.

Each platform on which surveys research is conducted brings its own set of challenges. Market researchers must adhere to rigorous methods for recruiting and selecting research respondents. Questionnaires and surveys must be constructed in such a way that they approximate the affective and cognitive processes that consumers use when actually contemplating, researching, and executing purchase decisions.

The right type and amount of incentives must be selected and dispatched in order to keep research respondents motivated and – importantly – that encourage the response behaviors that result in useful, reliable, and valid data. Eliciting the desired set of survey-taking behaviors from research participants is not a simple matter, and it is worthy of considered thought and planning on the part of market researchers.

How Honest Are Your Survey Respondents?

A perennial concern at the beginning stage of surveys research is figuring out efficacious ways to identify, mitigate, or eliminate the influence that dishonest respondents can have on the outcomes of surveys research.

Here, we focus on what is known as front-end data integrity. Surveys research depends on recruitment processes that are not unlike those used by job recruiters, college admissions directors, and membership committees of fraternal organizations. In each of these situations, not everyone will be accepted, nor should everyone gain entry to the select groups.Considerations for Commercially Developed Respondent Panels

If front-end data collection vulnerabilities are to be reduced, market research respondent panel must consist of the right potential participants selected through disciplined, purposeful methods. If a market researcher accesses respondent panels by paying market research service providers, then it is safe to assume that many of the challenges regarding front-end data vulnerability have been addressed.

Is It Worth the Money to Engage a Respondent Panel?

However, it is wise, when planning to work with commercial market research service providers, to consider that some firms have resorted to the construction and management of large respondent panels as a demonstration of their expertise and quality. All too often, these immense respondent panels have been constructed by adding panel members from purchased email lists. In these instances, the resultant respondent panel is doomed to be of poorer quality, and the vulnerability of front-end data will have been increased through these disreputable methods.

It is not unusual for commercially available respondent panels to need supplementation or adjustments in order to adequately meet the needs of particular surveys research. However, disclosure of those changes to the panel – whether additions, deletions, or substitutions – must offered and the rationale for the changes clearly explained to the market researchers who are purchasing access to the respondent panels.

Considerations for Construction DIY Respondent Panels

Below, examples of the methods used by market research service providers are listed along with some basic considerations and ideas for do-it-yourself market researchers who face the challenge of conducting surveys research without expert help.

Be Choosey Gather detailed information about potential respondents during the recruitment and registration process. Consider carefully the specific focus of the surveys research in which participants will be engaged.

Invite Best Fit Matches Resist the urge to pad the panel in advance of anticipated drop-outs. Email individual invitations to only those respondent who precisely match the profiles of questionnaire respondents that have been established as part of the overall research plan.

Follow-Up Be sure to do the necessary follow-up with respect to bounce-backs of emails.

Ensure email addresses are added to white lists or safe lists so that they won’t be caught in spam filters. Commercial market research service providers use third-party firms to certify that lists are safe and to monitor the email behavior of those answering invitations to participate in panels

Cull the Herd Be sure to cut out or partition off those email addresses that are suspect or that have bounced back. There are two types of bounce-backs: (1) Soft bounce-backs are temporary and caused by mailboxes that are over capacity, and the like. (2) Hard bounce-backs are permanent and indicate that the host server is inactive or that the mailbox no longer exists. Soft bounce-back emails can be resent to recipients after pre-determined intervals in an attempt to establish a connection. Hard bounce-back emails should be removed from the respondent panel list so as not to contaminate future correspondence or to overstate the member count of a respondent panel.

Don’t Be Kept Waiting When no response is received after several email invitations to the same recipient (for emails that have apparently been received and no bounce-backs have occurred), then it is time delete those email addresses from the safe list.

Cutting out the dead wood and refreshing the potential respondent database will help to avoid non-respondent bias issues and ensure increased response rates from higher quality respondents.