If you’ve followed the previous few weeks’ worth of survey design tips, you might notice I’ve been spending a lot of time in the efforts and questions that should be answered before you even start writing the survey. There’s a reason for that – taking the time up front to address these items will save time and effort in the long-run, and will have a major impact on everything from what questions you’ll want to ask, to what question types you’ll want to use, to what the plans are for the data. Today, let’s address another pre-survey-writing question: how will you field the survey?

What does “fielding” mean?

If you’re new to the field of market research, the term “fielding” might not make much sense. In short, fielding a survey just means gathering the responses. There are many ways surveys are fielded: phone, mail, paper, mall intercept, online, email, mobile, panel. Each of these has pros and cons, and will effect how you set up the survey.

A review of the types of fielding

• Phone. Phone fielding is perhaps the most traditional survey fielding method. However, recent changes to the rules around this may necessitate a shift away from this methodology. The benefits to phone fielding: higher response rates; higher data quality; potentially fewer issues with things like straightlining. Drawbacks: there are a number of biases that can be introduced into phone surveys, from a bias from the way the questions are being asked by the surveyor to biases like social acceptability from the respondent (where someone gives an answer that they think won’t make them look bad over an honest answer).
• Mail. Sending paper surveys by mail is also a very traditional survey fielding method. There are many studies that investigate how to get high response rates from this method, including how, what type, and when to offer rewards for responses. The benefits are similar to those obtained with phone surveys. In fact, often, mail and phone will be combined to achieve high response rates. However, one of the drawbacks to mail response is the data entry required for all of the paper surveys later and the high cost associated with this method, from postage to data processing later.
• Email/online. Interestingly, email, while ubiquitous, is still relatively new, having become more mainstream in the past 10 or so years. This survey fielding method is actually how I personally started in market research; all of the surveys I originally fielded were done via email. In this case, it meant the equivalent of a paper survey put into an email, so the respondent would simply reply to the email and include their responses in the text of the email. It quickly changed to being an email invitation to known recipients, with a link to the survey hosted online. At the time (around 2005), response rates for the latter email method were around 30%, which was considered really good (still is considered really good). However, this method of fielding has seen declines in recent years, which has given rise to all sorts of methods being employed to try to increase response rates again, from rewards to embedding the first question in the body of the email to kind of hook the survey participant. Benefits of this method include capturing data quickly. However, drawbacks include the fact that the response rates are declining and an increased need for data cleaning to make sure responses are valid.
• Mall intercept. Mall intercept surveys used to have a number of limitations: paper surveys that needed to have data manually entered later. However, this method is easier now with the ability to host a survey on a tablet and have participants answer on the tablet as opposed to recording responses on individual pieces of paper. One of the drawbacks to this is, depending where you are intercepting people, you might not be getting a truly representative sampling of the population that you intend to survey. (It’s also difficult to get people to even make eye contact when you’re trying to get their attention to take the survey you have in hand.)
• Web intercept. With so many businesses having an online presence now, web intercept has become the new mall intercept. However, this method has plenty of drawbacks, including the potential to drive away customers because they are annoyed at having a survey invitation pop up while they’re navigating your site, interrupting their online experience. Having been involved in fielding this type of survey before, getting responses can also be difficult, and requires a lot of promotion to the survey to get responses.
• Mobile. Mobile has kind of become the new email/online with the increase in people using mobile devices to access the internet. This particular fielding type is getting a lot of attention lately as researchers are finding that the same survey that was traditionally fielded online does not always work well for a mobile experience. This methodology definitely has its drawbacks, such as limitations on the question types that can be used for a good mobile experience, time limitations (if you want high response rates), and limited ability to gather in-depth open-ended responses along with the quantitative data you’re collecting in the survey. However, done right, you’ll get good response rates and good data.
• Survey panels. You can purchase panel responses from a variety of companies these days. And if you are looking for a specific demographic, this is an easier way to achieve that. However, research has shown that panel participants don’t always share the same behaviors as the general public, and the quality of panel responses varies. One experience I had recently with using panels for a survey project resulted in data being skewed because we had an over-representation from one particular sub-demographic on the panel. While weighting can be used after fielding to account for that, I personally think the less weighting that has to be applied to your data, the better.

How does the type of fielding affect my survey design?

As you might tell from the brief review of the variety of survey fielding options, all sorts of elements to your survey design can be affected. Timing, question types, types of biases to screen for, data cleaning — all are items to be considering depending on the method you intend to use to field your survey. I didn’t cover some of the other types of fielding such as QR codes or asking the participant to type in the address at the bottom of a receipt, though they also have their own drawbacks (population representativeness; response rates) and considerations to take into account when deciding to employ each. However, hopefully, you’ll include this consideration early in your design process.