Last year, I wrote a series of articles with survey design tips for new market researchers, whether doing DIY market research or new to the field in a market research company.
Those articles focused a lot on all of the preparatory work that should go into designing a questionnaire, but aside from an article about scales, didn’t spend a lot of time on the actual writing of a survey. That will be my focus for the next few months: survey writing tips. Because good survey writing is difficult; far more difficult than you might expect.
Screeners
One of the many items to consider for writing a survey is defining your target audience. Once you’ve decided who you actually want taking your survey, next comes the much needed step of writing the questions that will help qualify people to take your survey. In market research terms, this is often referred to as “qualifying respondents into a study” or “screening respondents into the study.” (We sound so high-falutin’ sometimes.)
These questions make up the “screener.” Think of it similar to a first-round interview with a recruiter for any given company (also often referred to as a “screening” interview). The questions can be fairly general, but they help both you and your audience figure out if the respondent is the right fit for the survey. In a survey, these questions might include gender, employment status, annual income, and knowledge tests to see if your audience knows enough about your topic to take your survey.
Here are some example questions that might be included in survey screeners.
- What year were you born?
- Please indicate your employment status:
- Work full-time
- Work part-time
- Student
- Unemployed
- Homemaker
- Please indicate your annual gross income range:
- Less than $30,000
- $30,001 – $49,999
- $50,000 – $79,999
- $80,000 – $99,999
- $100,000+
- What is your role in your company?
- CxO
- Director
- Sales
- Information technology
- Marketing
- Other (please indicate)
- Have you used X product in the past 6 months?
- Yes
- No
- How familiar are you with X product?
- Not at all familiar
- Somewhat familiar
- Very familiar
Depending on what information you need are the questions you’re going to want to ask. When it comes to income ranges or age ranges, be sure that you are creating brackets that you will actually care about, yet not so narrow that your option list is super long. After about five options in any given list, the reliability of the responses starts to drop. Additionally, these are the very first questions your audience will be seeing. If you start off with super long questions and long lists of answer options, you’re essentially telling your audience that they’re going to be in for a lot of reading, and you will likely lose a large number of potential audience members.
Termination points
If you need to be sure that only certain qualified candidates make it through to the heart of your survey, you’re going to need to create what market researchers like to call “termination points,” where you can “route respondents out of the survey.” Sometimes, we like get all fancy and just refer to these as “term points.” (Maybe it’s just so we don’t sound so morbid…) These are questions which, depending on the answer, will allow your candidates to continue the study, or else disqualify them from the study.
These term points should really be as early as possible in the study so as not to waste your audience’s time. If there is one particular question that would immediately tell you whether or not someone should qualify for your study, by all means, put that question first. It’s so much better to take only a few seconds before they find out they can’t finish your survey than to have them spend a few minutes answering questions before they learn they’re not qualified. Again, if you think of this in terms of a candidate going through an interview process for a company, the same general principle applies. You don’t want that candidate to go through a 30-minute phone screen, then an in-person interview, only to find out that you’re looking for an accountant, not a technical writer.
When using a survey programming tool, you will often find the option to include a termination point based on a particular answer to a specific question. If you need to gather a few answers before you find out whether or not the audience is qualified, you should be able to include logic with the termination point, such as, “If annual income < $49,999 AND employment status = unemployed, show thank you page." Last piece of advice on termination points: once you've determined where they will be, go back through and be sure that you actually do not want those people to provide their opinions. For example, I was working on a study recently that was focused on people who had experiences with a particular product. We realized that, even if someone didn’t have an experience with that product, there were still questions we wanted to ask them, such as whether they were familiar with the product and whether they had any interest in the product. While the perspective of those who had used the product was of primary importance, there was a lot we realized we could learn from those who hadn’t yet used it.
The “you did not qualify” message
Remember that on the other side of the survey screen is a person, first and foremost. This means be kind, and be grateful they even opened your link and attempted to take your survey. Simply create a thank you page to show unqualified respondents so that they know they didn’t qualify but thanking them for their time. I have actually taken surveys where I simply saw a blank screen instead of seeing any “thank you, but you didn’t qualify,” or even worse, a simple, “You did not qualify for this study.” Be kind in your wording instead of being abrupt. It may leave your audience more likely to come back for an encore.
Next post, I’ll focus on some easy-to-make question writing errors and how to fix them. Please leave me a comment and let me know what questions you have on survey design and survey writing!