Likert. Likert-style. Positive. Negative. Words. Numbers. There is so much about scales for surveys that can be so confusing when you get right down to it. Truth be told, choosing the right scales and right scale setup for your questions in your survey can be both easier than expected and tougher than expected. Let’s look at how scale selection and presentation can be made easy.
Mitigating for bias
There is a type of selection bias for scales that is called left-side bias. This bias simply means we are more likely to select the answer to the left-most side of a scale. Interestingly, this bias is more pronounced for 100+ responses, where the scale was presented with the most positive answer to the left. If you want to read more about this, check out a study by Jason Chan from the University of Texas at Austin (pdf download).
In other words, if you’re expecting more than 100 responses to your survey, make sure your scales are presented with the most negative answer to the left.
You might be thinking, “How about if I just don’t have a left OR right side to my scale, and show my scale vertically instead?”
In 2009, an article from the Journal of Office Statistics (pdf download) outlined the results of a series of studies looking into scale presentation. Horizontal scales seemed to produce less bias than vertical scales. This series of tests also included testing words versus numbers in scales and found that words tended towards less bias than numbers.
Net recommendation: to mitigate bias, arrange your scales left to right, negative options to the left, and use words instead of numbers in your scales.
Odd or even?
Next question: do you use an odd number of options or an even number of options?
Great question.
An article from 1972 in the Journal of Applied Psychology (pdf download)addressed this one. The researchers tested scales using 2-19 values, and testing to see the effects on time, how much of the scale was used, and how often the middle values were used (for the odd numbers of values). Results showed that anything over 4 values in the scale resulted in only about 60% of the scale being used; the more values in the scale, the less likely the middle value was selected; and time to complete the study was not consistently affected.
Another article I found showed that culturally, respondents will use scales differently. A 2002 article in Research in Nursing and Health showed that Chinese and American respondents fared better with 4-point scales; Japanese respondents fared better with 7-point scales. (The measure used for both was construct validity – whether the question was measuring what one intended to measure.) Chinese and Japanese respondents also tended to select middle values when the question had to do with measuring positive emotions; Americans were more likely in these questions to select the more positive emotions.
Net recommendation: generally speaking, consider how your audience typically responds, because culture plays a big part in how scales will be used. If you really want your survey participants to be more likely to take advantage of the full scale, use a 4-point scale.
Bipolar or unipolar scales
Bipolar scales are the type that range from one end of a spectrum to another, such as “Strongly agree” to “Strongly disagree.” Unipolar scales keep everything on the same spectrum, so “Strongly agree” to “Do not strongly agree.” While setting up a unipolar scale can feel a bit awkward at first, studies have shown that unipolar scales produce better results than bipolar scales. The reason is simple: survey participants have an easier time measuring across one single spectrum than having to define their opinion across two spectrums.
Net scale recommendations
4-point scales are good. Present the scales horizontally, using words when possible, with the most negative value to the left-most side of the scale. Keep the values unipolar, measuring across a single spectrum.
However, all that said, still take into account your audience, because cultural behaviors still very much apply.