Building Better Surveys – Effective Scales

A question I have been asked from time-to-time is, “How many choices should I give respondents when I’m asking them how much they agree or disagree with something?” As in . . .

Which is correct? Does it matter?

Yes, it matters.

Use an odd-numbered scale and, if you expect your results to be fully distributed from the positive to the negative end of the continuum, use a 5-point scale. If you expect your results to be either positively or negatively skewed, use a 7-point scale.

Odd or even-numbered scale

There are two purposes of a survey item. 1) Learn what people think. 2) Distribute respondents along a continuum in order to compare them individually or in groups to each other. (As in, “Will 25 year-old buyers have as much interest in my product as 20-year old buyers?”)

With an even number of alternatives, the survey developer is telling people, “You have to agree or disagree with this item. You can’t be neutral or undecided”. Maybe psychologists sometimes need to present respondents with forced-choices, but most of us are not psychologists. Why some survey developers think they should control respondents with even-numbered scales is something I have never understood and as a respondent, resent.

Moreover, even numbered scales never, never, never gather more information than odd-numbered scales. In fact, they impoverish our data sets. Ah, you say, but I want to know how those people who are neutral would respond if they were forced to choose. Why? What difference will it make? Wouldn’t you rather know how many of those people who are near the middle of the scale really can’t tell the difference between the choices, or haven’t made up their minds? Besides, I can tell you within a couple of percentage points how those neutrals will respond if the middle choice is taken away from them. They will respond the same as the other respondents in the survey. Try it some time. Ask the same question in two different parts of your survey once with an odd-numbered scale and once with an even-numbered scale and you can prove this point to yourself.

Scale length

The human mind can embrace 7-plus-or-minus-2 data bits at any given time. Those of you old enough to remember life before cell phones and the ubiquitous use of area codes remember that phone numbers could be easy to remember or peculiarly difficult.

Obviously those numbers used more often were easier to remember, but what else made some numbers easy to remember and others difficult? Patterns! Patterns reduced the number of data bits your mind had to retain. The number 222-4466 is much easier to remember than 497-5031. In the first number, the pattern is apparent. In the second number you have to retain all seven numbers as separate bits.

The application to surveys of the 7-plus-or-minus-2 rule is this. More than seven levels of “agreement” cannot be considered in one instant which makes scales with more than 7 levels difficult to respond to. They are fatiguing. They deprive the mind of the chance to embrace the scale as one and accurately make a selection from an array of balanced alternatives. What the mind does with a ten-point scale is first, split it into a positive and a negative half — an especially frustrating decision if the respondent is in the middle.

In sum, using scales that provide a full array of alternatives within an appropriate length permits respondents to complete your survey in a shorter amount of time and with greater accuracy.

• 4-point scales — an impoverished and inaccurate data set
• 5-point scales — accurate; MEETS YOUR NEEDS 90% OF THE TIME
• 7-point scales — accurate; necessary for skewed data
• 10-point scales — You deserve all of the “abandons” you get

Share This: