Scaling and Measurement

Now notice what the sum is. An ordinal figure treated as though it were interval, which is what everybody does and what nobody should do carelessly. Report the median and the distribution as well as any mean.

Other scaling techniques, named

Thurstone equal appearing intervals, where judges rate statements in advance so that the scale points are approximately equidistant. Rigorous and expensive.

Guttman scalogram, where items are cumulative, so agreeing with a strong item implies agreeing with the weaker ones.

Semantic differential, where a concept is rated between pairs of opposites such as fair and unfair, fast and slow.

Rating scale, a single item marked on a range.

And ranking, where the respondent orders alternatives rather than rating each.

For a student, the Likert scale and simple ranking are enough, and using a technique whose assumptions the researcher cannot state is worse than using a simple one well.

The dangers

Central tendency. Respondents avoid the extremes, so everything clusters in the middle.

Acquiescence. Respondents agree with whatever is put to them, particularly to a researcher perceived as educated or official. Reverse-worded items are the defence.

Social desirability. Respondents give the answer they think is expected, which in a study about awareness produces over-reporting of awareness.

Order effects. The answer to a later item is shaped by the earlier ones.

Untranslated concepts. A five point agreement scale administered orally in a language where the intermediate points have no natural expression produces three usable points, not five, and the researcher should say so.

And false precision, which is the worst. A mean of 3.47 on an ordinal scale from forty respondents chosen by convenience is a number with two decimal places and no content.

Measuring what can be counted instead

A practical recommendation worth more than the whole of scaling theory for most student work.

Wherever a fact can be counted, count it rather than scaling an opinion about it. Not how satisfied the respondent was with the number of visits, but how many visits they made. Not whether they found it expensive, but what they spent.

Counted facts are ratio data, permit real arithmetic, are far less exposed to the dangers above, and are much harder to dispute.

Chapter 910's worked example is built this way entirely, and it produces a finding an official can act on, which a satisfaction score does not.

Quick revision

Four levels: nominal, labels only, count them; ordinal, ordered with unknown gaps, use the median; interval, equal gaps and no true zero; ratio, equal gaps and a true zero, so all arithmetic is permitted.

The Likert scale: statements not questions; a constant symmetrical response set, usually five points; scored 5 to 1; some statements reverse worded and reverse scored; and the scores summed. The sum is ordinal treated as interval, so report the distribution and the median as well.

Other techniques: Thurstone, Guttman, semantic differential, rating and ranking.

Dangers: central tendency; acquiescence, defeated by reverse wording; social desirability; order effects; concepts that do not translate; and false precision.

The recommendation: count facts rather than scaling opinions about them, because counted facts are ratio data, permit real arithmetic and are harder to dispute.

Test yourself

1. Name the four levels of measurement and say what arithmetic each permits. Nominal, which permits counting only, since the numbers are labels; ordinal, which permits ordering and the median, since the gaps between points are unknown; interval, which has equal gaps but no true zero; and ratio, which has equal gaps and a true zero and permits all arithmetic.

2. Set out how a Likert scale is built. Write statements rather than questions; fix a constant symmetrical response set, usually five points from strongly agree to strongly disagree; score them, typically 5 to 1; reverse the wording and the scoring of some statements so that agreement is not always the favourable answer; and sum the scores across statements to give one figure per respondent.

3. Why are reverse worded items included? To defeat acquiescence, the common tendency of respondents to agree with whatever is put to them, especially where the researcher is perceived as educated or official. If some statements are worded so that agreement is unfavourable, a respondent who agrees with everything can be detected.

4. What is the single most useful practical recommendation in this chapter, and why? Count facts wherever possible rather than scaling opinions about them: how many visits, how many rupees, how many days, rather than how satisfied the respondent was. Counted facts are ratio data, permit genuine arithmetic, are far less exposed to acquiescence and social desirability, and produce findings an official can act on.