Processing and Analysing Data
Chapter
Syllabus topic 5, "Research Methods"
Pages 399 to 402 of 543
In one line
Collected data is not results: it has to be checked, coded, classified and tabulated before it means anything, and each of those four steps is a place where a study is quietly ruined.
In the wording a student can write in an exam: processing of data comprises editing, coding, classification and tabulation, by which raw responses are made complete, consistent and comparable and are arranged so that they can be counted; analysis is the examination of the tabulated data to establish magnitudes, distributions and relationships, using measures of central tendency and dispersion and, where appropriate, tests of association, in order to answer the research question and test the hypothesis.
Editing
What it is. Going through every completed instrument before anything is counted.
What to look for. Blanks; answers that contradict each other, such as a respondent who has never heard of legal aid but describes going to the office; answers outside the permitted range; illegible entries; and instruments completed in circumstances the investigator noted as unusual.
What may be done. Correct an obvious clerical slip; return to the respondent where possible; and mark an entry as missing.
What may never be done. Fill in an answer the researcher thinks the respondent would have given. That is fabrication, and it is invisible in the result.
Do it in the field, daily. An error found on day three can be repaired by revisiting; the same error found at the end cannot.
Coding
What it is. Assigning a symbol, usually a number, to each response so that it can be counted.
Closed questions code themselves, because the options were fixed.
Open questions have to be coded after collection, and this is the step that takes the time. Read a sample of the answers, build a set of categories from what is actually there, and then apply them to all the answers.
Three rules. The categories must be exhaustive, so everything can be placed. They must be mutually exclusive, so nothing can go in two. And there must be an other, which should be small; if other is a quarter of the responses, the categories are wrong and must be rebuilt.
Keep a codebook. A written record of every code and what it means, which is what allows another person to check the work and what allows the researcher to remember in month four what they decided in month one.
And code blind where you can. A researcher who knows which settlement an answer came from may code ambiguous answers in the direction they expect.
Classification
What it is. Grouping the coded data into classes for analysis: by attribute, such as sex or occupation, or by magnitude, such as age groups or ranges of expenditure.
Processing and Analysing Data
The choice of class intervals matters and is not neutral. Ages grouped as under 25, 25 to 40, 41 to 60 and over 60 produce a different picture from under 30 and 30 and over, and a researcher can, without dishonesty, choose intervals that flatter a conclusion.
So decide the intervals from the data's own shape and state them, and where a different grouping would change the picture, say so.
Tabulation
What it is. Arranging the classified data in rows and columns.
A simple table presents one variable: how many respondents had heard of free legal services.
A cross table presents two: awareness by sex, or awareness by settlement.
Every table needs: a number, a title saying exactly what it shows, the source, the total, and a note of missing cases.
And every table needs a sentence. A table alone leaves the reader to do the work; the sentence says what the table shows, and a second sentence says what it does not.
Analysis, at the level a law student needs
Measures of central tendency. The mean, or arithmetic average, which is pulled by extreme values; the median, the middle value, which is not and is usually the better measure for money and for days; and the mode, the commonest value, which is the only one available for nominal data.
A worked point. If seven claimants spent 200, 250, 300, 300, 350, 400 and 4,000 rupees, the mean is about 828 and the median is 300. The mean describes nobody. Report the median, and report the outlier separately, because the person who spent 4,000 is itself a finding.
Measures of dispersion. The range, and the standard deviation, which says how spread out the values are. Two districts with the same average waiting time and very different spreads are not the same place.
Proportions and percentages. Give the base every time. Sixty per cent of what; a percentage without its denominator is not a figure.
And do not compute a percentage on a small base. Three of seven is three of seven, not 42.9 per cent.
Association. Whether two things vary together: whether awareness differs by sex, whether expenditure differs by distance. A cross table shows it; a formal test of significance establishes whether it could have arisen by chance. A student who cannot state the assumptions of a test should not use it, and should present the cross table and describe the pattern instead, which is honest and adequate.
And the standing warning. Association is not causation. Two things varying together may both be produced by a third, chapter 1050.
Presenting it
Tables for exact values, charts for shape. A bar chart for categories; a line for change over time. Nothing more elaborate is needed.
Processing and Analysing Data
No chart that a table would say better, and no chart with fewer than about four values.
Label everything, including the base.
And put the raw instruments in an appendix if they are few, or say where they are kept, because the reader's only route to checking the work is through them.
A worked example
The awareness study of chapter 970, from collection to a finding.
Editing. Eighty-four completed schedules read the same evening each day. Three have question 7 answered where question 5 was no, which the filter should have skipped; those three are checked against the investigator's notes and two are corrected, one is marked missing.
Coding. Question 6, where did you hear about it, is open. Reading forty answers produces five categories: from a neighbour or relative; from a camp or clinic; from a lawyer; from television or radio; and from a government office. Other is four responses, which is acceptable. Codes are written into a codebook.
Classification. By sex, by settlement, and by whether the household had a legal problem in the last two years.
Tabulation. Table 1, awareness by settlement. Table 2, awareness by sex. Table 3, source of awareness among those aware.
Analysis. The proportion aware, with its base. The median number of journeys among those who had gone. And the cross table showing whether awareness differed by sex, described in a sentence, without a significance test the researcher cannot justify.
The finding. Stated as a proportion with its base and its limits, chapter 1280.
Quick revision
Processing is four steps: editing, done daily in the field, correcting clerical slips and marking missing but never inventing an answer; coding, with categories built from the data, exhaustive, mutually exclusive, with a small other, recorded in a codebook, and done blind where possible; classification, whose class intervals are not neutral and must be stated; and tabulation, every table carrying a number, a title, the source, the total, missing cases and a sentence saying what it shows.
Analysis: mean, pulled by extremes; median, better for money and days; mode, the only measure for nominal data; range and standard deviation for spread; percentages always with their base and never on a small one; and association shown by a cross table, with a significance test only if its assumptions can be stated.
Association is not causation.
Test yourself
1. Name the four steps of processing and say what may never be done in the first. Editing, coding, classification and tabulation. In editing, a researcher may correct an obvious clerical slip, return to the respondent, or mark an entry as missing, but may never supply an answer the respondent did not give, which is fabrication and is invisible in the result.
Processing and Analysing Data
2. Give the three rules for building codes for open questions. The categories must be exhaustive, so every answer can be placed; mutually exclusive, so no answer belongs in two; and there must be an other category which stays small, since a large other shows that the categories are wrong and must be rebuilt.
3. When should the median be preferred to the mean, and give an example. Whenever extreme values are present, which is usual for money, days and journeys. If seven claimants spent 200, 250, 300, 300, 350, 400 and 4,000 rupees, the mean of about 828 describes nobody while the median of 300 describes the typical case, and the outlier should be reported separately as a finding in itself.
4. What should a student do instead of a test of significance they cannot justify? Present the cross table and describe the pattern in words. That is honest and adequate, whereas a test whose assumptions the researcher cannot state produces a number that looks authoritative and cannot be defended.
The rest of this subject
These notes are cut from the University's printed syllabus. Open the syllabus itself, or the past papers, for the same subject.