What an Examination Is For
Chapter Thirty-Seven
Syllabus topic 3, "Examination reforms"
Pages 134 to 136 of 543
In one line
An examination is trying to do four different things at once, and they pull against each other, which is why every reform improves one of them and is attacked for damaging another.
In the wording a student can write in an exam: an examination in a professional course serves four purposes, namely certification, that the holder of the degree has reached a standard; selection, that candidates can be ranked for further opportunity; direction, that students are told by the marks what is worth learning; and feedback, that the student and the teacher learn what has not been understood; these purposes conflict, since an examination designed to certify reliably is not the one that gives the best feedback, and every proposal for reform must be judged by which of the four it improves and which it sacrifices.
The four purposes
Certification. The examination says to the world that this person has reached a standard. For a law degree this purpose is unusually heavy, because the degree opens a profession under section 24 of the Advocates Act 1961 and the public will rely on it.
Selection. Marks are used to rank: for postgraduate admission, for employment and for judicial service. This purpose requires the examination to spread candidates out, which is a different requirement from certifying a floor.
Direction. This is the purpose that is least discussed and most powerful. Students learn what is examined. An examination is therefore the strongest teaching instrument in a course, and it is exercising that power whether or not anybody intends it.
Feedback. The student should learn what they got wrong while there is still time to fix it, and the teacher should learn what the class did not understand while there is still time to teach it. A terminal examination cannot do this at all.
How the four conflict
Certification against feedback. A reliable certifying examination is supervised, terminal and secret. A useful feedback exercise is frequent, low stakes and returned with comments. The same instrument cannot be both.
Selection against certification. Selection needs the marks to spread; certification only needs a line. An examination made harder to spread the top of the class fails candidates who had reached the standard.
Direction against everything. Whatever is easiest to examine reliably tends to be what gets examined, and then it becomes what gets taught. Skills are hard to examine reliably, so they get examined less, so they get taught less. Chapter 390 is that argument applied to the clinical papers.
And feedback against workload. Frequent assessment with real comments is the most expensive thing a teacher does, and where the staffing does not allow it the frequency survives and the comments do not, which is how continuous assessment becomes continuous testing.
What an Examination Is For
What follows for reform
Every proposal in the rest of this part should be tested against the four. The semester system improves feedback and direction and complicates certification. Internal assessment improves feedback and direction and weakens certification, because internal marks are not comparable across colleges. Objective questions improve reliability and damage direction, because they reward recognition. Open book examinations improve direction and are harder to set. The All India Bar Examination improves certification and does nothing for feedback.
None of these is a defect of the reform. They are the trade-offs, and an answer that names the trade-off is doing what the topic asks.
And the reform that fails is the one that changes the paper without changing the direction. If the examination still rewards reproduction, teaching that produces reasoning is punished, and rational students and rational teachers will stop.
The Indian specific
The certification purpose carries more weight here than in most systems, because the profession is entered on the strength of the degree, and until 2010 nothing checked the degree afterwards.
That is exactly why the All India Bar Examination matters. Once an external examination exists, the certifying burden on the university examination is lighter, and the university examination is freer to do the other three things. That argument is worth making in an essay and is taken up at chapter 420.
And the direction purpose explains the guide. Where the examination can be passed from a printed summary, the summary is what students read, and no reform of teaching survives that. Chapter 350 takes the criticism apart.
A worked example
One course, one topic: the law of frustration. Four instruments.
A three hour terminal paper asking the candidate to discuss the doctrine of frustration certifies well, spreads candidates moderately, directs students to prepare expositions, and gives no feedback because it is returned as a number after the course is over.
A problem question on the same paper certifies as well, spreads better, directs students to practise application, and still gives no feedback.
A two page internal assignment marked and returned in a fortnight certifies poorly, since it is unsupervised and not comparable across colleges; spreads unreliably; directs students to write; and gives real feedback.
A five minute viva certifies moderately, spreads well in the hands of a good examiner, directs students to be able to speak about the subject, and gives immediate feedback.
No single one is best, and a course that uses all four is doing what the four purposes require. That is the whole architecture of rule 12 and of MU's 75:25 pattern, taken up at chapter 360.
Quick revision
Four purposes: certification, that a standard has been reached; selection, that candidates can be ranked; direction, that students are told what is worth learning; and feedback, that both student and teacher learn what was not understood.
What an Examination Is For
They conflict: certification wants secrecy and finality, feedback wants frequency and comment; selection wants spread, certification wants a line; direction is exercised whether intended or not; and feedback is the most expensive thing a teacher does.
In law the certification purpose is unusually heavy, because the degree opens a profession under section 24 of the Advocates Act 1961.
Test every reform against the four, name what it improves and what it sacrifices, and remember that a reform which leaves the direction unchanged changes nothing.
Test yourself
1. Name the four purposes of an examination and give one sentence on each. Certification, that the holder has reached a standard, which in law opens a profession; selection, that candidates can be ranked for further opportunity; direction, that the marks tell students what is worth learning; and feedback, that student and teacher discover what has not been understood in time to act.
2. Show how two of the purposes conflict. Certification requires an examination that is supervised, terminal and secret so that the result can be relied on; feedback requires assessment that is frequent, low stakes and returned with comments. One instrument cannot do both, so a course needs several.
3. Why is the direction purpose the most powerful? Because students allocate effort to what carries marks, so the examination decides what is actually learned and therefore what is worth teaching. It exercises that power whether or not anybody intends it, which is why what is hard to examine tends to stop being taught.
4. How does the All India Bar Examination change the burden on the university examination? By supplying an external check on certification, it lightens the certifying burden the university paper had to carry alone, which leaves the university examination freer to serve selection, direction and feedback.
The rest of this subject
These notes are cut from the University's printed syllabus. Open the syllabus itself, or the past papers, for the same subject.