What Examination Reform Actually Requires
Chapter Forty-Eight
Syllabus topic 3, "Examination reforms"
Pages 171 to 173 of 543
In one line
A reform that changes only the question paper changes nothing, because the paper is the last link in a chain that runs from what a college has, through what it can teach, to what it can assess.
In the wording a student can write in an exam: effective examination reform requires four conditions to be met together, namely that the assessment measures what the course is trying to produce, that the teaching is capable of producing it, that the institution has the staffing and facilities the teaching requires, and that the regulator inspects and enforces the assessment as it does the infrastructure; where any one of the four is missing the reform is nominal, which is why the same defects have been reported in 1958, in 2002 and in 2020.
The four conditions
One: the assessment must measure what is wanted. If the objective is a graduate who can draft, the assessment must require drafting. Chapter 390 shows what happens when it does not.
Two: the teaching must be capable of producing it. An assessment that requires drafting, in a course taught entirely by lecture, produces failure and resentment rather than skill. Reform of assessment without reform of method transfers the burden to students.
Three: the institution must have what the teaching requires. Rule 11 and Schedule III, chapter 170: library, full time faculty, rooms, clinic. A college with two full time teachers cannot mark thirty drafting exercises per student, and telling it to is not a reform but an instruction to pretend.
Four: the regulator must enforce the assessment. Rule 22 already makes the feasibility of standard clinical education an object of inspection, chapter 180. What has not happened is its use against colleges that examine practical papers as theory.
All four, together. That is the whole argument, and each of the three preceding parts of Module I supplies one of them.
Why the reforms so far have half worked
The semester system, chapter 360, met condition one partly and none of the others, so in many colleges it multiplied the annual examination.
Continuous internal assessment, chapter 360, met condition one well and condition three not at all, so its marks are widely given for attendance.
The clinical papers, chapter 390, met conditions one and two in the Schedule's design and conditions three and four almost nowhere, so they are examined as theory.
Objective questions, chapter 370, met the reliability problem and worsened the direction problem, which is a trade-off rather than a failure.
And the All India Bar Examination, chapter 420, met condition four in a new way, from outside, which is why it is the most consequential of them: it is the first reform that did not depend on the college's own willingness.
What Examination Reform Actually Requires
That pattern is the most useful thing a candidate can say on this topic, because it explains the outcomes rather than listing them.
What a college can do without money
Change what the internal marks are for. From attendance to a moot, a drafting exercise and a research exercise. This costs teacher time and a decision, and it makes every method in topic 2 possible, because students prepare for what is marked.
Reduce the choice on the paper. Four from five rather than four from seven improves sampling and ends question spotting, and it costs nothing. Chapter 350.
Set one problem question in every paper. It cannot be answered from a guide, and it directs preparation towards application.
Permit the bare Act. An open book component makes trivial questions impossible and devalues the summary.
Return the scripts. Feedback is one of the four purposes of an examination, chapter 340, and returning marked work is the cheapest way to serve it.
And publish the marking scheme in advance, which improves fairness, reduces revaluation applications and tells students what is wanted.
What only the regulator or the University can do
Prescribe the practical assessment centrally while leaving the marking local, which answers the comparability objection to clinical assessment.
Ask for the exercises at inspection. Rule 22 already permits it.
Define standards. This is the 184th Report's unenacted proposal, and it is the condition that makes enforcement possible at all, because a regulator cannot enforce an undefined standard.
And resolve the divided jurisdiction, chapter 110, so that two bodies cannot set inconsistent requirements.
A worked example
A college decides to reform its examinations and has no money.
It moves its twenty-five internal marks to three assessed pieces: a moot at eight, a drafting exercise at eight and a research exercise at nine, with the scheme published in week one.
Within a semester the teaching changes by itself. Students ask to be taught to moot, because it carries marks. The teacher who wanted to circulate judgments now has a reason. The library is used, because the research exercise requires it.
Within a year the students are different, and the University paper has not changed at all.
Now compare a college that instead redesigns its University paper into objective questions and keeps the internal marks on attendance. Marking became reliable and nothing else moved, because condition two and condition three were untouched.
Same effort, opposite results, and the difference is which of the four conditions was addressed. That comparison is the answer to the topic in a single example.
Quick revision
Four conditions, together: the assessment must measure what is wanted; the teaching must be able to produce it; the institution must have what the teaching requires under rule 11 and Schedule III; and the regulator must enforce the assessment under rule 22.
What Examination Reform Actually Requires
Why past reforms half worked: the semester system and internal assessment met the design condition and not the resource condition; the clinical papers met design and method and not resources or enforcement; objective questions traded direction for reliability; and the All India Bar Examination worked because it did not depend on the college's willingness.
Free reforms a college can make today: move the internal marks to assessed work; reduce the choice of questions; set one problem question per paper; permit the bare Act; return the scripts; publish the scheme in advance.
Only the regulator can: prescribe practical assessment centrally, ask for the exercises at inspection, define standards, and resolve the divided jurisdiction.
Test yourself
1. State the four conditions of effective examination reform. That the assessment measures what the course is trying to produce; that the teaching is capable of producing it; that the institution has the staffing and facilities the teaching requires; and that the regulator inspects and enforces the assessment as it does the infrastructure.
2. Explain, using two examples, why past reforms have only half worked. Continuous internal assessment met the design condition but not the resource condition, so its marks are widely awarded for attendance; and the clinical papers met the design and method conditions in Schedule II but neither the resource nor the enforcement condition, so they are examined as written theory.
3. Why is the All India Bar Examination the most consequential reform so far? Because it met the enforcement condition from outside the institution. Every other reform depended on the college's own willingness and resources; this one tests the graduate on a common instrument whatever the college did.
4. Name four reforms a college with no money can make immediately. Move the internal marks from attendance to assessed work such as a moot, a drafting exercise and a research exercise; reduce the choice of questions on the paper; set one problem question in every paper; and return the marked scripts to students.
The rest of this subject
These notes are cut from the University's printed syllabus. Open the syllabus itself, or the past papers, for the same subject.