Walkthrough, and How It Differs From an Inspection

• teaching: a new team member learns the fee module fastest by having its author walk through it;
• early drafts: an author who wants reactions before finishing a design;
• consensus: a team choosing between two approaches, where discussing alternatives is the point;
• stakeholders who are not programmers: an analyst walking the exam cell through the requirements, which is validation in the sense of Chapter Twenty-Six;
• work of moderate risk, where the cost of a full inspection is not justified.

Roles in a walkthrough

A walkthrough needs fewer roles than an inspection, and they are less fixed.

Leading a code walkthrough: tracing test cases by hand

A code walkthrough can drift into a line-by-line reading that nobody remembers. A more effective way for the author to lead is to trace test cases through the code by hand: participants bring a few test cases, each with its expected result, and the author "executes" the code on paper for each one, saying aloud what every line does to every variable. The participants follow, and a wrong value shows up at the line where it happens.

Here is the author of ExamReg's fee function leading such a walkthrough. The fee rules are the book's fixed ones: a form fee of Rs 800, Rs 150 for each backlog paper, the late fee of Chapter One, on what software testing is, forms more than 15 days late refused, a negative number of days refused as invalid input, and a concession that waives the form fee only, as the Scrum team of Chapter Seventeen established.

FORM_FEE, BACKLOG_FEE = 800, 150

def total_fee(days_late, backlog_papers, concession):
    if days_late < 0 or days_late > 15:
        raise ValueError("form not accepted")
    fee = FORM_FEE + BACKLOG_FEE * backlog_papers
    if concession:
        fee = 0                                  # the concession
    if days_late <= 0:
        late = 0
    elif days_late <= 7:
        late = 100
    else:
        late = 500
    return fee + late

print("author's own check before the walkthrough, TC1:", total_fee(0, 0, False))

The author ran one case before the meeting, and it passed:

author's own check before the walkthrough, TC1: 800

The participants bring three test cases:

The author traces each one aloud, and the scribe writes the trace on the board.

TC3 stops the room. At the concession line the trace sets fee to 0, wiping out the Rs 300 for two backlog papers along with the form fee. A participant asks what the concession is supposed to waive; the answer, the form fee only, is the fix: subtract the form fee instead of zeroing the total. The failing case was never run: the defect was found on paper, at the line that caused it. After the walkthrough the author makes the change, and the three test cases are run on both versions to confirm what the trace showed.

FORM_FEE, BACKLOG_FEE = 800, 150

def total_fee(days_late, backlog_papers, concession):         # as walked through
    if days_late < 0 or days_late > 15:
        raise ValueError("form not accepted")
    fee = FORM_FEE + BACKLOG_FEE * backlog_papers
    if concession:
        fee = 0                                                 # the concession
    if days_late <= 0:
        late = 0
    elif days_late <= 7:
        late = 100
    else:
        late = 500
    return fee + late

def total_fee_fixed(days_late, backlog_papers, concession):   # after the walkthrough
    if days_late < 0 or days_late > 15:
        raise ValueError("form not accepted")
    fee = FORM_FEE + BACKLOG_FEE * backlog_papers
    if concession:
        fee -= FORM_FEE                                         # waives the form fee only
    if days_late <= 0:
        late = 0
    elif days_late <= 7:
        late = 100
    else:
        late = 500
    return fee + late

cases = [("TC1", (0, 0, False), 800), ("TC2", (10, 1, False), 1450), ("TC3", (3, 2, True), 400)]
for version in (total_fee, total_fee_fixed):
    print(version.__name__)
    for name, args, expected in cases:
        actual = version(*args)
        mark = "" if actual == expected else "  <- the walkthrough's finding"
        print(f"  {name} {args}: expected {expected}, got {actual}{mark}")

total_fee
  TC1 (0, 0, False): expected 800, got 800
  TC2 (10, 1, False): expected 1450, got 1450
  TC3 (3, 2, True): expected 400, got 100  <- the walkthrough's finding
total_fee_fixed
  TC1 (0, 0, False): expected 800, got 800
  TC2 (10, 1, False): expected 1450, got 1450
  TC3 (3, 2, True): expected 400, got 400

The run agrees with the hand trace line for line: the version walked through returns 100 for TC3, and the fixed one 400. Notice also what this walkthrough did and did not do. It found a real defect and it taught the room how the concession works. But it did not use a checklist, did not classify the defect, did not measure anything, and nobody is assigned to verify the fix except the author. Those are exactly the things an inspection adds.

Walkthrough against inspection

Fagan, who defined the inspection, compared the two in his 1976 paper. His starting point was that walk-throughs "are practiced in many different ways in different places, with varying regularity and thoroughness. This inconsistency causes the results of walk-throughs to vary widely and to be nonrepeatable. Inspections, however, having an established process and a formal procedure, tend to vary less and produce more repeatable results."

His Table 4 sets the two processes side by side:

His note under the table makes the key point: "Note the separation of objectives in the inspection process." An inspection meeting does one thing; a walk-through meeting does three at once.

His Table 5 compares their key properties:

And in one comparison of equivalent pieces of an operating system, the inspected sample had "38 percent less errors" in later testing than the walk-through sample, as Chapter Twenty-Nine, on inspection, reported.

Putting Fagan's comparison together with the modern syllabus gives the table a 5-mark answer needs.

Choosing between them

The choice follows from the ISTQB syllabus's factors from Chapter Twenty-Eight, on reviews: the criticality of the work product, the objectives, and the need for an audit trail. On ExamReg, the new help pages would go to a walkthrough with the exam cell, because the aim is to check that students will understand them; the fee and payment code would go to an inspection, because a missed error there costs students money. Many teams use both on the same work product: a walkthrough while the design is taking shape, and an inspection once it is complete.

What it does not mean

A walkthrough is not a weak inspection. It has different objectives, and for teaching and consensus it does better than an inspection would.

A walkthrough does not mean no preparation is ever done. Preparation is optional, not forbidden, and participants who prepare find more.

Tracing by hand is not the same as testing. It checks the code against a few expected results in people's heads; the cases still need to be run, as they were after this walkthrough.

Being led by the author is not a flaw. It is what makes a walkthrough good at teaching; it is also why a walkthrough should not be the only review of critical work.

Quick revision

• Walkthrough: an author leads members of the review through a work product; participants ask questions and comment (ISO/IEC 20246). Led by the author; preparation optional (ISTQB).
• Objectives (ISTQB v4.0.1): evaluate quality and build confidence, educate reviewers, gain consensus, generate ideas, motivate authors, detect anomalies.
• Tracing test cases by hand is an effective way to lead a code walkthrough; the ExamReg trace found a concession that wiped out backlog fees.
• Fagan's Table 4: the inspection has five operations with separate objectives; the walk-through has preparation and one meeting with mixed objectives.
• Fagan's Table 5: inspections have moderator training, defined roles, checklists, error-type data, follow-up, feedback and process analysis; walk-throughs do not.
• Walk-through results "vary widely" and are "nonrepeatable"; inspections are more repeatable; the inspected sample had 38 per cent fewer errors.
• Use walkthroughs for teaching, drafts, consensus and stakeholders; inspections for critical work.

Test yourself

1. What is a walkthrough, and who leads it? A review in which the author leads members of the review through a work product while the participants ask questions and comment on possible issues. The author leads it, presenting the work in the order they choose.

2. What are the objectives of a walkthrough? Evaluating quality and building confidence in the work product, educating reviewers, gaining consensus, generating new ideas, motivating and enabling authors to improve, and detecting anomalies.

3. Differentiate between a walkthrough and an inspection. A walkthrough is led by the author, an inspection by a trained moderator who is never the author. A walkthrough aims at understanding and consensus as well as problems, and may discuss solutions; an inspection's meeting aims only at finding errors. A walkthrough is a meeting with optional preparation; an inspection has overview, preparation, inspection, rework and follow-up. An inspection uses defined roles, checklists, error-type data and metrics, and verifies fixes; a walkthrough usually does not, so its results are less repeatable.

4. How can an author lead a code walkthrough effectively? By tracing test cases through the code by hand: participants bring test cases with expected results, and the author executes the code on paper, saying what each line does to each variable, so that a wrong value appears at the line that causes it.

5. When would you choose a walkthrough rather than an inspection? When the aim is teaching, gathering reactions to an early draft, reaching consensus, or checking a work product with non-technical stakeholders, and the work is not so critical that the cost of a full inspection is justified.