The Defect Life Cycle

Two rules come with the states. "A problem cannot be in more than one state at any point in time", and since states change, "one must always specify a time when measuring the problem status".

ExamReg's life cycle

ExamReg's team uses nine states, which fill in Florac's generic ones and match the ISTQB syllabus's typical statuses:

Figure 74.1 ExamReg's defect life cycle: eleven moves; shaded states are closed

The moves between them, and who may make each:

Two moves are missing on purpose. There is no move from Assigned or Fixed straight to Closed, so no defect can be closed without a tester's check of the fix. And the developer cannot move a defect to Verified: whoever fixed it does not decide that it is fixed. Triage is the decision taken on each new report, often by a defect review board; Chapter Seventy-Seven, on tracking defects to closure, describes it.

Worked example: a tracker that enforces the rules

A defect tracker is a state machine, the kind Chapter Fifty-Seven tested, and its main job is to refuse the moves the life cycle forbids. Bugzilla's documentation describes its own workflow in exactly those terms: its configuration page shows every status twice, as the starting status and as the target, and "If the checkbox is checked, then the transition from the left to the top status is legal; if it's unchecked, that transition is forbidden."

The program writes ExamReg's life cycle as such a table, with the role allowed to make each move. It walks DR-205, a rounding defect in the fee page, through a life that includes a failed fix, and then tries three moves the rules forbid.

# ExamReg's defect life cycle: (from, to) -> the role allowed to make the move
MOVES = {("new", "assigned"): "triage",     ("new", "rejected"): "triage",
         ("new", "duplicate"): "triage",    ("new", "deferred"): "triage",
         ("deferred", "assigned"): "triage",
         ("assigned", "fixed"): "developer",
         ("fixed", "verified"): "tester",   ("fixed", "reopened"): "tester",
         ("verified", "closed"): "tester",  ("closed", "reopened"): "tester",
         ("reopened", "assigned"): "triage"}
# each state against Florac's generic statuses (SEI 1992): Open (recognized, evaluated, resolved), Closed
FLORAC = {"new": "open: recognized", "assigned": "open: evaluated", "deferred": "open: evaluated",
          "reopened": "open: evaluated", "fixed": "open: resolved", "verified": "open: resolved",
          "closed": "closed", "rejected": "closed", "duplicate": "closed"}

class Defect:
    def __init__(self, ident):
        self.ident, self.state, self.history = ident, "new", ["new"]

    def move(self, to, role):
        allowed = MOVES.get((self.state, to))
        if allowed is None:
            raise ValueError(f"{self.state} -> {to} is not a move in the life cycle")
        if allowed != role:
            raise PermissionError(f"{self.state} -> {to} is for the {allowed}, not the {role}")
        self.state = to
        self.history.append(to)

d = Defect("DR-205")                       # the fee page rounds a concession fee wrongly
for to, role in [("assigned", "triage"), ("fixed", "developer"), ("reopened", "tester"),
                 ("assigned", "triage"), ("fixed", "developer"), ("verified", "tester"),
                 ("closed", "tester")]:
    d.move(to, role)
print(d.ident, " -> ".join(d.history))
print("   in Florac's terms:", " -> ".join(FLORAC[s] for s in d.history))

for state, to, role in [("assigned", "closed", "developer"), ("fixed", "verified", "developer"),
                        ("duplicate", "assigned", "triage")]:
    e = Defect("DR-3xx")
    e.state = state
    try:
        e.move(to, role)
    except (ValueError, PermissionError) as refusal:
        print(f"refused ({type(refusal).__name__}): {refusal}")

DR-205 new -> assigned -> fixed -> reopened -> assigned -> fixed -> verified -> closed
   in Florac's terms: open: recognized -> open: evaluated -> open: resolved -> open: evaluated -> open: evaluated -> open: resolved -> open: resolved -> closed
refused (ValueError): assigned -> closed is not a move in the life cycle
refused (PermissionError): fixed -> verified is for the tester, not the developer
refused (ValueError): duplicate -> assigned is not a move in the life cycle

DR-205's history is typical of a real defect's: the first fix failed its confirmation test, the report went back to triage and a second fix passed. In Florac's terms it was open for its whole life until the last step, moving back from resolved to evaluated when the first fix failed, which is why counting defects by status always needs a date.

The three refusals are the tracker doing its job. A developer cannot close an assigned defect, because Closed is reachable only through Verified. A developer cannot verify a fix, because that move belongs to a tester. And a duplicate cannot be assigned, because its work belongs to the original report. The two kinds of refusal differ: the first and third moves do not exist in the life cycle at all, while the second exists but belongs to someone else.

Reading the life cycle for testing

Because the life cycle is a state machine, it can be tested like one: every valid move made, every invalid one attempted, as Chapter Fifty-Seven, on state transition testing, did for ExamReg's login. And its states are measurement points. Florac notes that "Measurements of the numbers of problem reports in the various problem states, shown with respect to time, will inform the manager of the rate the project is progressing towards a goal"; how many are open, how long they have been open, and how many were reopened are the tracking and defect metrics of Chapters Seventy-Seven and Seventy-Nine.

What it does not mean

There is no single standard life cycle. Tools and teams name their states differently; the ISTQB syllabus lists typical statuses, and Florac gives only Open and Closed with three generic sub-states.

Fixed is not closed. A fix is closed only after a tester has confirmed it, and the regression tests show it broke nothing.

Rejected is not ignored. A rejected report records why it is not a defect, and its reporter can challenge the decision.

Deferred is not forgotten. A deferred defect is still open, and it is counted as open when the release ships.

Quick revision

• Workflow from discovery to closure: log, analyse and classify, decide on a response, close (ISTQB); typical statuses: open, deferred, duplicate, waiting to be fixed, awaiting confirmation testing, re-opened, closed, rejected.
• Florac (SEI 1992): Open (recognized, evaluated, resolved) and Closed; one state at a time; state counts need a date; an invalid report may be closed at recognition.
• ExamReg: New, Assigned, Deferred, Reopened, Fixed, Verified, Closed, Rejected, Duplicate; eleven moves, each with its role.
• No move from Assigned or Fixed to Closed; the developer cannot verify.
• Worked example: DR-205 went new, assigned, fixed, reopened, assigned, fixed, verified, closed; three forbidden moves refused.
• The life cycle is a state machine: test it like one, and measure the counts in each state over time.

Test yourself

1. What is the defect life cycle? Draw a typical one. The set of states a defect report passes through from its discovery to its closure, and the transitions allowed between them. A typical one: New, then Assigned, Fixed, Verified and Closed, with New also leading to Rejected, Duplicate or Deferred, Deferred leading back to Assigned, and Fixed or Closed leading to Reopened, which leads back to Assigned.

2. Explain Florac's generic problem statuses. Open, meaning the problem is recognised and will be investigated and acted on, with the sub-states Recognized (enough data to evaluate), Evaluated (investigated enough to know the problem type) and Resolved (the rules for resolution satisfied); and Closed, meaning the investigation is complete and the action has been completed to everyone's satisfaction.

3. Why should a developer not be able to move a defect to Verified or Closed? Because verification is an independent check that the fix works and broke nothing else; if the person who made the fix could declare it verified, a fix that fails would be closed unchecked.

4. Distinguish rejected, duplicate and deferred. A rejected report is not a defect, for example a user's mistake or the software behaving as specified; a duplicate reports a defect already reported, and its work belongs to the original; a deferred report is a real defect whose fix has been postponed to a later release, and it stays open.

5. When is a defect reopened? When the confirmation test of its fix fails, or when a failure it caused appears again after the defect was closed.

6. How can a defect tracker's life cycle itself be tested? As a state machine: by making every valid transition, attempting every invalid one, and checking that the tracker refuses moves that do not exist and moves attempted by a role that may not make them.