The Cost of Quality

The first two are the costs of doing quality work: in the standard's words, the costs "associated with assuring that a product or service conforms to requirements". The last two are the "failure costs from non-conformance to requirements", Crosby's price of nonconformance. ASQ also names the cost of poor quality (COPQ), "the costs associated with providing poor quality products or services", which it divides into appraisal, internal failure and external failure costs; the two groupings answer different questions, and an answer should say which one it uses.

Worked example: release 2.0's cost of quality statement

The software house records effort in person-hours and charges Rs 800 an hour. Release 2.0's quality-related work comes from the book's data: the reviews, tests and fix counts of earlier chapters, and the average effort to fix a defect where it was found (1.0 hour in a review, 4.0 in testing, 8.0 after release). The program sets out the cost of quality statement.

# release 2.0's cost of quality, in person-hours (FINDINGS 5.10)
RATE, PROJECT_HOURS = 800, 4800                # Rs per person-hour; the whole project's effort
statement = {
    "prevention": {"SQA plan and procedures": 24, "training in reviews and test techniques": 40,
                   "standards and checklists kept up to date": 16},
    "appraisal": {"reviews": 62, "testing": 296, "SQA audits": 48},
    "internal failure": {"fixing 74 review finds": 74 * 1.0, "fixing 114 test finds": 114 * 4.0,
                         "redoing 13 reopened fixes": 30},
    "external failure": {"fixing 12 after-release defects": 12 * 8.0, "handling complaints": 40,
                         "correcting 9 wrong fees by hand": 12},
}
total = sum(sum(items.values()) for items in statement.values())
for category, items in statement.items():
    hours = sum(items.values())
    print(f"{category:<17}{hours:>6.0f} h  Rs {hours * RATE:>9,.0f}  {hours / total:>4.0%}")
    for item, h in items.items():
        print(f"   {item:<40}{h:>6.0f} h")
conformance = sum(statement["prevention"].values()) + sum(statement["appraisal"].values())
print(f"cost of quality {total:,.0f} h, Rs {total * RATE:,.0f}:"
      f" {total / PROJECT_HOURS:.1%} of the project's {PROJECT_HOURS:,} hours")
print(f"conformance (prevention + appraisal) {conformance / total:.0%},"
      f" nonconformance (failures) {1 - conformance / total:.0%}")

prevention           80 h  Rs    64,000    7%
   SQA plan and procedures                     24 h
   training in reviews and test techniques     40 h
   standards and checklists kept up to date    16 h
appraisal           406 h  Rs   324,800   34%
   reviews                                     62 h
   testing                                    296 h
   SQA audits                                  48 h
internal failure    560 h  Rs   448,000   47%
   fixing 74 review finds                      74 h
   fixing 114 test finds                      456 h
   redoing 13 reopened fixes                   30 h
external failure    148 h  Rs   118,400   12%
   fixing 12 after-release defects             96 h
   handling complaints                         40 h
   correcting 9 wrong fees by hand             12 h
cost of quality 1,194 h, Rs 955,200: 24.9% of the project's 4,800 hours
conformance (prevention + appraisal) 41%, nonconformance (failures) 59%

Where the money went. Release 2.0's quality costs came to 1,194 person-hours, about Rs 9.55 lakh, a quarter (24.9 per cent) of the whole project's effort. Almost half, 47 per cent, was internal failure: fixing and refixing the defects found before release. External failure was 12 per cent, appraisal 34 per cent, and prevention only 7 per cent.

What the shape says. Failures took 59 per cent of the cost of quality, the work of doing quality only 41 per cent, and prevention, the category whose purpose is to stop the others arising, was the smallest by far. That is the pattern Boehm and Basili describe across software, where projects "spend about 40 to 50 percent of their effort on avoidable rework". Much of release 2.0's 560 hours of internal failure was avoidable: 58 of its defects were input validation defects that one shared module and one checklist item later cut sharply (Chapter Eighty, on using defect data). The statement therefore points where Chapter One Hundred Two, on using quality costs for decision making, looks next: moving hours from failure to prevention and early appraisal.

What the figures leave out. The statement counts the software house's hours. External failure also costs the customer: the students who were charged wrong fees, the exam cell's lost time, and the trust in the portal. ASQ's list of external failure costs includes "Complaints: All work and costs associated with handling and servicing customers' complaints", but the harm to users rarely appears on anyone's timesheet, which is one reason external failure is worse than its hours suggest.

Reading ASQ's rule of thumb carefully

ASQ writes that "Many organizations will have true quality-related costs as high as 15-20% of sales revenue, some going as high as 40% of total operations", and that "A general rule of thumb is that costs of poor quality in a thriving company will be about 10-15% of operations." Those figures are percentages of sales revenue or of operations, and they are not directly comparable with ExamReg's 24.9 per cent of one project's effort. A cost of quality figure means something only against its own base and its own history: the useful comparison for ExamReg is release against release, which is the trend Chapter One Hundred Two computes.

What it does not mean

The cost of quality is not the cost of the quality department. It includes every hour spent because quality matters, most of it by developers fixing defects.

A low cost of quality is not automatically good. Cutting appraisal and prevention can lower it for one release while defects move to the customer, where failure costs are highest.

Prevention and appraisal are not waste. They are the costs of conformance; the goal is to spend them where they save more in failure costs.

The four categories are not fixed labels for activities. A review is appraisal; improving the review checklist so that defects are not made is prevention; the classification follows the purpose of the work.

Quick revision

• Cost of quality (ISO/IEC/IEEE 24774:2021): the life-cycle costs of assuring conformance to requirements, plus the failure costs of non-conformance.
• Four categories (ASQ): prevention (plans, training, standards, QA); appraisal (verification, reviews, testing, audits); internal failure (rework before delivery); external failure (repairs, complaints, returns after delivery).
• Conformance and nonconformance: prevention and appraisal against internal and external failure; Crosby's "price of nonconformance".
• History: Feigenbaum first classified quality costs into the four categories.
• Worked example: release 2.0: 1,194 hours (Rs 9,55,200), 24.9 per cent of the project; prevention 7, appraisal 34, internal failure 47, external failure 12 per cent; failures 59 per cent of the cost of quality.

Test yourself

1. Define the cost of quality and name its four categories. The total cost of everything done because quality matters: the costs of assuring that the product conforms to its requirements plus the costs of failing to conform. Its categories are prevention, appraisal, internal failure and external failure.

2. Give two examples of each category in a software project. Prevention: writing the quality plan and standards; training developers and testers. Appraisal: reviews and inspections; testing and SQA audits. Internal failure: fixing defects found in testing; retesting reopened fixes. External failure: fixing defects users report; handling complaints and correcting wrong results for users.

3. What is the difference between the cost of conformance and the cost of nonconformance? The cost of conformance is what is spent to make and check that the product conforms: prevention and appraisal. The cost of nonconformance is what is spent because it did not: internal and external failure, which Crosby called the price of nonconformance.

4. Draw up a cost of quality statement from given figures. List the items under the four categories with their hours or amounts, total each category, express each as a share of the total cost of quality, and compare the total with the project's effort. For ExamReg release 2.0: prevention 80 hours, appraisal 406, internal failure 560, external failure 148, a total of 1,194 hours, 24.9 per cent of the project.

5. What did release 2.0's statement show, and what does it suggest? That failures took 59 per cent of the cost of quality and prevention only 7 per cent, with internal failure the largest category at 47 per cent. It suggests shifting effort towards prevention and early appraisal, since many of the failures, such as the input validation defects, were avoidable.

6. Why is external failure worse than its hours suggest? Because its costs fall partly on the customer and are not in the supplier's accounts: students charged wrong fees, the exam cell's time, and lost trust in the product. A defect that reaches the user also costs the most to fix.