Pareto Diagrams

Worked example: release 2.0's defects, ranked two ways

The chart needs only a count (or a cost) by category. The program uses release 2.0's 200 defects by type (FINDINGS 5.2), which ASQ's steps 1 to 5 already fixed: the category is the defect type, the measurement is frequency, and the period is the whole release. It ranks them twice: once by how many defects of each type occurred, and once by how many hours they took to fix, using the average fix effort of FINDINGS 5.12. The second ranking is ASQ's first listed variation, the weighted Pareto chart, used when a rarer category may still matter more because each occurrence costs more.

# release 2.0's 200 defects by type (FINDINGS 5.2) and the average hours to fix one (FINDINGS 5.12)
defects = {"input validation": (58, 1.2), "logic and computation": (44, 4.6),
           "interface": (30, 3.6), "user interface": (24, 1.0), "data and database": (18, 5.5),
           "documentation": (12, 0.5), "performance": (8, 7.5), "security": (6, 9.5)}

def pareto(values, title):
    """Sort the categories largest first; print each with its share and the cumulative share."""
    total, running = sum(values.values()), 0
    print(f"{title}: {total:g} in all")
    for name, value in sorted(values.items(), key=lambda item: -item[1]):
        running += value
        print(f"   {name:<23}{value:>7g}{value / total:>6.0%}{running / total:>7.0%}")

pareto({name: n for name, (n, _) in defects.items()}, "defects by type")
pareto({name: n * hours for name, (n, hours) in defects.items()}, "hours to fix, by type")

defects by type: 200 in all
   input validation            58   29%    29%
   logic and computation       44   22%    51%
   interface                   30   15%    66%
   user interface              24   12%    78%
   data and database           18    9%    87%
   documentation               12    6%    93%
   performance                  8    4%    97%
   security                     6    3%   100%
hours to fix, by type: 626 in all
   logic and computation    202.4   32%    32%
   interface                  108   17%    50%
   data and database           99   16%    65%
   input validation          69.6   11%    77%
   performance                 60   10%    86%
   security                    57    9%    95%
   user interface              24    4%    99%
   documentation                6    1%   100%

Figure 104.1 Release 2.0's defects by type, ranked, with the cumulative line ASQ's construction adds

By count, the vital few are four types. Input validation, logic and computation, interface and user interface together are 156 of 200 defects, 78 per cent, the four bars the figure shades. Data and database brings the running total to 87 per cent with a fifth category. The three smallest types, documentation, performance and security, are 13 per cent of the count between them: Juran's useful many, not worth ignoring, but not where a first action should go.

By hours, the ranking changes. Weighted by the effort to fix one, logic and computation moves to the top on its own, 202.4 of 626 hours, 32 per cent, because each one takes nearly four times as long to fix as an input validation defect (4.6 hours against 1.2). Input validation, the largest category by count, drops to fourth by hours; security, the smallest category by count at only 6 defects, is the fifth-largest by hours because each one averages 9.5 hours. The two charts agree that logic, interface and data defects all belong near the top, but the count chart puts input validation first and the hours chart puts it fourth.

Which ranking to use. ASQ's first measurement choice is "frequency, quantity, cost and time"; both are legitimate Pareto charts on the same data, and the right one depends on what is scarce. Chapter Eighty acted on the count, correctly: the goal there was fewer defects reaching the customer, for which the largest count is the largest target. Chapter One Hundred One, on the cost of quality, and Chapter One Hundred Two, on using quality costs for decision making, act on cost and hours, for which the weighted chart is the one that answers the question being asked. A Pareto chart does not choose the measurement; the decision it is meant to support does.

What the two charts do not say

Neither chart says input validation defects are unimportant, or that logic defects are the only ones worth fixing. A Pareto chart ranks a fixed period's data; it does not predict next release's ranking, and Chapter Eighty showed that a targeted action can move a category from first to fourth in one release (2.90 to 1.12 defects per KLOC) while leaving every other type's rate unchanged or slightly higher. Ranking is a starting point for where to look, repeated each time there is new data, not a verdict fixed for all time.

What it does not mean

The vital few are not fixed for every measurement. The same data, counted differently, can rank differently: release 2.0's chart by count and by hours agree on three types out of five in the top half, not all.

80-20 is a tendency, not an exact split. Chapter Eighty-Two, on the Quality Movement's Shewhart, Deming and Juran, already recorded Boehm and Basili's range of 60 to 90 per cent of software defects from 20 per cent of modules across studies; ExamReg's own split by type, 78 per cent in four of eight types (50 per cent of the types), is one instance of the same tendency, not a proof of the exact 80/20 figure.

A Pareto chart ranks; it does not explain. It shows that input validation defects are the largest category, not why they occur. Chapter One Hundred Five, on cause-effect diagrams, asks why for the largest category of a different kind of event, the help desk's complaints about failed fee payments.

"Other" is a convenience, not a category to act on. ASQ allows grouping small categories as "other" to keep the chart readable; an action cannot be aimed at "other" because it is not one cause.

Quick revision

• Pareto chart (ASQ): a bar graph of frequency or cost, longest bar to shortest, showing which few categories matter most; variations include the weighted Pareto chart (by cost or time, not just count) and comparative Pareto charts.
• Pareto principle (Juran, 1950; ASQ's 80-20 rule): about 80 per cent of effects from about 20 per cent of causes; the vital few causes against the useful many (Juran's later term for what he first called the "trivial many").
• Construction (ASQ): fix the categories, the measurement and the period; subtotal each category; scale the left axis to the largest subtotal (or, with a cumulative line, to the total); draw bars largest to smallest; optionally add a right-hand percentage scale and a cumulative line from dots at each bar's running total, reaching 100 per cent.
• Worked example: release 2.0's 200 defects by type; by count, four of eight types make 78 per cent; by hours to fix, the ranking changes and logic and computation alone is 32 per cent.
• A Pareto chart ranks a measurement for a fixed period; it does not explain a cause or predict the next period's ranking.

Test yourself

1. What is a Pareto chart, and when is it used? A bar graph that ranks categories from the largest measurement to the smallest, usually with a cumulative percentage line, used when there are many possible problems or causes and the most significant few need to be found.

2. State the Pareto principle and Juran's two names for its two groups. About 80 per cent of effects come from about 20 per cent of possible causes. Juran called the 20 per cent of causes the "vital few" and the rest the "useful many", a term he adopted in place of "trivial many" once he judged no quality problem to be trivial.

3. List the main steps in constructing a Pareto chart. Decide the categories, the measurement (frequency, quantity, cost or time) and the period to cover; collect or assemble the data; subtotal each category; scale the chart (to the largest subtotal, or to the total if a cumulative line is added); draw the bars from largest to smallest; optionally add a percentage scale and a cumulative line of dots from each bar's running total to 100 per cent.

4. What is a weighted Pareto chart, and when would you use one? A Pareto chart ranked by cost or time rather than by count, used when a category with few occurrences still matters more because each occurrence is expensive. Release 2.0's defects ranked by hours to fix put logic and computation defects first, even though input validation had more defects, because each logic defect took nearly four times as long to fix.

5. Using release 2.0's defect data, which types are the vital few by count, and how does the ranking change by cost? By count, input validation, logic and computation, interface, and user interface are 78 per cent of the 200 defects. Weighted by average hours to fix, the ranking changes: logic and computation alone is 32 per cent of the 626 hours, and input validation drops to fourth.

6. Why does a Pareto chart not explain why a problem occurs, and what tool does? A Pareto chart only ranks categories by a measurement; it says which problem is largest, not what causes it. Chapter One Hundred Five, on cause-effect diagrams, explores the possible causes of a problem once the Pareto chart has identified which one to examine.