Writing a Test Plan
Chapter Eleven
Syllabus topic Module 1, "Software Testing Fundamentals: Test execution, reporting, and documentation"; the paired practical, "Prepare a Test Plan"
Pages 62 to 67 of 622
In one line
A test plan is the document that says, before testing starts, what will be tested and why, how, by whom, with what, by when, and what "finished" will mean.
In the wording a student can write in an examination: a test plan is a "detailed description of test objectives to be achieved and the means and schedule for achieving them, organized to coordinate testing activities for some test item or set of test items" (ISO/IEC/IEEE 29119-2:2021); in the older wording, a "document describing the scope, approach, resources, and schedule of intended test activities" (IEEE 1012-2024). The classic outline, from IEEE 829-1998, has sixteen headings, from the test plan identifier to approvals.
Why a plan is written at all
A plan's first value is not the document but the thinking it forces. The ISTQB syllabus says it well: test planning "guides the testers' thinking and forces the testers to confront the future challenges related to risks, schedules, people, tools, costs, effort, etc." A tester who has to write down what will not be tested, what will stop testing, and what happens if the test environment arrives late, has thought about all three before they happen.
The syllabus lists four things a finished plan does. It documents the means and schedule for achieving the test objectives. It helps ensure the test activities meet the established criteria. It serves as communication with the team and other stakeholders. And it shows that testing follows the organisation's test policy and strategy, or explains why it does not.
Policy, strategy, plan
The plan sits at the bottom of a three-level hierarchy, and the words are easy to confuse.
| Document | Definition | Scope |
|---|---|---|
| Test policy | "executive-level document that describes the purpose, goals, principles, and scope of testing within an organization" (ISO/IEC/IEEE 29119-3:2021) | The whole organisation, for years |
| Test strategy | "part of the test plan that describes the approach to testing for a specific project, test level, or test type" (ISO/IEC/IEEE 29119-2:2021) | One project, level or type |
| Test plan | Objectives, means and schedule for testing a test item or set of items (ISO/IEC/IEEE 29119-2:2021) | One project or one test level |
Inside the strategy sits the test approach, a "high-level test implementation choice, typically made as part of the test strategy design activity" (ISO/IEC/IEEE 29119-1:2022): for example, risk-based, with specification-based techniques at system level and automated regression at every build. A project may have one master test plan covering all levels and a level test plan for each level, which is how IEEE 829-2008 organised them.
The classic outline: IEEE 829-1998's sixteen headings
IEEE 829, the Standard for Software Test Documentation, was first published in 1983, revised in 1998 and again in 2008, and has since been superseded by ISO/IEC/IEEE 29119-3. Its 1998 outline for a test plan is still the one most textbooks and many companies use, and it is a sound checklist. The sixteen headings, each with what goes under it:
Writing a Test Plan
| No. | Heading | What goes under it |
|---|---|---|
| 1 | Test plan identifier | A unique name and version for this plan |
| 2 | Introduction | Purpose, scope, objectives, and the documents this plan relies on |
| 3 | Test items | The software items to be tested, with their versions |
| 4 | Features to be tested | Each feature or requirement that will be tested |
| 5 | Features not to be tested | What is deliberately left out, and why |
| 6 | Approach | How testing will be done: levels, types, techniques, tools, automation |
| 7 | Item pass/fail criteria | How to decide whether each test item has passed |
| 8 | Suspension criteria and resumption requirements | When to stop testing, and what must be true to restart |
| 9 | Test deliverables | Every document and artefact testing will produce |
| 10 | Testing tasks | The tasks, their dependencies and the skills they need |
| 11 | Environmental needs | Hardware, software, network, tools, test data, facilities |
| 12 | Responsibilities | Who does what: test, fix, provide environments, approve |
| 13 | Staffing and training needs | The people needed and any training they require |
| 14 | Schedule | Milestones and dates, tied to the development schedule |
| 15 | Risks and contingencies | What could go wrong with testing, and the fallback for each |
| 16 | Approvals | Who must sign the plan, with names and dates |
Heading 5 surprises students most. Writing down what will not be tested is not an admission of failure; it is the point at which a manager can object. If the plan says load testing of the payment gateway is not in scope because the vendor certifies it, the principal can either accept that risk or insist otherwise, before the deadline rush rather than during it.
The same content, as ISTQB describes a plan today
The ISTQB syllabus (2024) lists the typical content of a test plan more compactly: the context of testing (scope, objectives, test basis); assumptions and constraints; stakeholders (roles, responsibilities, hiring and training needs); communication (forms, frequency, templates); a risk register of product and project risks; the test approach (levels, types, techniques, deliverables, entry and exit criteria, independence of testing, metrics, test data and environment requirements, deviations from policy and strategy); and budget and schedule. The two lists cover the same ground:
| ISTQB v4.0.1 content | IEEE 829-1998 headings that carry it |
|---|---|
| Context of testing | 2 Introduction, 3 Test items, 4 and 5 Features to be tested and not to be tested |
| Assumptions and constraints | 2 Introduction, 15 Risks and contingencies |
| Stakeholders | 12 Responsibilities, 13 Staffing and training needs, 16 Approvals |
| Communication | (not a heading of its own in 1998) |
| Risk register | 15 Risks and contingencies |
| Test approach | 6 Approach, 7 Item pass/fail criteria, 8 Suspension and resumption, 9 Deliverables, 11 Environmental needs |
| Budget and schedule | 10 Testing tasks, 14 Schedule |
Writing a Test Plan
Entry and exit criteria in the plan
The plan is where entry and exit criteria are set, for each test level. The ISTQB syllabus gives typical ones. Entry criteria concern the availability of resources (people, tools, environments, test data, budget, time), of testware (test basis, testable requirements, test cases), and the initial quality of the test object, such as "all smoke tests have passed". Exit criteria are either measures of thoroughness (coverage achieved, unresolved defects, defect density, failed test cases) or yes/no criteria (planned tests executed, static testing performed, all defects reported, regression tests automated).
The syllabus adds a realistic note: "Running out of time or budget can also be viewed as valid exit criteria", provided "the stakeholders have reviewed and accepted the risk to go live without further testing." And in agile teams the same ideas have other names: exit criteria are the Definition of Done, and the entry criteria a user story must meet before work starts are the Definition of Ready.
How much effort? Estimation in the plan
The schedule and budget depend on an estimate of test effort. The ISTQB syllabus describes four techniques. Estimation based on ratios uses figures from past projects, such as a test-to-development effort ratio. Extrapolation measures the current project early and projects forward, for example averaging the last three iterations. Wideband Delphi has experts estimate separately, discuss the outliers and re-estimate until they agree; Planning Poker is its agile variant. Three-point estimation asks for an optimistic estimate a, a most likely m and a pessimistic b, and combines them as E = (a + 4m + b) / 6, with a spread SD = (b - a) / 6.
The program applies three-point estimation to four of ExamReg's test tasks, in person-hours.
tasks = { # optimistic, most likely, pessimistic
"design fee and form tests": (16, 24, 44),
"set up test environment": (6, 8, 16),
"execute cycle 1": (40, 56, 90),
"execute regression cycle": (12, 16, 26),
}
total = 0
for task, (a, m, b) in tasks.items():
estimate = (a + 4 * m + b) / 6
spread = (b - a) / 6
total += estimate
print(f"{task:<28} E = {estimate:5.1f} SD = {spread:4.1f} "
f"(likely {estimate - spread:.1f} to {estimate + spread:.1f})")
print(f"{'total expected effort':<28} E = {total:5.1f} person-hours")design fee and form tests E = 26.0 SD = 4.7 (likely 21.3 to 30.7)
set up test environment E = 9.0 SD = 1.7 (likely 7.3 to 10.7)
execute cycle 1 E = 59.0 SD = 8.3 (likely 50.7 to 67.3)
execute regression cycle E = 17.0 SD = 2.3 (likely 14.7 to 19.3)
total expected effort E = 111.0 person-hoursWriting a Test Plan
Notice that each estimate E is larger than the most likely value m. The pessimistic figure pulls it up, because in testing, as in most work, things tend to take longer rather than shorter than hoped: a blocked environment or a defect-heavy build costs far more time than a smooth one saves. A plan that used m alone would be optimistic by construction.
Worked example: a test plan for ExamReg release 2.0
Below is a complete plan in the IEEE 829-1998 outline, the form the paired practical asks for. It is short because ExamReg is small; a bank's plan under the same headings runs to fifty pages.
1. Test plan identifier. STP-EXAMREG-2.0, version 1.1.
2. Introduction. This plan covers system testing and user acceptance testing of ExamReg release 2.0, the college's online examination registration portal, before the examination form opens for the November session. Objectives: show that fees are charged exactly as the fee rules say, that eligible students can register and ineligible ones cannot, and that the portal handles the last-day rush. References: ExamReg requirements v2.0, fee rules circular, test policy TP-01.
3. Test items. ExamReg web application build 2.0.x; the fee calculation service; the hall ticket generator; the payment gateway adapter (the gateway itself is the vendor's).
4. Features to be tested. Login and lockout; exam form, including backlog papers and eligibility; fee calculation, including late fees and concessions; payment and receipts; hall ticket generation; admin reports.
5. Features not to be tested. The payment gateway's internal processing (certified by the vendor; only our adapter is tested); the college's existing student database (read only, unchanged in this release); translations (English only in 2.0).
6. Approach. Risk-based. Specification-based techniques (equivalence partitioning, boundary values, decision tables for fee rules, state transitions for login); automated regression of fee and form tests on every build; a load test of 500 simultaneous users before release; exploratory sessions on the exam form. Independent testing by the college IT cell's test team.
7. Item pass/fail criteria. An item passes when all its high-priority tests pass and it has no open critical or major defect.
8. Suspension criteria and resumption requirements. Suspend if a build fails the smoke test or more than 25 per cent of scheduled tests are blocked. Resume on a build that passes the smoke test with the blocking cause fixed.
9. Test deliverables. This plan; test scenarios and cases; traceability matrix; test data; execution reports for each cycle; defect reports; load test report; test completion report.
10. Testing tasks. Requirement review; test design; environment set-up; cycle 1; defect fixing and retesting; regression cycle; load test; user acceptance testing; completion report. Test design depends on the approved requirements; execution depends on build delivery.
Writing a Test Plan
11. Environmental needs. Test server TEST-2 matching production; a controllable server clock; the payment gateway's sandbox; six test student accounts with prepared papers; Chrome, Firefox and Edge on desktop and on Android.
12. Responsibilities. Test lead: plan, monitor, report. Two testers: design and execution. Developers: fixes and unit tests. IT cell: environments. Exam cell: acceptance testing and answers to requirement questions.
13. Staffing and training needs. Two testers and a test lead for four weeks; one tester trained on the load testing tool.
14. Schedule. Test design weeks 1 and 2; environment ready end of week 1; cycle 1 week 3; regression and load test week 4; acceptance testing and sign-off at the end of week 4, one week before the form opens.
15. Risks and contingencies. Late build: reduce cycle 1 scope to high-priority tests. Gateway sandbox unavailable: test the adapter against a simulator and retest when the sandbox returns. Requirement changes from the exam cell: frozen after week 1 except for defects.
16. Approvals. Test lead; development lead; head of the exam cell; principal. Names, signatures and dates.
What it does not mean
A test plan is not a list of test cases. It plans the testing; test cases are designed later, in test design, and live in their own documents.
IEEE 829-1998 is not the current standard. It was superseded by IEEE 829-2008 and then by ISO/IEC/IEEE 29119-3; its sixteen headings survive because they make a good checklist.
"Features not to be tested" is not a weakness. It records a decision and its reason, so the people who carry the risk can accept or reject it.
A plan is not written once and frozen. It is revised when the project changes, and test control acts on it throughout.
Quick revision
- Test plan: objectives, means and schedule for testing a test item or items (ISO/IEC/IEEE 29119-2); scope, approach, resources and schedule (IEEE 1012-2024).
- Hierarchy: test policy (organisation) above test strategy (project, level or type) above the test plan; the test approach is chosen within the strategy.
- IEEE 829-1998's 16 headings: identifier; introduction; test items; features to be tested; features not to be tested; approach; item pass/fail criteria; suspension and resumption; deliverables; testing tasks; environmental needs; responsibilities; staffing and training; schedule; risks and contingencies; approvals.
- ISTQB v4.0.1 content: context, assumptions and constraints, stakeholders, communication, risk register, test approach, budget and schedule.
- Entry and exit criteria per level; in agile, Definition of Ready and Definition of Done.
- Estimation: ratios, extrapolation, Wideband Delphi, three-point (E = (a + 4m + b) / 6, SD = (b - a) / 6).
Writing a Test Plan
Test yourself
1. Define a test plan and state its purposes. A document describing the test objectives and the means, resources and schedule for achieving them. It documents how the objectives will be met, helps ensure the test activities meet their criteria, communicates with the team and stakeholders, and shows that testing follows the test policy and strategy.
2. List the sixteen headings of the IEEE 829-1998 test plan. Test plan identifier; introduction; test items; features to be tested; features not to be tested; approach; item pass/fail criteria; suspension criteria and resumption requirements; test deliverables; testing tasks; environmental needs; responsibilities; staffing and training needs; schedule; risks and contingencies; approvals.
3. Distinguish test policy, test strategy and test plan. The policy is an executive document stating the purpose, goals and principles of testing across the organisation. The strategy describes the approach to testing for a particular project, level or type. The plan sets the objectives, means, resources and schedule for testing specific items.
4. Give typical entry and exit criteria for system testing. Entry: the build has passed its smoke test, the environment and test data are ready, and the test cases are approved. Exit: all high-priority tests executed and passed, requirements coverage complete, and no open critical defect.
5. Using three-point estimation, estimate a task with a = 6, m = 9 and b = 18 person-hours. E = (6 + 4 × 9 + 18) / 6 = 10 person-hours, with SD = (18 - 6) / 6 = 2, so the task is likely to take between 8 and 12 person-hours. This is the worked example in the ISTQB syllabus.
6. Why should a test plan say which features will not be tested? Because excluding a feature is a risk decision, and recording it with its reason lets the stakeholders who carry the risk accept or reject it before testing, not discover it after release.
The rest of this subject
These notes are cut from the University's printed syllabus. Open the syllabus itself, or the past papers, for the same subject.