Exploratory Testing

The ISTQB syllabus agrees on the conditions for success: exploratory testing "will be more effective if the tester is experienced, has domain knowledge and has a high degree of essential skills, like analytical skills, curiosity and creativeness".

Managing it: charters, sessions and debriefs

Unscripted does not have to mean unmanaged. Jonathan Bach's session-based test management, introduced in 2000, gives exploratory testing a unit of work that can be planned, counted and reviewed: the session, "an uninterrupted block of reviewable, chartered test effort".

The charter. Each session has a mission. A charter "states the mission and perhaps some of the tactics to be used", and may be chosen by the tester or assigned by the test lead. Bach's example charters for a decision-analysis product include short ones such as "Check UI against Windows interface standards." For ExamReg, a charter might read: explore the fee payment page with interrupted journeys: back, refresh, a second tab.

The time box. In the 2000 article's team, "sessions last 90 minutes, give or take"; one closer to 45 minutes is a short session, and one closer to two hours a long one. "Uninterrupted" means "no significant interruptions, no email, meetings, chatting or telephone calls."

The session sheet. Each session produces a report, the session sheet, in a fixed tagged format so that a program can read it: the charter and the areas tested, the tester, the start time, a task breakdown, the bugs found, and the issues raised. The task breakdown divides the session into three kinds of work, "test design and execution, bug investigation and reporting, and session setup", with the tester's estimate of each, and records the share of time spent "on charter" against "on opportunity", where "Opportunity testing is any testing that doesn't fit the charter of the session." Bugs are "concerns about the quality of the product"; issues are "questions or problems that relate to the test process or the project at large".

The debrief. "Each session is debriefed." The test lead's "primary objective in the debriefing is to understand and accept the session report. Another objective is to provide feedback and coaching to the tester."

The ISTQB syllabus describes the same arrangement: exploratory testing "is performed within a defined time box. The tester uses a test charter containing test objectives to guide the testing. The test session is usually followed by a debriefing". It adds a point about coverage: in this approach "test objectives may be treated as high-level test conditions. Coverage items are identified and exercised during the test session."

Worked example: four sessions on ExamReg

Four exploratory sessions were run on ExamReg before release, two by each of two testers, each with its own charter. Their session sheets, in the article's tagged format:

CHARTER
Explore the fee payment page with interrupted journeys: back, refresh, a second tab.
#AREAS
Page | Fee payment
START
2026-10-12 10:00
TESTER
Neha
#DURATION
normal
#TEST DESIGN AND EXECUTION
55
#BUG INVESTIGATION AND REPORTING
35
#SESSION SETUP
10
#CHARTER VS. OPPORTUNITY
90/10
BUGS
#BUG 1
Refreshing the confirmation page sends a second payment request.
#BUG 2
Back from the gateway shows the fee as unpaid although the payment went through.
ISSUES
#ISSUE 1
What should the page show if the gateway has not answered after 60 seconds?
====
CHARTER
Explore the fee payment page on a slow mobile connection, as a student in a hostel would.
#AREAS
Page | Fee payment
Platform | Android
START
2026-10-12 14:00
TESTER
Arjun
#DURATION
short
#TEST DESIGN AND EXECUTION
70
#BUG INVESTIGATION AND REPORTING
20
#SESSION SETUP
10
#CHARTER VS. OPPORTUNITY
100/0
BUGS
#BUG 3
The Pay button can be pressed again while the first press is still loading.
ISSUES
====
CHARTER
Explore the hall ticket download after payment, including a payment made minutes before the deadline.
#AREAS
Page | Hall ticket
Page | Fee payment
START
2026-10-13 10:00
TESTER
Neha
#DURATION
long
#TEST DESIGN AND EXECUTION
60
#BUG INVESTIGATION AND REPORTING
15
#SESSION SETUP
25
#CHARTER VS. OPPORTUNITY
70/30
BUGS
#BUG 4
A hall ticket downloaded before the payment is confirmed shows no seat number.
ISSUES
#ISSUE 2
The test database had no student with a concession; setup took longer than planned.
====
CHARTER
Explore the exam form's paper selection with unusual combinations of regular and backlog papers.
#AREAS
Page | Exam form
START
2026-10-13 14:00
TESTER
Arjun
#DURATION
normal
#TEST DESIGN AND EXECUTION
80
#BUG INVESTIGATION AND REPORTING
10
#SESSION SETUP
10
#CHARTER VS. OPPORTUNITY
100/0
BUGS
ISSUES

The session-based team read their sheets with a tool that "breaks them down into their basic elements, normalizes them, and summarizes them into tables and metrics". The program below does the same for these four: it reads each sheet, turns the duration into minutes, and totals the task breakdown weighted by session length.

import re

MINUTES = {"short": 45, "normal": 90, "long": 120}      # Bach: about 90 minutes; short, long

def parse(sheet):
    """One session sheet as {section: [lines]}; each #BUG and #ISSUE heading is one entry."""
    sections, tag = {}, None
    for line in sheet.strip().splitlines():
        if re.fullmatch(r"#(BUG|ISSUE) \d+", line):            # "#BUG 3", not "#BUG INVESTIGATION"
            sections.setdefault(line.split()[0], []).append(line)
            tag = None                                   # the bug's text is not a section
        elif line.isupper() or line.startswith("#"):
            tag = line
            sections.setdefault(tag, [])
        elif tag:
            sections[tag].append(line)
    return sections

sheets = [parse(s) for s in open("sessions.txt").read().split("====")]
totals = {"test": 0, "bug": 0, "setup": 0, "charter": 0, "minutes": 0}
print(f"{'tester':<7}{'minutes':>8}{'test':>6}{'bug':>5}{'setup':>6}{'on charter':>11}{'bugs':>6}{'issues':>7}")
for s in sheets:
    minutes = MINUTES[s["#DURATION"][0]]
    test, bug, setup = (int(s[k][0]) for k in ("#TEST DESIGN AND EXECUTION",
                        "#BUG INVESTIGATION AND REPORTING", "#SESSION SETUP"))
    on = int(s["#CHARTER VS. OPPORTUNITY"][0].split("/")[0])
    bugs, issues = len(s.get("#BUG", [])), len(s.get("#ISSUE", []))
    print(f"{s['TESTER'][0]:<7}{minutes:>8}{test:>5}%{bug:>4}%{setup:>5}%{on:>10}%{bugs:>6}{issues:>7}")
    for key, share in (("test", test), ("bug", bug), ("setup", setup), ("charter", on)):
        totals[key] += minutes * share / 100
    totals["minutes"] += minutes

m = totals["minutes"]
print(f"{len(sheets)} sessions, {m} minutes: test design and execution {100 * totals['test'] / m:.0f}%,"
      f" bug investigation {100 * totals['bug'] / m:.0f}%, setup {100 * totals['setup'] / m:.0f}%,"
      f" on charter {100 * totals['charter'] / m:.0f}%")
areas = sorted({a.split("|")[1].strip() for s in sheets for a in s["#AREAS"]})
print("areas explored:", ", ".join(areas))

tester  minutes  test  bug setup on charter  bugs issues
Neha         90   55%  35%   10%        90%     2      1
Arjun        45   70%  20%   10%       100%     1      0
Neha        120   60%  15%   25%        70%     1      1
Arjun        90   80%  10%   10%       100%     0      0
4 sessions, 345 minutes: test design and execution 65%, bug investigation 20%, setup 15%, on charter 87%
areas explored: Android, Exam form, Fee payment, Hall ticket

Four sessions, 345 minutes of testing, and four bugs, two of them in the first session, which is typical of a charter aimed at a risky area: interrupted payment journeys are where a web application's state goes wrong. Neither bug was in any test case, and neither would have been: the specification says what happens when a payment succeeds, not what happens when the student presses Back in the middle of it.

The totals are what a test lead reads before the debriefs. About two-thirds of the time went on test design and execution, a fifth on investigating and reporting bugs, and the rest on setup. The third session is the one to ask about: 25 per cent setup and 30 per cent off charter, and its issue says why, because the test data had no student with a concession. That is a problem with the test environment, not the tester, and the debrief is where it gets fixed. The last session found nothing, which is also a result: the exam form's paper selection survived 90 minutes of determined attempts to break it, and the notes of what was tried are the evidence.

Where exploratory testing fits

The ISTQB syllabus names its natural territory: it "is useful when there are few or inadequate specifications or there is significant time pressure on the testing", and "to complement other more formal test techniques". It "can incorporate the use of other test techniques (e.g., equivalence partitioning)". Bach's own list adds, among others, providing rapid feedback on a new feature, learning a product quickly, diversifying testing after scripts have been run, and investigating a particular defect. He also notes that "most formal written test procedures were probably created through a process of some sort of exploratory testing."

Strengths and limits

Strengths. It finds the problems no one wrote a test for, because it follows the software instead of a plan. It adapts at once to what it finds, spending time where the problems are. It needs little preparation and gives fast feedback. And, run in sessions, it can be planned, counted and reported.

Limits. It depends on the tester's skill more than any other technique. It is hard to repeat exactly, since the next session will explore differently. Its coverage is judged by charters and notes, not by a count of items fixed in advance. And without the discipline of charters, notes and debriefs, it degenerates into the aimless clicking it is sometimes mistaken for.

What it does not mean

Exploratory testing is not ad hoc testing. It has a mission, a time box, notes and a review; freestyle exploring is one end of a continuum, not the whole of it.

It is not undocumented. In session-based test management every session produces a sheet that a third party can review.

It does not replace scripted testing. It complements it: scripted regression tests check what is known, exploration looks for what is not.

A session that finds no bugs is not wasted. It is evidence about an area, and its notes say what was tried.

Quick revision

• Bach: "Exploratory testing is simultaneous learning, test design, and test execution"; a continuum from pure scripted to freestyle.
• ISO/IEC/IEEE 29119-2: unscripted experience-based testing, tests designed and executed spontaneously, guided by heuristics.
• Explorer's skills (Bach): test design, careful observation, critical thinking, diverse ideas (heuristics), rich resources.
• Session-based test management (Jonathan Bach, 2000): a session is "an uninterrupted block of reviewable, chartered test effort", about 90 minutes; a charter gives the mission; a session sheet records charter, areas, tester, start, task breakdown (test design and execution, bug investigation and reporting, session setup; on charter against opportunity), bugs and issues; every session is debriefed.
• Worked example: 4 sessions, 345 minutes, 4 bugs; 65 per cent test design and execution, 20 per cent bug investigation, 15 per cent setup, 87 per cent on charter.
• Useful with few or inadequate specifications, under time pressure, and as a complement (ISTQB).

Test yourself

1. Define exploratory testing. How does it differ from scripted testing? Exploratory testing is simultaneous learning, test design and test execution: the tester designs each test while testing, using what earlier tests revealed. In scripted testing the tests are designed in advance, often by someone else, and executed as written, without the results changing the tests.

2. What is a test charter? Write one for ExamReg. A short statement of the mission of an exploratory session, what to test and what to look for, perhaps with tactics. For example: explore the fee payment page with interrupted journeys (back, refresh, a second tab) and report any payment made twice or lost.

3. Explain session-based test management. A way of managing exploratory testing in sessions: uninterrupted, time-boxed blocks of chartered testing of about 90 minutes. Each session produces a session sheet recording the charter, areas, tester, time, task breakdown, bugs and issues, and each is debriefed by the test lead, who accepts the report and coaches the tester. The sheets are totalled into metrics that show progress.

4. What is the task breakdown of a session sheet, and what can a test lead learn from it? The tester's estimate of the share of session time spent on test design and execution, on bug investigation and reporting, and on session setup, and of time on charter against on opportunity. It shows where testing time goes: much setup or much off-charter work points to problems such as missing test data, as in the worked example's third session.

5. When is exploratory testing most useful? When specifications are few or inadequate, when time is short, when fast feedback on a new feature is needed, when a product must be learnt quickly, after scripted tests to diversify the testing, and to investigate a particular defect or risk.

6. What skills does an exploratory tester need? Test design, careful observation that separates what is seen from what is inferred, critical thinking about one's own logic, a supply of diverse ideas helped by heuristics, and resources such as tools, data and colleagues; with domain knowledge, curiosity and creativity.