Integration Testing: Why Units That Work Can Fail Together
Chapter Thirty-Seven
Syllabus topic Module 1, "Software Testing Strategies: Integration Testing"
Pages 204 to 208 of 622
In one line
Integration testing checks that units which each work on their own also work together, because the commonest integration defects live not inside any unit but in the agreements between them: what is passed, in what units and format, in what order, and what happens when something goes wrong.
In the wording a student can write in an examination: integration is the "process of combining software components, hardware components, or both into an overall system" (ISO/IEC/IEEE 24765), and integration testing is testing in which components "are combined and tested to evaluate the interaction among them" (ISO/IEC 29110-5-1-2:2025). Its object is the interface, the "point at which two or more logical, physical, or both, system elements or software system elements meet and act on or communicate with each other" (ISO/IEC/IEEE 12207:2026). It finds defects that unit tests cannot: mismatched parameters, wrong units or formats, wrong call order, and mishandled failures between components. It is done incrementally, a few components at a time, rather than big bang, all at once, so that a failure points to the interface just added.
Why units that pass can fail together
A unit test checks a unit against the unit's own specification. It cannot check that two specifications agree. If the fee calculator is designed to return an amount in paise and the payment module is designed to accept rupees, both can be built exactly to their designs, both can pass every unit test, and the system can still charge a student a hundred times too much. The defect is in neither unit. It is in the space between them, which only integration testing looks at.
The 2018 ISTQB syllabus lists the typical defects of component integration testing, and every one of them is a disagreement across an interface:
- "Incorrect data, missing data, or incorrect data encoding"
- "Incorrect sequencing or timing of interface calls"
- "Interface mismatch"
- "Failures in communication between components"
- "Unhandled or improperly handled communication failures between components"
- "Incorrect assumptions about the meaning, units, or boundaries of the data being passed between components"
The same syllabus sets integration testing's focus accordingly: "if integrating module A with module B, tests should focus on the communication between the modules, not the functionality of the individual modules, as that should have been covered during component testing."
Worked example: two correct units, one wrong system
Chapter Two, on errors, faults and failures, listed this defect in a table: the fee page calls the payment module with the amount in paise, and the payment module expects rupees. Here it is as running code. Each unit is written to its own specification and passes the tests written from that specification; then the two are joined.
FORM_FEE, BACKLOG_FEE = 800, 150
def fee_in_paise(days_late, backlog_papers, concession): # unit A: the fee page's calculator
"""Its design says: return the total fee in PAISE, for the page to display."""
if days_late < 0 or days_late > 15 or backlog_papers < 0:
raise ValueError("form not accepted")
rupees = FORM_FEE + BACKLOG_FEE * backlog_papers - (FORM_FEE if concession else 0)
rupees += 0 if days_late == 0 else (100 if days_late <= 7 else 500)
return rupees * 100
class PaymentModule: # unit B
"""Its interface document says: charge() takes the amount in RUPEES."""
def __init__(self):
self.ledger = []
def charge(self, roll_no, amount):
if amount <= 0:
raise ValueError("nothing to charge")
self.ledger.append((roll_no, amount))
return f"R-{1041 + len(self.ledger)}"
# each unit passes the tests written from its own specification
calculator_ok = fee_in_paise(10, 1, False) == 145000 # Rs 1,450 is 145,000 paise
payments = PaymentModule()
payment_ok = payments.charge("TEST", 1450) == "R-1042" and payments.ledger == [("TEST", 1450)]
print("unit test, fee calculator:", "pass" if calculator_ok else "FAIL")
print("unit test, payment module:", "pass" if payment_ok else "FAIL")
# integration: the fee page hands its total straight to the payment module
payments = PaymentModule()
payments.charge("2026CS014", fee_in_paise(10, 1, False))
charged = payments.ledger[0][1]
print("integration test, fee page to payment:", "pass" if charged == 1450 else "FAIL",
f"(Rs {charged} charged; the rule says Rs 1450, {charged // 1450} times less)")Integration Testing: Why Units That Work Can Fail Together
unit test, fee calculator: pass
unit test, payment module: pass
integration test, fee page to payment: FAIL (Rs 145000 charged; the rule says Rs 1450, 100 times less)Both unit tests pass, and both are right: the calculator really does return 145,000 paise, and the payment module really does charge whatever number of rupees it is given. The integration test fails because it is the first test that checks the one thing neither unit test could, the agreement between them. A student 10 days late with one backlog paper would be charged Rs 1,45,000 instead of Rs 1,450.
Notice how the fix is chosen. Neither unit is wrong by its own design, so the fault is in the designs: the two sides never agreed on one unit of money. The lasting fix is to write the unit into the interface specification, a "document that specifies the interface characteristics of an existing or planned system or component" (ISO/IEC/IEEE 24765), and change whichever side disagrees with it; the integration test then guards the agreement from then on.
The same defect, in space: the Mars Climate Orbiter
The ExamReg defect is invented; the pattern is not. On 23 September 1999 NASA lost the Mars Climate Orbiter as it arrived at Mars. The Mishap Investigation Board's report found a single root cause: "the failure to use metric units in the coding of a ground software file, 'Small Forces,' used in trajectory models."
Integration Testing: Why Units That Work Can Fail Together
The failure was an interface defect in the strict sense. The report explains that the output of one program, SM_FORCES, "as required by a MSOP Project Software Interface Specification (SIS) was to be in metric units of Newton-seconds (N-s). Instead, the data was reported in English units of pound-seconds (lbf-s)." The navigation software that read the file assumed the specified units, so it "underestimated the effect on the spacecraft trajectory by a factor of 4.45, which is the required conversion factor from force in pounds to Newtons." Over the nine-month journey the small errors added up, and at arrival "the spacecraft trajectory was approximately 170 kilometers lower than planned."
The board's findings on testing read like a list of what integration testing is for. "The Software Interface Specification (SIS) was developed but not properly used in the small forces ground software development and testing. End-to-end testing to validate the small forces ground software performance and its applicability to the specification did not appear to be accomplished." And: "The interface control process and the verification of specific ground system interfaces was not completed or was completed with insufficient rigor." Each program did its own job; the specification that joined them was written and then not tested against.
| ExamReg's fee page | Mars Climate Orbiter | |
|---|---|---|
| Producer | Fee calculator, in paise | SM_FORCES, in pound-seconds |
| Consumer | Payment module, expecting rupees | Navigation software, expecting newton-seconds |
| What each assumed | Its own unit | The unit in the interface specification |
| Size of the error | A factor of 100 | A factor of 4.45 |
| The test that would have caught it | An integration test across the fee-to-payment interface | End-to-end testing of the small forces software against the specification |
Big bang or incremental
There are two ways to bring units together.
- Big-bang integration combines all the units at once and tests the whole. It needs no stubs or drivers, but when a test fails, the fault could be in any unit or any interface. The 2018 syllabus warns that "The greater the scope of integration, the more difficult it becomes to isolate defects to a specific component or system, which may lead to increased risk and additional time for troubleshooting."
- Incremental integration adds a few units at a time and tests after each addition. The syllabus's advice is plain: "In order to simplify defect isolation and detect defects early, integration should normally be incremental (i.e., a small number of additional components or systems at a time) rather than 'big bang' (i.e., integrating all components or systems in one single step)."
On ExamReg the difference is concrete. The portal has seven small units under four subsystems. Integrated all at once, a failing registration could come from any of them and any of the links between them. Integrated one unit at a time, a new failure points to the unit just added and its interfaces, because everything before it was already passing. The price of incremental integration is the scaffolding it needs, stubs for units not yet added or drivers for units not yet called; the next chapter, on top-down, bottom-up and sandwich integration, counts that price for each order.
Integration Testing: Why Units That Work Can Fail Together
| Big bang | Incremental | |
|---|---|---|
| How | All units combined, then tested | A few units added at a time, tested after each |
| Stubs and drivers | None | Some, depending on the order |
| Locating a fault | Hard: anywhere in the system | Easier: the newest unit and its interfaces |
| When defects are found | Late, all together | Early, one addition at a time |
| Suits | Very small systems | Almost everything else |
What integration testing checks
Integration tests are designed from the documents that describe the joins: the architecture, the interface specifications, sequence diagrams and protocols. ISO/IEC/IEEE 24765 defines interface testing as "testing conducted to evaluate whether systems or components pass data and control correctly to one another", and a good set of integration tests for one interface checks each of these:
- Data: every parameter present, of the right type, in the right order, units and format; values at the boundaries of what the other side accepts.
- Control: calls made in the right order, the right number of times, with the right results returned.
- Failures: what the caller does when the called unit refuses, times out or is absent.
- Shared resources: data both sides read and write, such as a student record, left consistent.
What it does not mean
Integration testing is not repeating the unit tests on a bigger build. Its tests aim at the communication between units, not at each unit's functions.
Passing unit tests do not make integration safe. Both units in the example passed; the system still failed.
An interface defect is not always in one side's code. Often both sides match their own documents, and the documents disagree.
Big bang is not always wrong. For two or three tiny units it can be the quickest route; it becomes costly as the number of interfaces grows.
Quick revision
- Integration: combining components into an overall system. Integration testing: testing combined components to evaluate their interaction. Interface: where elements meet and act on or communicate with each other.
- Typical integration defects (ISTQB 2018): incorrect or missing data or encoding; wrong sequencing or timing of calls; interface mismatch; communication failures and their handling; wrong assumptions about "meaning, units, or boundaries".
- Worked example: calculator (paise) and payment module (rupees) each pass their unit tests; together they charge Rs 1,45,000 for a Rs 1,450 fee.
- Mars Climate Orbiter (1999): pound-seconds where the interface specification required newton-seconds; trajectory effect underestimated by a factor of 4.45; about 170 km too low; end-to-end testing against the specification not accomplished.
- Big bang (all at once, hard to locate faults) against incremental (a few at a time, faults found early and located easily; needs stubs and drivers).
Integration Testing: Why Units That Work Can Fail Together
Test yourself
1. Why can units that pass their unit tests fail when integrated? Because a unit test checks a unit against its own specification, and cannot check that two specifications agree. Defects in the agreements between units, such as the units, format, order or meaning of data passed, show only when the units run together.
2. List four typical defects found by integration testing. Incorrect, missing or wrongly encoded data passed between components; incorrect sequencing or timing of interface calls; interface mismatch, such as parameters of the wrong number, type or order; and wrong assumptions about the meaning, units or boundaries of the data, as in paise passed where rupees were expected.
3. Explain the loss of the Mars Climate Orbiter as an integration failure. One ground program, SM_FORCES, produced thruster data in pound-seconds although the software interface specification required newton-seconds, and the navigation software read it as newton-seconds, underestimating its effect by a factor of 4.45. The trajectory drifted about 170 km low. The investigation found the interface specification was not properly used in testing and end-to-end testing against it was not accomplished.
4. Compare big-bang and incremental integration. Big-bang integration combines all units at once, needs no stubs or drivers, but makes faults hard to locate and finds them late. Incremental integration adds a few units at a time and tests after each, so a failure points to the newest unit and its interfaces, at the cost of writing stubs or drivers.
5. What should an integration test of one interface check? That data passes with the right parameters, types, order, units, format and boundary values; that calls happen in the right order and number with the right results; that failures such as refusals, timeouts or absence are handled; and that shared data is left consistent.
The rest of this subject
These notes are cut from the University's printed syllabus. Open the syllabus itself, or the past papers, for the same subject.