Practical 19: Load Testing with Apache JMeter

PHP's documentation says its built-in server runs a single process that handles one request at a time. Several processes can be asked for with PHP_CLI_SERVER_WORKERS, but not on Windows, so the lab, which usually runs the portal with four, runs it here with one, as a student's is. Everything this chapter measures is that one process.

The two curl lines time one request each, with nobody else using the site: the results page took about 4 milliseconds, and a login about 300. A login checks the password against the stored hash in inc/users.php, and that check is the slow part. Under load, every request waits for the ones in front of it, and the logins make the queue long.

The test plan

Build it in the GUI. Right-click the Test Plan, Add, Threads (Users), Thread Group:

Figure 22.1 Right-click, Add: the thread group is under Threads (Users). The other entries of Add are where config elements, listeners and assertions are found.

Fill in the thread group. Instead of fixed numbers, each field holds ${__P(name,default)}, JMeter's __P function, which reads a value given on the command line with -J, or uses the default. One plan then serves both scenarios:

Figure 22.2 Number of Threads ${__P(users,5)}, Ramp-up period ${__P(rampup,5)} and Loop Count ${__P(loops,10)}. With no -J values: 5 users, started over 5 seconds, 10 times round each.

Then add, to the thread group:

Figure 22.3 The whole plan in the tree on the left. Log in sends the form with its two parameters. Follow Redirects is off, so the login's own response, the 302 that sends the browser to the dashboard, is the sample; Dashboard is the next request.

Why the login is split into two requests: JMeter's manual says that when a request follows a redirect, the redirect and the page it leads to appear as extra samples under it, and the parent's time includes all of them. Two plain requests keep one sample per page, so every count in the reports means the same thing.

Why 1500 milliseconds: it is the frustration threshold JMeter's dashboard uses for its APDEX score by default. A page slower than that counts as a frustrated user in the report's own terms, so the plan counts it as an error.

Save the plan as portal-load.jmx. This is the file, with what the GUI wrote shortened to the settings used:

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.6.3">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Practice Portal load test">
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Students">
        <stringProp name="ThreadGroup.num_threads">${__P(users,5)}</stringProp>
        <stringProp name="ThreadGroup.ramp_time">${__P(rampup,5)}</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller">
          <stringProp name="LoopController.loops">${__P(loops,10)}</stringProp>
          <boolProp name="LoopController.continue_forever">false</boolProp>
        </elementProp>
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
      </ThreadGroup>
      <hashTree>
        <ConfigTestElement guiclass="HttpDefaultsGui" testclass="ConfigTestElement" testname="HTTP Request Defaults">
          <stringProp name="HTTPSampler.protocol">http</stringProp>
          <stringProp name="HTTPSampler.domain">localhost</stringProp>
          <stringProp name="HTTPSampler.port">8080</stringProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
        </ConfigTestElement>
        <hashTree/>
        <CookieManager guiclass="CookiePanel" testclass="CookieManager" testname="HTTP Cookie Manager">
          <collectionProp name="CookieManager.cookies"/>
          <boolProp name="CookieManager.clearEachIteration">true</boolProp>
        </CookieManager>
        <hashTree/>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Home page">
          <stringProp name="HTTPSampler.path">/index.php</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree/>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Results page">
          <stringProp name="HTTPSampler.path">/results.php</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree>
          <ResponseAssertion guiclass="AssertionGui" testclass="ResponseAssertion" testname="Page heading present">
            <collectionProp name="Asserion.test_strings">
              <stringProp name="1">Semester Results</stringProp>
            </collectionProp>
            <stringProp name="Assertion.test_field">Assertion.response_data</stringProp>
            <boolProp name="Assertion.assume_success">false</boolProp>
            <intProp name="Assertion.test_type">16</intProp>
          </ResponseAssertion>
          <hashTree/>
        </hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Results data">
          <stringProp name="HTTPSampler.path">/api/results.php</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree/>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Log in">
          <stringProp name="HTTPSampler.path">/login.php</stringProp>
          <stringProp name="HTTPSampler.method">POST</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">false</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments">
              <elementProp name="username" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">true</boolProp>
                <stringProp name="Argument.name">username</stringProp>
                <stringProp name="Argument.value">asha</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
                <boolProp name="HTTPArgument.use_equals">true</boolProp>
              </elementProp>
              <elementProp name="password" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">true</boolProp>
                <stringProp name="Argument.name">password</stringProp>
                <stringProp name="Argument.value">Asha@2026</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
                <boolProp name="HTTPArgument.use_equals">true</boolProp>
              </elementProp>
            </collectionProp>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree>
          <ResponseAssertion guiclass="AssertionGui" testclass="ResponseAssertion" testname="Sent to the dashboard">
            <collectionProp name="Asserion.test_strings">
              <stringProp name="1">302</stringProp>
            </collectionProp>
            <stringProp name="Assertion.test_field">Assertion.response_code</stringProp>
            <boolProp name="Assertion.assume_success">false</boolProp>
            <intProp name="Assertion.test_type">8</intProp>
          </ResponseAssertion>
          <hashTree/>
        </hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Dashboard">
          <stringProp name="HTTPSampler.path">/dashboard.php</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree>
          <ResponseAssertion guiclass="AssertionGui" testclass="ResponseAssertion" testname="Welcome shown">
            <collectionProp name="Asserion.test_strings">
              <stringProp name="1">Welcome, Asha Patil</stringProp>
            </collectionProp>
            <stringProp name="Assertion.test_field">Assertion.response_data</stringProp>
            <boolProp name="Assertion.assume_success">false</boolProp>
            <intProp name="Assertion.test_type">16</intProp>
          </ResponseAssertion>
          <hashTree/>
        </hashTree>
        <DurationAssertion guiclass="DurationAssertionGui" testclass="DurationAssertion" testname="Answered within 1.5 seconds">
          <stringProp name="DurationAssertion.duration">1500</stringProp>
        </DurationAssertion>
        <hashTree/>
      </hashTree>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

The misspelt Asserion.test_strings is JMeter's own spelling, kept for old plans; type it as it is.

Scenario 1: light load

In a terminal, in the project folder: 5 users started over 5 seconds, 10 rounds each, 5 requests a round, so 250 samples:

$ jmeter -n -t portal-load.jmx -Jjmeter.reportgenerator.overall_granularity=2000 -l light.jtl -e -o light-report 2>&1 | tail -3
summary =    250 in 00:00:13 =   19.5/s Avg:   165 Min:     0 Max:   730 Err:     0 (0.00%)
Tidying up ...    @ 2026 Sep 30 20:21:32 IST (1790779892151)
... end of run

The options: -n runs without the GUI, -t names the plan, -l the file every sample is written to, and -e -o builds the HTML dashboard into a new folder when the test ends. The -J setting is a JMeter property from its documentation: the report generator's overall_granularity, one minute by default, is lowered to 2 seconds so the dashboard's charts have a point every two seconds; the manual warns against going below one second. 2>&1 | tail -3 joins JMeter's warnings to its normal output and keeps only the last three lines, which hold the result: the screen stays quiet until the test ends, then shows three lines like these. Without it you also see some start-up lines first, among them four warnings about "package scanning" from the logging library inside JMeter 5.6.3 itself, not from your plan. Then comes a progress line at every 30-second mark of the clock (summariser.interval, 30 seconds by default), so how many you see depends on when a run starts and how long it takes.

The summary line is the result: 250 samples in 13 seconds, 19.5 per second; average 165 ms, minimum 0, maximum 730; 0 errors. Every page came back in well under a second, and every assertion held. The last two lines say the test has ended.

Scenario 2: heavy load

The same plan with 50 users, -Jusers=50, started over the same 5 seconds: 2500 samples.

$ jmeter -n -t portal-load.jmx -Jusers=50 -Jjmeter.reportgenerator.overall_granularity=2000 -l heavy.jtl -e -o heavy-report 2>&1 | tail -3
summary =   2500 in 00:02:08 =   19.6/s Avg:  2378 Min:     1 Max:  4857 Err:  2204 (88.16%)
Tidying up ...    @ 2026 Sep 30 20:37:21 IST (1790780841080)
... end of run

2500 samples in 2 minutes 8 seconds, 19.6 per second; average 2378 ms, maximum 4857 ms; 2204 errors, 88.16%.

The graphical reports

Each run left a dashboard: open heavy-report/index.html or light-report/index.html in a browser. The figures here are from a second pair of runs made in the lab for the pictures, and their numbers are not the ones printed above. That difference is explained below, and it is a lesson in itself.

The dashboard page gives the APDEX score, the pass and fail shares, and the Statistics table: for each request, its samples, failures, error %, average, median and percentiles, and its throughput.

Figure 22.4 The heavy run of the figure pair: 2500 samples, 78.48% failed, average 2628 ms, 17.71 transactions a second in all. Log in is the worst request: 94.80% failed, average 3797 ms, 90th percentile 5662 ms.

Charts, Over Time, Response Times Over Time plots each request's average response time every two seconds. Heavy against light:

Figure 22.5 50 users. The lines climb while the threads start, during the 5-second ramp-up, then swing between about 1.5 and 6 seconds until the users finish. The top line, purple in the legend, is Log in.

Figure 22.6 5 users. Log in, purple, between about 360 and 780 ms; every other page close to zero on this scale.

Analysing the results

From the printed pair of runs:

• Throughput did not grow with the users. Ten times the users gave the same 19.5 to 19.6 requests a second. That is the server's ceiling: one PHP process can finish only so many requests a second, however many are waiting.
• So the waiting grew instead. The average response time rose from 165 ms to 2378 ms, about fourteen times. With the server at its ceiling, each extra user only lengthens the queue every request waits in.
• The errors are late answers, not wrong ones. In the lab's results files every failure carried the Duration Assertion's message, "The operation lasted too long"; the pages themselves were right. At 50 users 88% of the requests were too slow, the logins worst of all, because each carries the slow password check.
• The capacity limit lies between the two loads: at 5 users nothing was slow; at 50, nearly everything was. The next step in a real test would be scenarios in between, 10, 20, 30 users, to find where the error % starts to rise.

Why the figure pair differs. Its heavy run served 17.71 requests a second with 78.48% errors, against 19.6 and 88.16% above: close, but not the same. Another pair of runs in the lab went further apart: 39.08 requests a second and 27.28% errors, the same plan and the same site at twice the speed. The lab's machine is shared, and when other work was using its processors, the PHP process had less time. JMeter's own guidance begins with sizing the machine that runs the test. A load test measures the whole machine, not only the application, so results are compared only between runs made on the same, otherwise idle, machine, and every test is repeated before a number is believed. Each pair here was run back to back, and within each pair the picture is the same: the heavy load at the server's ceiling and far slower.

Observations

Result

A load test plan was designed in JMeter with a parameterized thread group, five HTTP requests with assertions and a 1.5-second time limit, and run in CLI mode in two scenarios. With 5 users the Practice Portal answered in 165 ms on average with no errors; with 50 users its throughput stayed at about 19.6 requests a second, its ceiling on one PHP process, while the average response time rose to 2378 ms and 88.16% of the requests missed the time limit. JMeter's HTML dashboards showed the same in their statistics table and response-time charts. A second pair of runs on the busy shared machine gave different numbers with the same pattern, which is why load test results are compared only on one idle machine.

Where marks are lost

Running the load in GUI mode. JMeter's manual says CLI mode must be used for load testing.

No ramp-up. A ramp-up of 0 starts every thread at the same instant, which tests a crowd arriving in one moment, not a load. JMeter's manual suggests starting with the ramp-up equal to the number of threads.

No assertions. Without them a page that says "Error" still counts as a success. Assert something only the right page contains.

Reading only the average. It hides the slowest requests; read the 90th percentile and the maximum too.

Comparing runs made at different times on a busy machine. Their numbers differ for reasons that have nothing to do with the application.

Following redirects without knowing it. The redirect adds hidden samples, and the counts stop meaning one page each.

For the journal

Aim; the table of JMeter's parts; the thread group settings and why ${__P(...)}; the table of elements added; the two command lines and their summary lines; the dashboard's statistics table and the response-time charts; the analysis table and what it shows; a note on the machine the test ran on; the result.

Quick revision

• Thread group: number of threads (users), ramp-up period, loop count.
• Ramp-up: 10 threads over 100 seconds start 10 seconds apart.
• Samples are users × loops × requests per loop: 5 × 10 × 5 = 250, and 50 × 10 × 5 = 2500.
• Response time is elapsed time; throughput is requests per second; error % is failed samples over all samples.
• jmeter -n -t plan.jmx -l results.jtl -e -o report: CLI run with the HTML dashboard; summary + lines are interim, summary = is the total.
• ${__P(users,5)} reads -Jusers=50, or uses 5.
• At the server's ceiling, more users raise response time, not throughput.
• A load test measures the whole machine: repeat it on an idle one.

Questions you must be able to answer

1. What do the number of threads, the ramp-up period and the loop count control? How many virtual users run, how long JMeter takes to start all of them, and how many times each repeats the steps.

2. Why is the test run from the command line and not from the GUI? JMeter's manual says GUI mode is only for creating the test script; the GUI uses the machine being measured, so load is run in CLI mode.

3. Throughput was about 19.6 requests a second under both loads. What does that tell you? The server was already at its maximum under the light load's pace; more users could not make it serve more requests a second, so they waited longer instead.

4. Why did the average response time rise so much under heavy load? The single PHP process serves one request at a time, so with 50 users each request waited behind many others.

5. What counted as an error in this plan? A failed assertion or HTTP error, and above all any sample slower than 1.5 seconds, the Duration Assertion's limit.

6. The two heavy runs gave 88.16% and 27.28% errors. Which one is right? Both are true measurements of different conditions: the machine was busier during the first. Load tests must be repeated on an idle machine before their numbers are compared.