Software Testing and Quality Assurance Practical Notes | B.Sc. (Computer Science) Semester 5 | Mumbai University | munotes

Get access to whole semester resourcesSemester Pass

Semester 1 Whole semester notes View

Official Notes munotes.in

Software Testing and Quality Assurance Practical

B.SC. (COMPUTER SCIENCE) · SEMESTER 5

Strictly as per the University of Mumbai NEP syllabus in force for B.Sc. (Computer Science)

For B.Sc. (Computer Science) students of the University of Mumbai and all its affiliated colleges

Open the book ↓

munotes.in Third Year

Software Testing and Quality Assurance Practical

Written and first published by munotes.in, 2026.

This book is free for individual students to read at munotes.in. No part of it may be reproduced, distributed, stored, translated or used for institutional or classroom purposes in any form without a prior written licence from munotes.in.

Licensing and permissions: contact@munotes.in

The text of statutes and of judgments reproduced in this book is in the public domain under section 52(1)(q) of the Copyright Act 1957. The commentary, arrangement, examples and questions are the original work of munotes.in.

munotes.in is an independent study resource for MU students. It is not affiliated with, endorsed by, or officially connected to the University of Mumbai. Course names and university references describe the students and syllabus the material relates to.

Module I Selenium IDE, manual test cases, Selenium Grid and Selenium WebDriver: suites, logins, web elements and waits, data-driven testing with Apache POI, dynamic tables, counting objects, and lists

How This Practical Is Examined: the Journal, the 80 Per Cent Rule and the Two-Hour Paper 1
The Testing Laboratory from Zero: the Practice Portal, the Browsers and the Tools 7
Practical 1: Creating a Test Suite with Selenium IDE 14
Practical 2: Cross-Website Functional Testing with Selenium IDE 26
Practical 3: Manual Test Case Design and Execution 35
Selenium WebDriver from Zero: Java, Maven, the Drivers, Locators and the First Script 44
Practical 4: Selenium Grid, Hub and Nodes, and Remote Execution 53
Practical 5: Login Automation and Validation 59
Practical 6: Web Elements, Alerts, Frames and Waits 65
Practical 7: Data-Driven Testing with Apache POI 71
Practical 8: Extracting Marks from a Dynamic Web Table 77
Practical 9: Identifying and Counting the Objects on a Page 82
Practical 10: List and Combo Box Verification 86

Module II Checkboxes, sorting and filtering, uploads and downloads with the Robot class, Log4j and screenshots, the Page Object Model, TestNG, Jenkins, cross-browser Grid runs, JMeter load tests and Bugzilla

Practical 11: Checkbox Identification and Validation 90
Practical 12: Dynamic Web Table, Sorting and Filtering 93
Practical 13: File Upload and Download with the Robot Class 98
Practical 14: Screenshot on Failure and Logging with Log4j 106
Practical 15: The Page Object Model Framework 113
Practical 16: TestNG Annotations, Groups, Priorities and Reports 121
Practical 17: Continuous Integration with Jenkins 128
Practical 18: Cross-Browser Testing with Selenium Grid 135
Practical 19: Load Testing with Apache JMeter 142
Practical 20: Bug Tracking and the Defect Life Cycle in Bugzilla 149
The Two-Hour Paper: Sitting the Examination 157

Module I

Selenium IDE, manual test cases, Selenium Grid and Selenium WebDriver: suites, logins, web elements and waits, data-driven testing with Apache POI, dynamic tables, counting objects, and lists

Chapter One

How This Practical Is Examined: the Journal, the 80 Per Cent Rule and the Two-Hour Paper

Syllabus topic Module 1 and Module 2, with MU's assessment rules for a 2-credit practical course: "Certified Journal is compulsory for appearing at the time of Practical Exam", "Minimum 80% practical are required to be completed.", "Mid – Term Practical Examination 15 marks", "Final Journal: 5 marks", and the programme's "Individual Passing in Internal and External Examination".

Aim

To know, before writing a single test, exactly how this paper is marked, what has to be in the journal, what the two hours in the examination hall contain, and what it takes to pass.

Why this chapter comes first

Every other chapter in this book teaches you to do something: build a Selenium IDE suite, write a WebDriver script, configure Jenkins, load a server with JMeter. This one tells you what the marks are for.

It comes first because students lose marks on a practical paper without making a single mistake in a test. Three of MU's rules are about paperwork and attendance at the machine, and one of them can stop you from sitting the examination at all. Read this chapter once now and once again a week before the practical examination.

What this paper is

Software Testing and Quality Assurance Practical is a Major Elective paper of Semester 5 of the B.Sc. (Computer Science). In MU's own words its Type is Practical: there is no written theory examination for this paper. It carries 2 credits, MU allots 60 hours of laboratory work to it, and it is marked out of 50.

"Elective" means you chose it. MU's Semester 5 structure prints the electives as "any one": either Software Testing and Quality Assurance with this practical, or Wireless and Sensor Networks with its own practical. The theory paper and this practical are two separate papers with separate marks. This book is the practical. It recalls theory only where an exercise needs it.

The paper has two modules of thirty hours each, and each module is ten exercises. Module 1 runs from Selenium IDE to WebDriver scripts that read tables, lists and pages. Module 2 runs from checkboxes and file uploads to TestNG, Jenkins, Selenium Grid, JMeter and Bugzilla.

The paper in one table

		Marks
Internal	Mid-term practical examination	15
Internal	Final journal	5
External	Q.1, a practical question on Module 1	15
External	Q.2, a practical question on Module 2	15
	Total	50

The external examination runs for two hours and carries 30 of the 50 marks. The other 20 are internal and are decided during the semester, before you walk into the hall.

What it takes to pass

The programme's own particulars print two lines that matter here: "Individual Passing in Internal and External Examination", and a standard of passing of "40% in each component".

So there are two separate passes, not one:

Component	Out of	40 per cent of it
Internal	20	8
External	30	12

A student who scores 20 out of 20 internally and 10 out of 30 in the practical examination has 30 out of 50, which looks like a pass, and has still failed the external component. A strong journal cannot carry a weak examination, and a strong examination cannot carry a missing journal.

How This Practical Is Examined: the Journal, the 80 Per Cent Rule and the Two-Hour Paper

The two rules that can stop you sitting the paper

"Certified Journal is compulsory for appearing at the time of Practical Exam." Those are MU's words. A journal that has not been signed by your teacher is not a certified journal. If you arrive for the practical examination without one, no amount of skill with Selenium will get you into the hall.

"Minimum 80% practical are required to be completed." There are twenty practicals in this paper, ten in each module. Eighty per cent of twenty is sixteen. So at least sixteen of the twenty have to be done, written up and signed.

Sixteen is a floor, not a target. Q.1 is set on Module 1 and Q.2 on Module 2, and nobody tells you in advance which exercise either will be. A student who has skipped four practicals has skipped four of the twenty things the examiner chooses from. Do all twenty.

The internal marks

Mid-term practical examination, 15 marks. A practical test during the semester, at a machine, on the practicals done so far. Treat it as a rehearsal of the final paper: same kind of question, same need to show the script, the run and the result.

Final journal, 5 marks. The journal you hand in at the end, complete and certified. Five marks is small, but the journal is also the document that decides whether you may sit the external examination at all, so its real weight is much larger than five.

What "completed" means for one practical

A practical is complete when your journal, for that practical, contains all of this:

The practical number and its title, in MU's own words.
Aim. One or two lines saying what the exercise is for.
Tool. MU prints a tool for every exercise: Selenium IDE, Selenium WebDriver, Apache POI, TestNG, Jenkins, JMeter, Bugzilla. Name it, with its version.
Theory. What you need to know to do it. Short: this is a practical journal, not a theory answer.
The application or page under test. Which page, which fields, which behaviour.
Test cases or procedure. The steps, numbered, or the test cases in a table.
The script, the test suite, or the configuration, written out.
The output: the run, exactly as the tool printed it, or the test result.
Observations. The counts, the pass and fail figures, the comparison, the timings. This is the part most students leave out and the part the examiner reads first.
Conclusion. One or two lines saying what the run showed.
The date and your teacher's signature.

How This Practical Is Examined: the Journal, the 80 Per Cent Rule and the Two-Hour Paper

Every chapter of this book ends with a section called For the journal that says what the write-up for that practical has to contain.

What a testing practical adds

A testing practical has one thing a programming practical does not: an expected result. A test is a comparison between what should happen and what did happen. A script that clicks through a page and prints nothing has tested nothing, however much it did.

So in every write-up, and in every answer in the hall, say what you expected before you show what you got. "Expected: the welcome message reads Welcome, Asha Patil. Actual: it does. Result: pass." That one habit is most of what separates a testing answer from a programming answer.

The two-hour paper: how to spend the time

Two hours, two questions, fifteen marks each. That is an hour a question, and the two questions come from two different modules. One might be a Selenium IDE suite or a WebDriver script on a web table; the other might be a TestNG suite, a JMeter test plan or a Bugzilla defect report.

A working plan:

Minutes	What you are doing
0 to 5	Read both questions. Decide which one you are surer of.
5 to 10	Write the aim, the tool and the test cases for the first question on paper.
10 to 45	Build the test or the script, run it, fix it, and get a result.
45 to 55	Write the output, the observations and the conclusion into the answer.
55 to 105	The same for the second question.
105 to 120	Check both: is the output written down, are the observations there, is the conclusion there.

Start with the question you are surer of. A test that runs and is written up completely is worth more than two half-finished ones.

Never leave the output blank. If the script will not run, write down the exact error the tool gave, say what you think it means, and move on. Testers read error messages for a living; an honest error with a sentence of diagnosis reads far better than an empty page. Chapter six, which teaches WebDriver from zero, produces the errors a beginner meets first on purpose, so that you recognise them in the hall.

What the examiner is actually looking at

The examiner has fifteen marks for one practical question. A fair division, and the one this book is written to satisfy, is roughly this:

Part	Weight
The test or script is correct and runs	high
The result is present and matches what was run	high
Expected against actual stated, with a pass or fail	high
The observations: counts, timings, comparisons	medium
Aim, tool and procedure written out	medium
You can explain what your own test did	medium

How This Practical Is Examined: the Journal, the 80 Per Cent Rule and the Two-Hour Paper

The last line is not a separate viva: this paper prints no viva component, and all fifteen marks belong to the practical question. But an examiner standing at your machine will ask about what is on your screen, and your answer is part of how the fifteen marks are decided. Every chapter therefore ends with Questions you must be able to answer, and they are the questions asked at a machine: what does this locator find, why this wait, what happens if I change this value.

The laboratory this book uses

Every exercise needs something to test. MU's own wording says "a given web application" and "a specified web page", which means your college may give you its own. This book cannot know which one, so it brings its own: a small practice web application called the Practice Portal, and a second small site called the City Library, both set up in the next chapter. Every page they contain is printed in the chapter that tests it, and every program in the book was run against them.

If your college gives you a different site, nothing changes except the address and the locators, and each chapter says where those go.

A warning about copying

Twenty students in one laboratory, one internet, and the same twenty practicals. It is obvious to an examiner when four journals carry the same script with the same variable names and the same comment in the same place, and just as obvious when the student in front of them cannot say what one line of it does.

Write the scripts. Run them. Break one on purpose and read what the error says. That is what makes the questions at the machine easy, and it is the only preparation for this paper that works.

The shape of every chapter after this one

Each of the twenty practicals gets a chapter, in MU's printed order, laid out the same way so that you can read it straight into your journal:

Aim. MU's own words for the exercise.
What you need to know before you start. The theory, cut to what this exercise needs.
The page under test. The part of the practice site the exercise works on, printed in full.
The procedure, step by step.
The script or the suite, complete, and its real output.
Observations, filled in from the run.
Result. What the run showed.
Where marks are lost.
For the journal.
Quick revision. Eight or ten lines for the day before.
Questions you must be able to answer.

Result

The assessment of this paper was recorded: 20 internal marks, a mid-term practical examination for 15 and the final journal for 5; a Semester End Practical Examination of two hours for 30 marks, one practical question on each module for 15 marks each; a certified journal compulsory for appearing; at least sixteen of the twenty practicals completed; and separate passing, 8 of 20 internal and 12 of 30 external.

How This Practical Is Examined: the Journal, the 80 Per Cent Rule and the Two-Hour Paper

Where marks are lost

An uncertified journal. Get each practical signed in the week you do it, not in the week before the examination.

Fewer than sixteen practicals. Count them. Twenty minus four is the line, and it is a hard line.

No expected result. A test with no expected result is a demonstration, not a test.

No observations. Half the exercises in this paper ask you to count, compare or analyse something. A script that runs but reports no figure has done half the exercise.

Output not in the journal. The script is not the practical. The script plus its run plus what you concluded is the practical.

Spending ninety minutes on one question. Two questions, fifteen marks each. Half the paper is in the other module.

For the journal

This chapter is not one of the twenty practicals and does not go in the journal. Use it as the index page of your practical file: write out the list of twenty practicals from MU's syllabus, tick each one when it is signed, and count the ticks.

Quick revision

50 marks: 20 internal, 30 external. Type: Practical. No theory examination for this paper.
Internal 20: mid-term practical examination 15, final journal 5.
External 30: two hours, Q.1 on Module 1 for 15, Q.2 on Module 2 for 15.
Passing is separate: 40 per cent of each, which is 8 of 20 and 12 of 30.
Certified journal compulsory for appearing at the practical examination.
Minimum 80 per cent of the practicals, which is sixteen of twenty.
2 credits, 60 hours, thirty hours a module, ten exercises a module.
A Major Elective: chosen with its theory paper, against Wireless and Sensor Networks.
No separate viva marks are printed for this paper.
Every test states its expected result before its actual result.

Questions you must be able to answer

1. How many marks is this paper out of, and how are they split? Fifty. Twenty internal, a mid-term practical examination for 15 and the final journal for 5, and thirty external in a two-hour practical examination.

2. You score 19 internal and 11 external. Have you passed? No. Passing is individual in each component at 40 per cent. Nineteen out of twenty clears the internal line of 8, but eleven out of thirty is below the external line of 12.

How This Practical Is Examined: the Journal, the 80 Per Cent Rule and the Two-Hour Paper

3. How many practicals must be completed? At least eighty per cent, which is sixteen of the twenty MU prints.

4. What happens if your journal is not certified? MU's rule is that a certified journal is compulsory for appearing at the practical examination.

5. What does "elective" mean for this paper? It was chosen: MU offers Software Testing and Quality Assurance with this practical, or Wireless and Sensor Networks with its practical, and a student takes one pair.

6. Is there a viva in this paper? No viva component is printed. All fifteen marks of each question belong to the practical, although the examiner's questions about your screen help decide them.

7. What does a testing write-up need that a programming write-up does not? An expected result for every check, stated before the actual result, and a pass or fail.

8. Your script will not run and ten minutes are left. What do you write? The aim, the tool, the test cases, the script as far as you have it, the exact error message, and one sentence saying what you think the error means. Never an empty output.

Contents This chapter on its own page

Chapter Two

The Testing Laboratory from Zero: the Practice Portal, the Browsers and the Tools

Syllabus topic The laboratory every exercise in the paper is done in. MU's own description of the course: "hands-on exposure to automated and manual testing methodologies using industry-standard tools", and her wording for the thing tested: "a given web application", "a specified web page".

Aim

To set up, on your own machine, everything the twenty practicals test and run in: a practice web application with every kind of page MU's exercises name, a second small website for the cross-website exercise, and the browsers; and to know which tool each practical needs and where in this book it is installed.

What a testing laboratory needs

Three things, and only one of them is a testing tool.

Something to test. Testers call it the application under test, often shortened to AUT. MU's wording is "a given web application" and "a specified web page". It has to have pages of the right kinds: a login, a form, a table, a list, checkboxes, a file upload.
Browsers. Every exercise in this paper drives a real browser.
The tools. Selenium IDE, Java with Selenium WebDriver, TestNG, Jenkins, JMeter and Bugzilla. None of them is needed on day one, and each is installed in the chapter that first uses it.

The first of the three is the one students most often do not have. A public website changes without notice, is slow on a laboratory connection, and cannot be loaded with a thousand requests or filled with test records without being a nuisance to its owner. So this book brings its own.

The Practice Portal and the City Library

The Practice Portal is a small college website written for this book in PHP. It has a login, a registration form, a form whose fields change as you fill it, a student records page you can edit, a marks table that loads after the page does, a sortable results table, lists, checkboxes, alerts, a frame, a file upload and download, an XML response and a JSON API. Every exercise in the paper has its page.

The City Library is a second, deliberately different website, used in Practical 2, where MU asks for "two different websites".

Three things make a practice site better than a real one for learning:

You can read its source. Every page is printed in the chapter that tests it, so when a locator fails you can look at the HTML and see why.
It behaves the same every time. Its records start from fixed files, and one page puts them back.
It has defects on purpose. A tester needs something to find. Practical 3 finds five, and Practical 12 finds one more.

The site runs on your own machine, at an address starting http://localhost:8080/. localhost means "this computer": the browser, the tests and the site are all on the same machine, and nothing goes over the internet.

Step 1: PHP

The site needs PHP, and only PHP: no database and no web server, because PHP carries a small web server of its own. Its own documentation describes it as meant "to aid application development" and warns that it "should not be used on a public network". That is exactly the use here.

The Testing Laboratory from Zero: the Practice Portal, the Browsers and the Tools

If your laboratory has XAMPP, PHP is already there, in C:\xampp\php\php.exe. If not, download PHP for Windows from windows.php.net, unzip it into a folder such as C:\php, and add that folder to the Path. On Ubuntu it is sudo apt install php-cli.

Open a Command Prompt (or a terminal) and check it:

$ php -v
PHP 8.4.26 (cli) (built: Sep 24 2026 17:16:18) (NTS)
Copyright (c) The PHP Group
Built by Debian
Zend Engine v4.4.26, Copyright (c) Zend Technologies
    with Zend OPcache v8.4.26, Copyright (c), by Zend Technologies

Any PHP from 8.0 onward runs the site. What is printed above is the laboratory machine these notes were checked on, which runs PHP on Debian Linux; yours will name your own version and build, and on Windows says nothing about Debian.

Step 2: the folder and the shared files

Make a folder called practice, and inside it these folders: inc, assets, api, seed, data, files and uploads. The files in this chapter are the ones every page shares. Each page itself is printed in the chapter of the practical that tests it, so the site grows as you go.

The router. PHP's built-in server reads this file first on every request. Save it as practice/router.php:

<?php
// router.php: read only by PHP's built-in server (php -S). Apache ignores it.
// It refuses the three folders a browser has no business reading, answers 404
// for a page that does not exist (PHP 8.4 would otherwise serve index.php in its
// place), and hands every other request back to the server to serve as usual.
$path = parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH);
if (preg_match('#^/(data|seed|inc)/#', $path)) {
    http_response_code(403);
    echo 'Forbidden';
    return true;
}
if ($path !== '/' && !file_exists(__DIR__ . $path)) {
    http_response_code(404);
    echo 'Not found';
    return true;
}
return false;

Returning false tells the server to serve the file that was asked for, in the ordinary way. Returning true means the router has answered the request itself. The 404 rule is there because PHP 8.4 changed the built-in server: asked for a page that does not exist, it now looks for an index.php and serves that instead. A test that opens a page you have not created yet should see "Not found", not the home page.

The top and bottom of every page. Save these as practice/inc/top.php and practice/inc/bottom.php:

<?php
// inc/top.php: the start of every page. A page sets $title and then includes this.
?>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title><?= htmlspecialchars($title) ?> - Practice Portal</title>
<link rel="stylesheet" href="assets/site.css">
</head>
<body>
<header><strong>Practice Portal</strong> <a href="index.php">Home</a></header>
<main>

The Testing Laboratory from Zero: the Practice Portal, the Browsers and the Tools

</main>
<footer>Practice Portal, a practice site for software testing exercises.</footer>
</body>
</html>

Notice the title: a page that sets $title = 'Login' gets the browser title Login - Practice Portal. Half the scripts in this book check a title, and this is where it comes from.

The record store. The site keeps its records in small JSON files in data/, copied the first time from seed/. Save this as practice/inc/store.php:

<?php
// inc/store.php: the site keeps its records in small JSON files in data/.
// The first time a file is needed it is copied from seed/, so deleting the
// files in data/ puts the site back exactly as it started (reset.php does that).

function data_file($name) {
    return __DIR__ . '/../data/' . $name . '.json';
}

function load_records($name) {
    $file = data_file($name);
    if (!file_exists($file)) {
        copy(__DIR__ . '/../seed/' . $name . '.json', $file);
    }
    return json_decode(file_get_contents($file), true);
}

function save_records($name, $records) {
    file_put_contents(data_file($name), json_encode($records, JSON_PRETTY_PRINT), LOCK_EX);
}

The reset page. Practical 7 edits records and Practical 13 uploads files. This page puts everything back. Save it as practice/reset.php:

<?php
// reset.php: puts every record back as it was and empties the uploads folder.
foreach (glob(__DIR__ . '/data/*.json') as $file) { unlink($file); }
foreach (glob(__DIR__ . '/uploads/*') as $file) { unlink($file); }
$title = 'Reset';
include 'inc/top.php';
echo '<p id="message" class="success">The practice site has been reset.</p>';
include 'inc/bottom.php';

The style sheet, practice/assets/site.css. It only makes the pages readable; no test depends on a colour:

body { font-family: Arial, Helvetica, sans-serif; margin: 0; color: #222; }
header { background: #1f4e79; color: #fff; padding: 10px 16px; }
header a { color: #fff; margin-left: 16px; }
main { padding: 16px; max-width: 900px; }
footer { padding: 10px 16px; color: #666; font-size: 13px; border-top: 1px solid #ddd; }
label { display: block; margin-top: 10px; }
input, select, textarea, button { font-size: 15px; padding: 4px 6px; }
table { border-collapse: collapse; margin-top: 10px; }
th, td { border: 1px solid #999; padding: 4px 10px; text-align: left; }
th { background: #e8eef5; }
.error { color: #b00020; }
.success { color: #1b5e20; }
.hidden { display: none; }

The home page, practice/index.php, which links to every page the practicals add:

<?php $title = 'Home'; include 'inc/top.php'; ?>
<h1>Practice Portal</h1>
<p>Every page on this site exists so that it can be tested.</p>
<ul id="pages">
  <li><a href="about.php">About the portal</a> (an ordinary HTML page)</li>
  <li><a href="api/notices.php">Notices</a> (an XML response)</li>
  <li><a href="api/students.php">Students API</a> (a JSON response)</li>
  <li><a href="enquiry.php">Admission enquiry</a> (a form that changes as you fill it)</li>
  <li><a href="register.php">Student registration</a></li>
  <li><a href="login.php">Login</a></li>
  <li><a href="elements.php">Form elements, alerts and a frame</a></li>
  <li><a href="students.php">Student records</a></li>
  <li><a href="marks.php">Marks table</a></li>
  <li><a href="objects.php">Objects on a page</a></li>
  <li><a href="lists.php">Lists and combo boxes</a></li>
  <li><a href="checkboxes.php">Checkboxes</a></li>
  <li><a href="results.php">Results table (sort and filter)</a></li>
  <li><a href="upload.php">Upload a file</a></li>
  <li><a href="downloads.php">Download a file</a></li>
</ul>
<?php include 'inc/bottom.php'; ?>

The Testing Laboratory from Zero: the Practice Portal, the Browsers and the Tools

Until a practical adds a page, its link answers "Not found". That is the router doing its job.

If your college serves pages with XAMPP's Apache instead, put a file called .htaccess containing the single line Require all denied in each of data, seed and inc, which does for Apache what the router does for PHP's own server.

Step 3: start the site

In the Command Prompt, go into the practice folder and start PHP's server on port 8080, naming the router:

$ php -S localhost:8080 router.php
[Wed Sep 30 15:47:08 2026] PHP 8.4.26 Development Server (http://localhost:8080) started
[Wed Sep 30 15:47:08 2026] [::1]:51430 Accepted
[Wed Sep 30 15:47:08 2026] [::1]:51430 [200]: GET /
[Wed Sep 30 15:47:08 2026] [::1]:51430 Closing

The first line says the server has started. The server keeps running and holds the window, so leave that window open, and open a second one for everything else. The three lines after the first are the log of one request, the first page anybody asked for: the connection accepted, the answer given ([200] and the page), and the connection closed. The server prints those three lines for every request, which is often the quickest way to see what a test actually asked for.

Now open http://localhost:8080/ in a browser and the home page appears. From the second window you can ask for a page without a browser at all, with curl, which Microsoft has shipped with Windows 10 since build 17063 and which every Linux has. On Windows, use it from the Command Prompt:

$ curl -s "http://localhost:8080/"
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Home - Practice Portal</title>
<link rel="stylesheet" href="assets/site.css">
</head>
<body>
<header><strong>Practice Portal</strong> <a href="index.php">Home</a></header>
<main>
<h1>Practice Portal</h1>
<p>Every page on this site exists so that it can be tested.</p>
<ul id="pages">
  <li><a href="about.php">About the portal</a> (an ordinary HTML page)</li>
  <li><a href="api/notices.php">Notices</a> (an XML response)</li>
  <li><a href="api/students.php">Students API</a> (a JSON response)</li>
  <li><a href="enquiry.php">Admission enquiry</a> (a form that changes as you fill it)</li>
  <li><a href="register.php">Student registration</a></li>
  <li><a href="login.php">Login</a></li>
  <li><a href="elements.php">Form elements, alerts and a frame</a></li>
  <li><a href="students.php">Student records</a></li>
  <li><a href="marks.php">Marks table</a></li>
  <li><a href="objects.php">Objects on a page</a></li>
  <li><a href="lists.php">Lists and combo boxes</a></li>
  <li><a href="checkboxes.php">Checkboxes</a></li>
  <li><a href="results.php">Results table (sort and filter)</a></li>
  <li><a href="upload.php">Upload a file</a></li>
  <li><a href="downloads.php">Download a file</a></li>
</ul>
</main>
<footer>Practice Portal, a practice site for software testing exercises.</footer>
</body>
</html>

That is the exact HTML a browser receives, with inc/top.php and inc/bottom.php around the page's own part. Ask for a page that does not exist, and the router answers as it should. -w "%{http_code}" makes curl print only the status code the server sent back:

The Testing Laboratory from Zero: the Practice Portal, the Browsers and the Tools

$ curl -s -o /dev/null -w "%{http_code}\n" "http://localhost:8080/no-such-page.php"
404

On Windows, write -o NUL in place of -o /dev/null.

One process, one request at a time. PHP's documentation says the built-in server "runs only one single-threaded process, so PHP applications will stall if a request is blocked". For the exercises that is harmless: a browser asks for one thing after another. It matters in exactly one place, the load test in Practical 19, and that chapter comes back to it.

Step 4: the second website

The City Library lives in its own folder, library, next to practice, and runs on the next port, 8081, in its own window: php -S localhost:8081 router.php from inside library. Its pages are printed in Practical 2, the only exercise that uses it.

Two sites on two ports are two different websites to a browser: they do not share cookies, logins or anything else, which is the point of MU's cross-website exercise.

Step 5: the browsers

Install Google Chrome and Mozilla Firefox; on Windows, Microsoft Edge is already there. Practical 4 and Practical 18 run the same test on several browsers and need at least two.

Two browser facts will matter before the first practical is over:

Selenium IDE, the tool of Practicals 1 and 2, now installs only in Firefox. Google has switched off the old kind of extension that Selenium IDE's Chrome version was built as. Practical 1 explains what to do on a Chrome-only machine.
You do not download browser drivers by hand. Since Selenium 4.6, a part of Selenium called Selenium Manager finds or downloads the right driver for your browser by itself. Chapter six shows it working.

Step 6: the map of the tools

Practicals	Tool	Installed in
1, 2	Selenium IDE	Practical 1
3	a spreadsheet: MS Excel or Google Sheets	nothing to install
4, 18	Selenium Server, which is Selenium Grid	Practical 4
5 to 16	Java, Eclipse, Maven and Selenium WebDriver	chapter six
7	Apache POI	chapter six, through Maven
14	Log4j	chapter six, through Maven
15, 16	TestNG	chapter six, through Maven
17	Jenkins	Practical 17
19	Apache JMeter	Practical 19
20	Bugzilla	Practical 20

Chapter six is Selenium WebDriver from Zero, the chapter between Practical 3 and Practical 4. One Maven project, set up there, carries every Java library the rest of the book uses, so nothing has to be downloaded twice.

The machine these notes were checked on

Every program in this book was compiled and run on Java 17, Java 21 and Java 25, against this Practice Portal, with Chromium and Firefox on Debian Linux. Chromium is the open-source browser that Google Chrome is built from, and Selenium treats the two alike. A student on Windows with Chrome sees the same results; where Windows genuinely differs, as with the file dialog in Practical 13, the chapter says so and shows both.

The Testing Laboratory from Zero: the Practice Portal, the Browsers and the Tools

Procedure

Install PHP, or find XAMPP's, and check php -v.
Make the practice folder and its seven sub-folders.
Save router.php, inc/top.php, inc/bottom.php, inc/store.php, reset.php, assets/site.css and index.php.
In the practice folder, run php -S localhost:8080 router.php and leave the window open.
Open http://localhost:8080/ in a browser and check the home page appears.
Install Chrome and Firefox.

Result

The Practice Portal was set up and served at http://localhost:8080/ by PHP's built-in web server through its router script, the home page was returned, and a page not yet added was answered with 404.

Where marks are lost

Closing the server window. The site stops the moment its window closes, and every test after that fails with a connection error that looks like a Selenium problem and is not.

Running php -S in the wrong folder. The server serves the folder it was started in. Started one level up, every address gains an extra folder name and every page is "Not found".

Putting the router first on a custom document root. php -S localhost:8080 router.php -t folder does not do what it looks like; the router must come last. The simplest way is the way shown: start inside the folder and name only the router.

Blaming the tool for a missing page. When a test cannot find an element, open the page in a browser first. "Not found" means the page was never created.

For the journal

This chapter is not one of the twenty practicals. In the front of your journal, record the laboratory: the operating system, the browsers and their versions, the PHP version, and the address of the site under test. Every later practical refers back to it.

Quick revision

Application under test (AUT): the thing being tested. Here, the Practice Portal and the City Library.
localhost means this computer; nothing goes over the internet.
Start the site: in practice, php -S localhost:8080 router.php. Keep that window open.
The City Library: in library, php -S localhost:8081 router.php.
PHP's built-in server is one single-threaded process; fine for tests, a limit for load tests.
The router answers 403 for data, seed and inc, and 404 for a page that does not exist.
reset.php puts every record back and empties uploads.
Selenium IDE installs in Firefox; Chrome's extension was switched off.
Selenium Manager finds browser drivers by itself since Selenium 4.6.

The Testing Laboratory from Zero: the Practice Portal, the Browsers and the Tools

Questions you must be able to answer

1. What is an application under test? The software being tested. In this book it is the Practice Portal, and in Practical 2 also the City Library.

2. Why test a practice site rather than a public website? Because it behaves the same every time, its source can be read, it can be loaded and filled with test records without troubling anybody, and it contains defects on purpose.

3. What does localhost:8080 mean? Port 8080 on this same computer. The browser and the site are on one machine.

4. What does a router script do for PHP's built-in server? It runs first on every request. Returning false lets the server serve the file as usual; returning true means the script has answered the request itself.

5. Why does this site's router answer 404 for a missing page? Because since PHP 8.4 the built-in server looks for an index.php when the requested page does not exist and serves that, which would make a missing page look like the home page.

6. Can PHP's built-in server handle many requests at once? No. It is a single-threaded process. On Linux and macOS extra workers can be set with PHP_CLI_SERVER_WORKERS; PHP's documentation says multiple workers are not supported on Windows.

7. A test fails with a connection refused error. What do you check first? That the server window is still open and was started in the right folder, by opening the address in a browser.

Contents This chapter on its own page

Chapter Three

Practical 1: Creating a Test Suite with Selenium IDE

Syllabus topic Module 1, "Creation of Test Suite Using Selenium IDE", "Tool: Selenium IDE (Chrome/Firefox Extension)", "Install Selenium IDE and create a test suite containing a minimum of four test cases to validate different web page formats (HTML page, XML response, JSON API output, and a dynamic form page). Execute the suite and export the automation script in WebDriver format."

Aim

To install Selenium IDE, to create a test suite of four test cases that validate four different web page formats (an HTML page, an XML response, a JSON API output and a dynamic form page), to execute the suite, and to export the automation script in WebDriver format.

What you need to know before you start

Selenium IDE is a record-and-playback tool for testing web pages. It runs as an add-on inside the browser. You either record what you do on a page, or type the steps in yourself, and the IDE plays them back and tells you whether the page behaved as expected. No programming is needed, which is why MU puts it first.

The IDE's words, which you will need in the journal:

Command. One step: open a page, click something, type into a box, assert that something is true.
Target. What the command acts on. For open it is an address; for click it is a locator, a way of finding one element on the page.
Value. A second argument when the command needs one: the text to type, or the text an assertion expects.
Test case. A named list of commands that checks one thing.
Test suite. A named group of test cases that are run together.
Project. Everything together, saved as one file whose name ends in .side.

Every command is one row: Command | Target | Value. This book writes each test case the same way, one command a line, so you can type it straight into the IDE.

Locators tell the IDE which element you mean. The ones used in this practical:

Locator	Finds	Example
`id=`	the element with that id attribute	`id=fee`
`css=`	the first element a CSS selector matches	`css=h1`
`xpath=`	the first element an XPath expression matches	one cell of the office-hours table, in the test case below

An id is the best locator when a page has one, because a page should use each id only once.

Assert against verify. Both check something. When an assert fails the test case stops there. When a verify fails, the failure is recorded and the test carries on. The IDE's own description of verify text says exactly that: the test "will continue even if the verify fails".

The four page formats MU names. A web server does not only send web pages. It says what it is sending in a Content-Type header, and the four formats are four answers:

Format	What the server sends	Content-Type on the Practice Portal
HTML page	a page for a person to read	`text/html`
XML response	data marked up with tags of its own	`application/xml`
JSON API output	data as JavaScript Object Notation, for programs	`application/json`
Dynamic form page	an HTML page whose content changes after it loads, by JavaScript	`text/html`

Practical 1: Creating a Test Suite with Selenium IDE

The first is what the IDE was built for. The next two are data, not pages, and the IDE needs a different technique for them, which is the most instructive part of this practical. The fourth changes while you look at it, which is what waits are for.

Installing Selenium IDE: the one fact that decides the browser

MU names Selenium IDE as a "Chrome/Firefox Extension". Only one of those two still exists.

Chrome. Selenium IDE's Chrome extension was built on an old extension format, Manifest V2, which Google has removed from Chrome. Google's own timeline records that from 24 July 2025, with Chrome 138, Manifest V2 extensions were disabled for all users, who "can no longer turn them back on", and that on 31 August 2026 all remaining Manifest V2 extensions were removed from the Chrome Web Store. Users reported the Selenium IDE extension switched off in January 2025, and a Selenium project member answered on the project's own issue tracker: "v4 is an Electron app, it is not a browser extension."

Firefox. The Firefox add-on, version 3.17.4, is still published on Firefox's add-ons site, and it is what this chapter uses. Every step below was done in it.

If your laboratory has only Chrome, the Selenium project's own route is Selenium IDE 4, a separate desktop program that drives Chrome, published on the project's GitHub releases page as version 4.0.1-beta.14. It is marked beta and its installers are not code-signed, so Windows warns before running one. This book's steps and screenshots are for the Firefox add-on.

To install it in Firefox:

Open Firefox and go to addons.mozilla.org.
Search for Selenium IDE and open its page.
Click Add to Firefox. Firefox lists the permissions it asks for; click Add.
A Selenium IDE icon appears among the toolbar icons, or under the jigsaw-piece Extensions button. Pin it to the toolbar so it is always one click away.
Click the icon. The IDE opens in a window of its own.

Selenium IDE's welcome window with four choices

Figure 3.1 Selenium IDE 3.17.4 opening in Firefox, as the lab's own Firefox showed it.

The version number, 3.17.4, is printed in the header. Write it in your journal.

The pages under test

Make sure the Practice Portal is running (php -S localhost:8080 router.php inside practice), then add these four files to it.

The HTML page, practice/about.php:

<?php $title = 'About'; include 'inc/top.php'; ?>
<h1>About the Practice Portal</h1>
<p id="intro">The Practice Portal is a small college website built for testing practice.</p>
<h2>Departments</h2>
<ul id="departments">
  <li>Computer Science</li>
  <li>Information Technology</li>
  <li>Commerce</li>
</ul>
<h2>Office hours</h2>
<table id="hours">
  <tr><th>Day</th><th>Time</th></tr>
  <tr><td>Monday to Friday</td><td>10:00 to 17:00</td></tr>
  <tr><td>Saturday</td><td>10:00 to 13:00</td></tr>
</table>
<?php include 'inc/bottom.php'; ?>

Practical 1: Creating a Test Suite with Selenium IDE

The XML response, practice/api/notices.php. It sends XML and says so in its Content-Type header:

<?php
// api/notices.php: the college notices, as XML.
header('Content-Type: application/xml; charset=utf-8');
echo '<?xml version="1.0" encoding="UTF-8"?>' . "\n";
?>
<notices>
  <notice id="1" date="2026-07-01">
    <title>Semester 5 lectures begin</title>
  </notice>
  <notice id="2" date="2026-07-15">
    <title>Journal submission for Practical 1</title>
  </notice>
  <notice id="3" date="2026-08-05">
    <title>Mid-term practical examination timetable</title>
  </notice>
</notices>

The XML declaration is printed with echo rather than written out, because <? at the start of a line means "PHP starts here" to some PHP installations.

The JSON API, practice/api/students.php, with the records it serves in practice/seed/students.json:

<?php
// api/students.php: the student list as JSON. ?roll=103 returns one student.
require __DIR__ . '/../inc/store.php';
header('Content-Type: application/json; charset=utf-8');
$students = load_records('students');

if (isset($_GET['roll'])) {
    foreach ($students as $s) {
        if ($s['roll'] === (int) $_GET['roll']) {
            echo json_encode($s);
            exit;
        }
    }
    http_response_code(404);
    echo json_encode(['error' => 'No student with that roll number']);
    exit;
}
echo json_encode(['count' => count($students), 'students' => $students]);

[
    {"roll": 101, "name": "Aarav Shah", "course": "B.Sc. CS", "city": "Mumbai", "marks": 72},
    {"roll": 102, "name": "Diya Nair", "course": "B.Sc. CS", "city": "Thane", "marks": 65},
    {"roll": 103, "name": "Kabir Khan", "course": "B.Sc. IT", "city": "Pune", "marks": 58},
    {"roll": 104, "name": "Meera Iyer", "course": "B.Sc. CS", "city": "Mumbai", "marks": 81},
    {"roll": 105, "name": "Rohan Patil", "course": "B.Com.", "city": "Nashik", "marks": 49},
    {"roll": 106, "name": "Sara D'Souza", "course": "B.Sc. IT", "city": "Mumbai", "marks": 77},
    {"roll": 107, "name": "Vivaan Joshi", "course": "B.Sc. CS", "city": "Thane", "marks": 90},
    {"roll": 108, "name": "Ananya Rao", "course": "B.Com.", "city": "Pune", "marks": 63},
    {"roll": 109, "name": "Ishaan Gupta", "course": "B.Sc. IT", "city": "Mumbai", "marks": 55},
    {"roll": 110, "name": "Zoya Sheikh", "course": "B.Sc. CS", "city": "Nashik", "marks": 84}
]

The dynamic form, practice/enquiry.php. The Year box is hidden until a course is chosen, the fee appears only then, and the room choice appears only when the hostel box is ticked. None of that is in the page the server sends; JavaScript does it in the browser:

<?php $title = 'Admission Enquiry'; include 'inc/top.php'; ?>
<h1>Admission Enquiry</h1>
<form id="enquiry">
  <label>Full name <input type="text" id="name"></label>
  <label>Email <input type="text" id="email"></label>
  <label>Course
    <select id="course">
      <option value="">-- Select --</option>
      <option>B.Sc. CS</option>
      <option>B.Sc. IT</option>
      <option>B.Com.</option>
    </select>
  </label>
  <p id="fee"></p>
  <label id="yearLabel" class="hidden">Year
    <select id="year">
      <option>FY</option>
      <option>SY</option>
      <option>TY</option>
    </select>
  </label>
  <label><input type="checkbox" id="hostel"> I need a hostel room</label>
  <div id="rooms" class="hidden">
    <label><input type="radio" name="room" id="single" value="Single"> Single room</label>
    <label><input type="radio" name="room" id="shared" value="Shared"> Shared room</label>
  </div>
  <p id="errors" class="error"></p>
  <button type="button" id="submit">Send enquiry</button>
</form>
<p id="confirmation" class="success"></p>
<script>
const fees = { 'B.Sc. CS': 'Rs. 38,000', 'B.Sc. IT': 'Rs. 36,000', 'B.Com.': 'Rs. 24,000' };
const course = document.getElementById('course');
course.addEventListener('change', function () {
  const chosen = course.value !== '';
  document.getElementById('fee').textContent = chosen ? 'Annual fee: ' + fees[course.value] : '';
  document.getElementById('yearLabel').classList.toggle('hidden', !chosen);
});
document.getElementById('hostel').addEventListener('change', function () {
  document.getElementById('rooms').classList.toggle('hidden', !this.checked);
});
document.getElementById('submit').addEventListener('click', function () {
  const name = document.getElementById('name').value.trim();
  const email = document.getElementById('email').value.trim();
  const problems = [];
  if (name === '') problems.push('Please enter your name.');
  if (!/^[^@\s]+@[^@\s]+\.[a-z]{2,}$/i.test(email)) problems.push('Please enter a valid email.');
  if (course.value === '') problems.push('Please choose a course.');
  document.getElementById('errors').textContent = problems.join(' ');
  if (problems.length > 0) return;
  const year = document.getElementById('year').value;
  let text = 'Thank you, ' + name + '. Your enquiry for ' + course.value + ' (' + year + ') has been recorded.';
  const room = document.querySelector('input[name="room"]:checked');
  if (document.getElementById('hostel').checked && room) text += ' Hostel: ' + room.value + ' room.';
  document.getElementById('enquiry').classList.add('hidden');
  document.getElementById('confirmation').textContent = text;
});
</script>
<?php include 'inc/bottom.php'; ?>

Practical 1: Creating a Test Suite with Selenium IDE

The IDE window

Figure 3.2 The IDE with one test case open: the test list on the left, the command table in the middle, the command editor below it and the log at the bottom.

Four areas matter:

The left panel lists the project's test cases. Its heading is a menu: Tests, Test suites or Executing, which shows what is running.
The toolbar above the table: Run all tests (a triangle with lines), Run current test (a plain triangle), Step over, the speed control, and at the right the red REC button for recording.
The playback base URL box, which holds the start of every address. With http://localhost:8080 in it, a test can say open /about.php.
The command table and, under it, the Command, Target, Value and Description boxes where the selected row is edited.

At the bottom are two tabs: Log, where a run reports every step, and Reference, which explains the command in the selected row.

Step 1: a project and its base URL

In the welcome window, click Create a new project.
The IDE asks "Name your new project". Type Practical 1 and click OK.
The project opens with one empty test case called Untitled. In the Playback base URL box, type http://localhost:8080.

Step 2: test case 1, the HTML page

Rename Untitled: hover over it in the left panel, click the three-dot button beside it, choose Rename, and call it HTML page. Then click the first empty row of the table and type each command into the Command, Target and Value boxes below it. The IDE suggests command names as you type.

Test: HTML page
open | /about.php
assert title | About - Practice Portal
assert text | css=h1 | About the Practice Portal
assert element present | id=departments
verify text | xpath=//table[@id='hours']//tr[2]/td[2] | 10:00 to 17:00

Practical 1: Creating a Test Suite with Selenium IDE

Line by line:

open /about.php loads the page. The IDE joins the base URL and the target.
assert title checks the browser's title bar text. The title comes from inc/top.php, which adds " - Practice Portal" to the page's own title.
assert text on css=h1 checks the heading.
assert element present on id=departments checks the departments list exists at all.
verify text reads one cell of the office-hours table: the second row, second column. It is a verify, so if it fails the test still finishes.

Why XPath for the cell: the table cells have no ids, so the locator describes the path to the cell instead. //table[@id='hours'] finds the table, //tr[2] its second row, and /td[2] that row's second cell.

Step 3: test case 2, the XML response

Add a new test case with the + button at the top of the left panel and call it XML response.

The obvious test is wrong, and it is worth seeing why. The obvious test opens the XML address and asserts the text of a notice's title:

Test: XML opened directly
open | /api/notices.php
assert text | xpath=//notice[@id='2']/title | Journal submission for Practical 1

Suite: Direct
XML opened directly

Played in the IDE, the open succeeds and the assertion fails with a message about the page itself, not about the text:

Running 'XML opened directly'
1. open on /api/notices.php OK
2. assertText on xpath=//notice[@id='2']/title with value Journal submission for Practical 1 Failed:
    can't access property "addEventListener", _win is null
'XML opened directly' ended with 1 error(s)

The reason: the IDE works by placing a small program of its own inside each page it tests, and an XML document is not a web page. Firefox shows it as a formatted tree, and there is no page window for the IDE's program to work in. The same is true of JSON, which Firefox shows in a viewer of its own.

The technique that works is to open an ordinary page of the same site and ask the browser to fetch the data, which is exactly what a real web page does. The IDE's execute async script command runs a piece of JavaScript in the page; in the IDE's own words, the script "must return a Promise", and its result "will be saved on the variable if you use the 'return' keyword". The assert command then checks that variable:

Test: XML response
open | /index.php
execute async script | return fetch('/api/notices.php').then(r => r.headers.get('Content-Type')) | type
assert | type | application/xml; charset=utf-8
execute async script | return fetch('/api/notices.php').then(r => r.text()).then(t => new DOMParser().parseFromString(t, 'application/xml').getElementsByTagName('notice').length) | noticeCount
assert | noticeCount | 3
execute async script | return fetch('/api/notices.php').then(r => r.text()).then(t => new DOMParser().parseFromString(t, 'application/xml').querySelector('notice[id="2"] title').textContent) | secondTitle
assert | secondTitle | Journal submission for Practical 1

Practical 1: Creating a Test Suite with Selenium IDE

Three checks, each on a different property of the response:

The format. The Content-Type header must say application/xml. A server that sent the right text with the wrong header would fail here, and many programs that read XML would refuse it.
The structure. DOMParser turns the text into an XML document, and there must be exactly three notice elements.
The content. The title inside the notice whose id is 2 must be the expected text.

fetch(...) asks the server for the address and returns a Promise; .then(r => r.text()) waits for the body; the arrow functions (r => ...) are JavaScript's short way of writing a small function. You do not need to write JavaScript anywhere else in this paper, and these lines can be copied as they are.

Step 4: test case 3, the JSON API

Add a test case called JSON API. The same technique, with r.json(), which reads the body as JSON and turns it into JavaScript objects:

Test: JSON API
open | /index.php
execute async script | return fetch('/api/students.php').then(r => r.headers.get('Content-Type')) | type
assert | type | application/json; charset=utf-8
execute async script | return fetch('/api/students.php').then(r => r.json()).then(d => d.count) | count
assert | count | 10
execute async script | return fetch('/api/students.php?roll=103').then(r => r.json()).then(s => s.name) | name
assert | name | Kabir Khan

Again three checks: the header says JSON, the count field of the full list is 10, and asking for roll number 103 alone returns the student named Kabir Khan. The IDE's own description of assert says the variable's value "will be converted to a string for comparison", which is why a number, 10, can be compared with the text 10.

Step 5: test case 4, the dynamic form

Add a test case called Dynamic form. This one fills in the enquiry form the way a person would:

Test: Dynamic form
open | /enquiry.php
type | id=name | Priya Sharma
type | id=email | priya@example.com
select | id=course | label=B.Sc. CS
wait for element visible | id=year | 5000
assert text | id=fee | Annual fee: Rs. 38,000
select | id=year | label=TY
click | id=hostel
wait for element visible | id=shared | 5000
click | id=shared
click | id=submit
assert text | id=confirmation | Thank you, Priya Sharma. Your enquiry for B.Sc. CS (TY) has been recorded. Hostel: Shared room.

The two wait for element visible lines are what make this a test of a dynamic page. The Year box does not exist on screen until a course is chosen, and the room choices do not exist until the hostel box is ticked. Each wait holds the test until the element can be seen, for at most the Value, 5000 milliseconds. Without them, a test run at full speed can reach for an element in the moment before the page's JavaScript has shown it.

Practical 1: Creating a Test Suite with Selenium IDE

select chooses from a dropdown. label=B.Sc. CS picks the option whose visible text is B.Sc. CS.

Step 6: the suite

A test suite is what MU asks for, and four separate test cases are not yet one.

Open the left panel's heading menu and choose Test suites. The IDE already shows one called Default Suite.
Click + and name the new suite Page formats.
Hover over the suite, click its three-dot button, and choose Add tests.
Tick the four test cases and click Select.

Suite: Page formats
HTML page
XML response
JSON API
Dynamic form

The suite's menu: Add tests, Rename, Delete, Settings and Export

Figure 3.3 Each suite has its own menu; Add tests is how test cases are put into it.

The suite's Settings hold its timeout, and whether its tests run in parallel or share one browser session. Leave them as they are for this practical.

Step 7: execute the suite

Click the arrow beside the suite to open it, click the first test case inside it, then click Run all tests in suite (the triangle with lines).

If you click Run before selecting a test case inside the suite, the IDE answers with a message instead of running: "To play a suite you must select a test case from within that suite." That message is where a lot of laboratory time goes.

The IDE opens a new Firefox window, plays each test case in it, and writes every step into the Log tab. This is the whole log of the suite, as the IDE printed it (the IDE also shows the time of day beside each line):

Running 'HTML page'
1. open on /about.php OK
2. assertTitle on About - Practice Portal OK
3. assertText on css=h1 with value About the Practice Portal OK
4. assertElementPresent on id=departments OK
5. verifyText on xpath=//table[@id='hours']//tr[2]/td[2] with value 10:00 to 17:00 OK
'HTML page' completed successfully
Running 'XML response'
1. open on /index.php OK
2. executeAsyncScript on return fetch('/api/notices.php').then(r => r.headers.get('Content-Type')) with value type OK
3. assert on type with value application/xml; charset=utf-8 OK
4. executeAsyncScript on return fetch('/api/notices.php').then(r => r.text()).then(t => new DOMParser().parseFromString(t, 'application/xml').getElementsByTagName('notice').length) with value noticeCount OK
5. assert on noticeCount with value 3 OK
6. executeAsyncScript on return fetch('/api/notices.php').then(r => r.text()).then(t => new DOMParser().parseFromString(t, 'application/xml').querySelector('notice[id="2"] title').textContent) with value secondTitle OK
7. assert on secondTitle with value Journal submission for Practical 1 OK
'XML response' completed successfully
Running 'JSON API'
1. open on /index.php OK
2. executeAsyncScript on return fetch('/api/students.php').then(r => r.headers.get('Content-Type')) with value type OK
3. assert on type with value application/json; charset=utf-8 OK
4. executeAsyncScript on return fetch('/api/students.php').then(r => r.json()).then(d => d.count) with value count OK
5. assert on count with value 10 OK
6. executeAsyncScript on return fetch('/api/students.php?roll=103').then(r => r.json()).then(s => s.name) with value name OK
7. assert on name with value Kabir Khan OK
'JSON API' completed successfully
Running 'Dynamic form'
1. open on /enquiry.php OK
2. type on id=name with value Priya Sharma OK
3. type on id=email with value priya@example.com OK
4. select on id=course with value label=B.Sc. CS OK
5. waitForElementVisible on id=year with value 5000 OK
6. assertText on id=fee with value Annual fee: Rs. 38,000 OK
7. select on id=year with value label=TY OK
8. click on id=hostel OK
9. waitForElementVisible on id=shared with value 5000 OK
10. click on id=shared OK
11. click on id=submit OK
12. assertText on id=confirmation with value Thank you, Priya Sharma. Your enquiry for B.Sc. CS (TY) has been recorded. Hostel: Shared room. OK
'Dynamic form' completed successfully

Practical 1: Creating a Test Suite with Selenium IDE

Read it the way an examiner will. Each test case starts with a Running line and ends with completed successfully; each numbered line is one command, with the target and value it was given, and OK means it passed. The command names in the log are the IDE's internal ones, assertTitle for "assert title" and so on.

Figure 3.4 After the run: every test case in the suite ticked green, and the end of the log below it.

Step 8: save the project

Click the Save project button at the top right (or press Ctrl+S). Firefox asks where to save; save it as practical-1.side in your practical folder. The .side file is plain text in JSON, and holds the project's name, its base URL, every test case with every command, and every suite. It is also what the command-line runner in Practical 2 plays.

Step 9: export the automation script in WebDriver format

MU's last instruction is to "export the automation script in WebDriver format". WebDriver is Selenium's programming interface, the one every later practical uses from Java; "WebDriver format" means code that does through WebDriver what the IDE did.

In the Test suites view, open the suite's three-dot menu and choose Export.
The IDE asks you to "Select language". It offers C# NUnit, C# xUnit, Java JUnit, JavaScript Mocha, Python pytest and Ruby RSpec. Choose Java JUnit.
Leave the three boxes below unticked (origin tracing comments, step descriptions as comments, and Selenium Grid).
Click Export, and save the file.

The Export dialog with its six languages and three options

Figure 3.5 Java JUnit is the one this book uses: every later practical is in Java.

This is the file the IDE wrote, exactly:

Practical 1: Creating a Test Suite with Selenium IDE

// Generated by Selenium IDE
import org.junit.Test;
import org.junit.Before;
import org.junit.After;
import static org.junit.Assert.*;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.core.IsNot.not;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.Dimension;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.interactions.Actions;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.Alert;
import org.openqa.selenium.Keys;
import java.util.*;
import java.net.MalformedURLException;
import java.net.URL;
public class PageformatsTest {
  private WebDriver driver;
  private Map<String, Object> vars;
  JavascriptExecutor js;
  @Before
  public void setUp() {
    driver = new FirefoxDriver();
    js = (JavascriptExecutor) driver;
    vars = new HashMap<String, Object>();
  }
  @After
  public void tearDown() {
    driver.quit();
  }
  @Test
  public void hTMLpage() {
    driver.get("http://localhost:8080/about.php");
    assertThat(driver.getTitle(), is("About - Practice Portal"));
    assertThat(driver.findElement(By.cssSelector("h1")).getText(), is("About the Practice Portal"));
    {
      List<WebElement> elements = driver.findElements(By.id("departments"));
      assert(elements.size() > 0);
    }
    assertThat(driver.findElement(By.xpath("//table[@id=\'hours\']//tr[2]/td[2]")).getText(), is("10:00 to 17:00"));
  }
  @Test
  public void xMLresponse() {
    driver.get("http://localhost:8080/index.php");
    vars.put("type", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/notices.php\').then(r => r.headers.get(\'Content-Type\')).then(callback).catch(callback);"));
    assertEquals(vars.get("type").toString(), "application/xml; charset=utf-8");
    vars.put("noticeCount", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/notices.php\').then(r => r.text()).then(t => new DOMParser().parseFromString(t, \'application/xml\').getElementsByTagName(\'notice\').length).then(callback).catch(callback);"));
    assertEquals(vars.get("noticeCount").toString(), "3");
    vars.put("secondTitle", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/notices.php\').then(r => r.text()).then(t => new DOMParser().parseFromString(t, \'application/xml\').querySelector(\'notice[id=\"2\"] title\').textContent).then(callback).catch(callback);"));
    assertEquals(vars.get("secondTitle").toString(), "Journal submission for Practical 1");
  }
  @Test
  public void jSONAPI() {
    driver.get("http://localhost:8080/index.php");
    vars.put("type", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/students.php\').then(r => r.headers.get(\'Content-Type\')).then(callback).catch(callback);"));
    assertEquals(vars.get("type").toString(), "application/json; charset=utf-8");
    vars.put("count", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/students.php\').then(r => r.json()).then(d => d.count).then(callback).catch(callback);"));
    assertEquals(vars.get("count").toString(), "10");
    vars.put("name", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/students.php?roll=103\').then(r => r.json()).then(s => s.name).then(callback).catch(callback);"));
    assertEquals(vars.get("name").toString(), "Kabir Khan");
  }
  @Test
  public void dynamicform() {
    driver.get("http://localhost:8080/enquiry.php");
    driver.findElement(By.id("name")).sendKeys("Priya Sharma");
    driver.findElement(By.id("email")).sendKeys("priya@example.com");
    {
      WebElement dropdown = driver.findElement(By.id("course"));
      dropdown.findElement(By.xpath("//option[. = 'B.Sc. CS']")).click();
    }
    {
      WebDriverWait wait = new WebDriverWait(driver, 5);
      wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("year")));
    }
    assertThat(driver.findElement(By.id("fee")).getText(), is("Annual fee: Rs. 38,000"));
    {
      WebElement dropdown = driver.findElement(By.id("year"));
      dropdown.findElement(By.xpath("//option[. = 'TY']")).click();
    }
    driver.findElement(By.id("hostel")).click();
    {
      WebDriverWait wait = new WebDriverWait(driver, 5);
      wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("shared")));
    }
    driver.findElement(By.id("shared")).click();
    driver.findElement(By.id("submit")).click();
    assertThat(driver.findElement(By.id("confirmation")).getText(), is("Thank you, Priya Sharma. Your enquiry for B.Sc. CS (TY) has been recorded. Hostel: Shared room."));
  }
}

How to read it:

It is a JUnit test class. JUnit is a Java testing framework; @Before runs before each test, @Test marks a test, @After runs after it.
setUp() starts Firefox with new FirefoxDriver(), because the IDE was running in Firefox. An export made from IDE 4 in Chrome says ChromeDriver instead.
Each test case became one @Test method. The IDE built the Java names from the test names: "HTML page" became the method hTMLpage and the suite "Page formats" became the class PageformatsTest. Name your tests in plain words and the Java names follow.
open became driver.get(...) with the base URL joined on; assert title became assertThat(driver.getTitle(), is(...)); type became sendKeys; execute async script became js.executeAsyncScript(...), with the IDE adding the callback that plays the Promise's role in Java.
assert element present became a block ending in assert(elements.size() > 0);. That is Java's own assert statement, and Java ignores it unless assertions are switched on when the program is started.

Practical 1: Creating a Test Suite with Selenium IDE

Before you try to run this file, read chapter six. The export was written for Selenium 3, and two of its lines use a form of WebDriverWait that Selenium 4 no longer has; compiled against today's Selenium they are errors. Chapter six sets up Java and Maven, compiles this very file, shows both errors, and fixes them.

Recording instead of typing

Everything above can also be recorded. Click REC, give the base URL when asked, and the IDE opens a browser window and writes a command for every click and every piece of typing you do in it. Click the red button again to stop. Assertions are added afterwards, by typing them in or by right-clicking an element in the recorded page and choosing a Selenium IDE assertion.

Recording is quick, but it records exactly what you did, including every misclick, and it chooses its own locators. For a practical that has to be explained in the journal, typing the commands is clearer, which is why this chapter types them.

Observations

Test case	Page format	Commands	Checks made	Result
HTML page	HTML	5	title, heading, element present, table cell	passed
XML response	XML	7	Content-Type, number of notices, one title	passed
JSON API	JSON	7	Content-Type, record count, one record	passed
Dynamic form	dynamic HTML	12	fee appears, fields appear, confirmation text	passed

Opening the XML address directly and asserting on it failed with the IDE's own error message, which is why the XML and JSON tests fetch their data from an ordinary page.

Result

A Selenium IDE 3.17.4 project with four test cases, one for each page format MU names, was created and grouped into the suite Page formats. The suite was executed and all four test cases passed. The suite was exported in WebDriver format as a Java JUnit class.

Where marks are lost

Trying to install the Chrome extension. It is switched off in Chrome and gone from the store. Use Firefox, or Selenium IDE 4 on a Chrome-only machine.

Asserting on an XML or JSON page directly. It fails with an error about the page, not the data. Fetch the data from an ordinary page and assert on the result.

Running a suite with no test selected. Select a test case inside the suite first.

No base URL. A test that says open /about.php needs the base URL to know which site.

Leaving out the waits on the dynamic form. The test may pass once and fail the next time.

Writing only the commands. The journal needs the log too: it is the proof the suite was executed.

Practical 1: Creating a Test Suite with Selenium IDE

Exporting and stopping. Name the language you exported to, and keep the exported file; chapter six runs it.

For the journal

Aim; tool and version (Selenium IDE 3.17.4 in Firefox); the four pages under test and their formats; the four test cases as command tables; the suite and its members; the log of the run; the observations table; the exported Java file; the result.

Quick revision

Selenium IDE: record and playback in the browser. Command, Target, Value.
Test case: named list of commands. Suite: named group of test cases. Project: the .side file.
Chrome disabled Manifest V2 extensions from Chrome 138 (24 July 2025); the IDE's Chrome extension is gone. Firefox add-on 3.17.4 still installs.
Base URL plus open /about.php gives the full address.
Assert stops the test on failure; verify records it and carries on.
XML and JSON: open an ordinary page, execute async script with fetch, then assert the stored variable.
Dynamic pages: wait for element visible before touching what appears later.
Run a suite: select a test inside it, then Run all tests in suite.
Export: suite menu, Export, Java JUnit. The export was written for Selenium 3.

Questions you must be able to answer

1. What are the three parts of a Selenium IDE command? The command itself, the target it acts on, and a value when it needs one, such as the text to type or the text expected.

2. What is the difference between a test case and a test suite? A test case is one named list of commands that checks one thing. A suite is a named group of test cases run together.

3. Why does this chapter use Firefox and not Chrome? Selenium IDE's Chrome extension was built on Manifest V2, which Chrome disabled for all users from Chrome 138 in July 2025. The Firefox add-on still installs.

4. What is the difference between assert text and verify text? If assert text fails, the test stops. If verify text fails, the failure is recorded and the test continues.

5. Why not open the XML address and assert on it directly? Because the IDE cannot work inside an XML document; the attempt fails with an error about the page. Fetching the XML from an ordinary page of the same site and checking the result works.

6. What does execute async script do with return? It runs the JavaScript in the page, waits for the Promise it returns, and saves the result in the variable named in the Value.

7. What would happen without the wait for element visible lines? The test could try to use the Year box or the room choice before the page's JavaScript had shown them, and fail on some runs and not others.

Practical 1: Creating a Test Suite with Selenium IDE

8. What does "export in WebDriver format" produce here? A Java JUnit class in which every test case is a @Test method that does the same steps through Selenium WebDriver.

9. The exported Java does not compile with Selenium 4. Why? It uses new WebDriverWait(driver, 5), the Selenium 3 form with a number of seconds. Selenium 4 takes a Duration instead.

Contents This chapter on its own page

Chapter Four

Practical 2: Cross-Website Functional Testing with Selenium IDE

Syllabus topic Module 1, "Cross-Website Functional Testing Using Selenium IDE", "Tool: Selenium IDE", "Develop and execute a test suite for two different websites. Perform functional operations such as clicking links, filling forms, selecting dropdown options, and verifying displayed content. Analyze test results and generate execution logs."

Aim

To develop and execute one test suite covering two different websites; to perform functional operations on them, clicking links, filling forms, selecting dropdown options and verifying displayed content; to analyse the test results; and to generate execution logs.

What you need to know before you start

Functional testing checks that each feature does what it is supposed to do, seen from outside, the way a user uses it: click this, type that, is the right thing shown. It does not look at the code. Every Selenium IDE test is a functional test.

Cross-website means one suite reaching more than one site. A Selenium IDE project has one base URL, and open with a target that starts with / is joined to it. To reach a second site, open is given the full address instead, http://localhost:8081/, and the IDE goes there. One project, one suite, two websites.

Two websites are genuinely different to a browser when their addresses differ in host or port: they share no cookies, no login and no storage. The Practice Portal is http://localhost:8080 and the City Library is http://localhost:8081.

A test result has three possible meanings, and analysing results means deciding which one you have:

The test failed because	Then
the application is wrong	it is a defect: report it
the test's expectation is wrong	the test is wrong: fix the test
the test could not do its step (page not up, wrong locator)	the environment or the script is wrong: fix that, then run again

A test that passes is only evidence if it could have failed. That is why this chapter also runs checks that fail on purpose, and reads what the IDE says.

The second website: the City Library

The City Library runs in its own folder, library, beside practice, on port 8081. Its pages share a header with three links, a search form with a category dropdown, a book page, an opening-hours table and a contact form. Create the folder with the sub-folders inc and assets, and these files.

The router, library/router.php, is the Practice Portal's with one folder less to protect:

<?php
// router.php: read only by PHP's built-in server. It refuses inc/, answers 404
// for a page that does not exist, and serves everything else as usual.
$path = parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH);
if (preg_match('#^/inc/#', $path)) {
    http_response_code(403);
    echo 'Forbidden';
    return true;
}
if ($path !== '/' && !file_exists(__DIR__ . $path)) {
    http_response_code(404);
    echo 'Not found';
    return true;
}
return false;

The top and bottom of every page, library/inc/top.php and library/inc/bottom.php. The title is built as the page's own title, a hyphen and "City Library", so the Opening Hours page is titled Opening Hours - City Library:

Practical 2: Cross-Website Functional Testing with Selenium IDE

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title><?= htmlspecialchars($title) ?> - City Library</title>
<link rel="stylesheet" href="assets/library.css">
</head>
<body>
<header>
  <span class="brand">City Library</span>
  <nav>
    <a href="index.php">Catalogue</a>
    <a href="hours.php">Opening Hours</a>
    <a href="contact.php">Contact</a>
  </nav>
</header>
<main>

</main>
<footer>City Library, a second practice website.</footer>
</body>
</html>

The catalogue, library/inc/books.php, eight real books:

<?php
// inc/books.php: the library's catalogue.
return [
    1 => ['title' => 'The Art of Software Testing', 'author' => 'Glenford J. Myers', 'category' => 'Computing', 'status' => 'Available'],
    2 => ['title' => 'The C Programming Language', 'author' => 'Brian W. Kernighan and Dennis M. Ritchie', 'category' => 'Computing', 'status' => 'Issued'],
    3 => ['title' => 'Introduction to Algorithms', 'author' => 'Thomas H. Cormen and others', 'category' => 'Computing', 'status' => 'Available'],
    4 => ['title' => 'Software Testing Techniques', 'author' => 'Boris Beizer', 'category' => 'Computing', 'status' => 'Available'],
    5 => ['title' => 'Pride and Prejudice', 'author' => 'Jane Austen', 'category' => 'Literature', 'status' => 'Available'],
    6 => ['title' => 'Godaan', 'author' => 'Munshi Premchand', 'category' => 'Literature', 'status' => 'Issued'],
    7 => ['title' => 'A Brief History of Time', 'author' => 'Stephen Hawking', 'category' => 'Science', 'status' => 'Available'],
    8 => ['title' => 'On the Origin of Species', 'author' => 'Charles Darwin', 'category' => 'Science', 'status' => 'Available'],
];

The style sheet, library/assets/library.css:

body { font-family: Georgia, serif; margin: 0; color: #2b2b2b; background: #fbf8f1; }
header { background: #6b3e26; color: #fff; padding: 12px 18px; }
header .brand { font-size: 20px; font-weight: bold; }
header nav a { color: #fce9c9; margin-left: 18px; }
main { padding: 18px; max-width: 860px; }
footer { padding: 10px 18px; color: #777; font-size: 13px; }
label { display: block; margin-top: 10px; }
input, select, textarea, button { font-size: 15px; padding: 4px 6px; }
table { border-collapse: collapse; }
th, td { border: 1px solid #c9b8a0; padding: 4px 10px; }
.thanks { color: #1b5e20; }

The search page, library/index.php. It searches title and author together, filters by category, and says how many books it found:

<?php
$books = require 'inc/books.php';
$q = trim($_GET['q'] ?? '');
$category = $_GET['category'] ?? 'All';
$searched = isset($_GET['q']);
$found = [];
foreach ($books as $id => $b) {
    $textMatch = $q === '' || stripos($b['title'] . ' ' . $b['author'], $q) !== false;
    $categoryMatch = $category === 'All' || $b['category'] === $category;
    if ($textMatch && $categoryMatch) {
        $found[$id] = $b;
    }
}
$title = 'Catalogue';
include 'inc/top.php';
?>
<h1>Search the Catalogue</h1>
<form method="get" action="index.php">
  <label>Title or author <input type="text" name="q" id="q" value="<?= htmlspecialchars($q) ?>"></label>
  <label>Category
    <select name="category" id="category">
      <?php foreach (['All', 'Computing', 'Literature', 'Science'] as $c): ?>
        <option<?= $c === $category ? ' selected' : '' ?>><?= $c ?></option>
      <?php endforeach; ?>
    </select>
  </label>
  <p><button type="submit" id="searchBtn">Search</button></p>
</form>
<?php if ($searched): ?>
  <p id="resultCount"><?= count($found) ?> book(s) found</p>
  <ul id="results">
    <?php foreach ($found as $id => $b): ?>
      <li><a href="book.php?id=<?= $id ?>"><?= htmlspecialchars($b['title']) ?></a></li>
    <?php endforeach; ?>
  </ul>
<?php endif; ?>
<?php include 'inc/bottom.php'; ?>

Practical 2: Cross-Website Functional Testing with Selenium IDE

A book's own page, library/book.php, and the opening hours, library/hours.php:

<?php
$books = require 'inc/books.php';
$id = (int) ($_GET['id'] ?? 0);
if (!isset($books[$id])) {
    http_response_code(404);
    exit('No such book');
}
$b = $books[$id];
$title = $b['title'];
include 'inc/top.php';
?>
<h1 id="title"><?= htmlspecialchars($b['title']) ?></h1>
<table>
  <tr><th>Author</th><td id="author"><?= htmlspecialchars($b['author']) ?></td></tr>
  <tr><th>Category</th><td id="bookCategory"><?= $b['category'] ?></td></tr>
  <tr><th>Status</th><td id="status"><?= $b['status'] ?></td></tr>
</table>
<p><a href="index.php">Back to the catalogue</a></p>
<?php include 'inc/bottom.php'; ?>

<?php $title = 'Opening Hours'; include 'inc/top.php'; ?>
<h1>Opening Hours</h1>
<table id="hours">
  <tr><th>Day</th><th>Hours</th></tr>
  <tr><td>Monday to Saturday</td><td>9:00 to 20:00</td></tr>
  <tr><td>Sunday</td><td>Closed</td></tr>
</table>
<?php include 'inc/bottom.php'; ?>

The contact form, library/contact.php. Sent, it answers on the same page:

<?php
$sent = $_SERVER['REQUEST_METHOD'] === 'POST';
$title = 'Contact';
include 'inc/top.php';
?>
<h1>Contact the Library</h1>
<?php if ($sent): ?>
  <p id="thanks" class="thanks">Thank you, <?= htmlspecialchars(trim($_POST['name'] ?? '')) ?>.
  We will reply to <?= htmlspecialchars(trim($_POST['email'] ?? '')) ?> within two working days.</p>
<?php else: ?>
<form method="post" action="contact.php">
  <label>Your name <input type="text" name="name" id="name"></label>
  <label>Email <input type="text" name="email" id="email"></label>
  <label>Topic
    <select name="topic" id="topic">
      <option>Membership</option>
      <option>Book request</option>
      <option>Other</option>
    </select>
  </label>
  <label>Message <textarea name="message" id="message" rows="3"></textarea></label>
  <p><button type="submit" id="sendBtn">Send</button></p>
</form>
<?php endif; ?>
<?php include 'inc/bottom.php'; ?>

Start it in a second window, inside library: php -S localhost:8081 router.php. With the Practice Portal still running on 8080, both sites are up.

Step 1: the project

Open Selenium IDE, choose Create a new project, name it Practical 2, and set the playback base URL to http://localhost:8080, the Practice Portal. The library tests will use full addresses.

Step 2: two test cases on the Practice Portal

Clicking links. Start at the home page, follow a link, check where it led, and come back through the header:

Test: Portal: links
open | /
click | linkText=About the portal
assert title | About - Practice Portal
click | linkText=Home
assert title | Home - Practice Portal

linkText= finds a link by its exact visible text. It is the natural locator for "click the link that says...".

Filling a form wrongly, on purpose. A functional test is not only the happy path. This one submits the enquiry form from Practical 1 with an email address that has no domain ending and no course chosen, and checks the page refuses both:

Test: Portal: enquiry errors
open | /enquiry.php
type | id=name | Priya Sharma
type | id=email | priya@example
click | id=submit
assert text | id=errors | Please enter a valid email. Please choose a course.

Practical 2: Cross-Website Functional Testing with Selenium IDE

Step 3: three test cases on the City Library

Searching, with a dropdown, then following a result:

Test: Library: search
open | http://localhost:8081/
type | id=q | testing
select | id=category | label=Computing
click | id=searchBtn
assert text | id=resultCount | 2 book(s) found
click | linkText=The Art of Software Testing
assert text | id=author | Glenford J. Myers
assert text | id=status | Available

The search matches "testing" in two Computing titles, The Art of Software Testing and Software Testing Techniques. The test then opens the first result and checks the displayed author and status: two pieces of displayed content, verified.

The contact form, filled completely:

Test: Library: contact form
open | http://localhost:8081/contact.php
type | id=name | Priya Sharma
type | id=email | priya@example.com
select | id=topic | label=Book request
type | id=message | Please add Software Testing Techniques.
click | id=sendBtn
assert text | id=thanks | Thank you, Priya Sharma. We will reply to priya@example.com within two working days.

The page's source breaks the thank-you line in two and indents the second half. The IDE reads the text as the browser shows it, with the break and the indentation collapsed into single spaces, which is why one line of expected text matches.

Navigation through the header, and a table cell:

Test: Library: opening hours
open | http://localhost:8081/
click | linkText=Opening Hours
assert title | Opening Hours - City Library
verify text | xpath=//table[@id='hours']//tr[3]/td[2] | Closed

Step 4: the suite

In the Test suites view add a suite called Two websites and, through its menu's Add tests, put all five test cases in it:

Suite: Two websites
Portal: links
Portal: enquiry errors
Library: search
Library: contact form
Library: opening hours

Then open the suite, select its first test case and click Run all tests in suite.

Step 5: the execution log

This is the IDE's log of the whole suite:

Running 'Portal: links'
1. open on / OK
2. click on linkText=About the portal OK
3. assertTitle on About - Practice Portal OK
4. click on linkText=Home OK
5. assertTitle on Home - Practice Portal OK
'Portal: links' completed successfully
Running 'Portal: enquiry errors'
1. open on /enquiry.php OK
2. type on id=name with value Priya Sharma OK
3. type on id=email with value priya@example OK
4. click on id=submit OK
5. assertText on id=errors with value Please enter a valid email. Please choose a course. OK
'Portal: enquiry errors' completed successfully
Running 'Library: search'
1. open on http://localhost:8081/ OK
2. type on id=q with value testing OK
3. select on id=category with value label=Computing OK
4. click on id=searchBtn OK
5. assertText on id=resultCount with value 2 book(s) found OK
6. click on linkText=The Art of Software Testing OK
7. assertText on id=author with value Glenford J. Myers OK
8. assertText on id=status with value Available OK
'Library: search' completed successfully
Running 'Library: contact form'
1. open on http://localhost:8081/contact.php OK
2. type on id=name with value Priya Sharma OK
3. type on id=email with value priya@example.com OK
4. select on id=topic with value label=Book request OK
5. type on id=message with value Please add Software Testing Techniques. OK
6. click on id=sendBtn OK
7. assertText on id=thanks with value Thank you, Priya Sharma. We will reply to priya@example.com within two working days. OK
'Library: contact form' completed successfully
Running 'Library: opening hours'
1. open on http://localhost:8081/ OK
2. click on linkText=Opening Hours OK
3. assertTitle on Opening Hours - City Library OK
4. verifyText on xpath=//table[@id='hours']//tr[3]/td[2] with value Closed OK
'Library: opening hours' completed successfully

Practical 2: Cross-Website Functional Testing with Selenium IDE

All five test cases completed successfully: two on one website, three on the other, in one run of one suite.

The IDE has no button that saves the log to a file. For the journal, select the log text and copy it, or use the command-line runner below, which prints the same kind of record every time it runs.

Step 6: analysing results, with failures made on purpose

A result can only be analysed if you know what a failure looks like. So add two more test cases, each designed to fail, and put them in a suite of their own called Analysis, so that the real suite stays clean:

Test: Analysis: wrong expectation
open | http://localhost:8081/hours.php
verify text | xpath=//table[@id='hours']//tr[3]/td[2] | 10:00 to 14:00
assert title | Opening Hours - City Library

Test: Analysis: partial title
open | /about.php
assert title | About

Suite: Analysis
Analysis: wrong expectation
Analysis: partial title

Run the Analysis suite. The IDE's log:

Running 'Analysis: wrong expectation'
1. open on http://localhost:8081/hours.php OK
2. verifyText on xpath=//table[@id='hours']//tr[3]/td[2] with value 10:00 to 14:00 Failed:
    Actual value "Closed" did not match "10:00 to 14:00"
3. assertTitle on Opening Hours - City Library OK
'Analysis: wrong expectation' ended with 1 error(s)
Running 'Analysis: partial title'
1. open on /about.php OK
2. assertTitle on About Failed:
    Actual value 'About - Practice Portal' did not match 'About'
'Analysis: partial title' ended with 1 error(s)

Read each failure as the table at the start of the chapter asks:

Wrong expectation. The verify expected Sunday's hours to be 10:00 to 14:00 and the page says Closed. The page is right: the library is closed on Sundays. The test is wrong. Notice what verify did: it recorded the failure and went on, so the next command, the title check, still ran and passed, and the test case ended with one error rather than stopping.

Partial title. The test asserted the title "About" and the IDE failed it, reporting the actual title in full. This is worth knowing exactly, because the IDE's own Reference text describes assert title as confirming that the title "contains the provided text". The run shows the comparison is of the whole text: "About" is not the whole title, so the assertion fails. The test is wrong; the full title is the expected value.

Practical 2: Cross-Website Functional Testing with Selenium IDE

Neither failure is a defect in either website. That is the most common result of analysing a failed test, and the reason an unanalysed "failed" in a journal is worth nothing.

Step 7: execution logs from the command line

The Selenium project publishes a command-line runner, selenium-side-runner, that plays a saved .side project without opening the IDE, prints a log of every test, and can run the same project in a different browser. It needs Node.js, and is installed once with npm install -g selenium-side-runner, the command given in the Selenium IDE documentation.

Save the project as practical-2.side (Save project, at the top right of the IDE). Then, in a terminal, in the folder where you saved it, run only the Two websites suite in Firefox. The runner's own help describes -c as "Webdriver capabilities" and -f as "Run suites matching name":

$ selenium-side-runner -c "browserName=firefox" -f "Two websites" practical-2.side
info: Running test Portal: links
info: Finished test Portal: links Success
info: Running test Portal: enquiry errors
info: Finished test Portal: enquiry errors Success
info: Running test Library: search
info: Finished test Library: search Success
info: Running test Library: contact form
info: Finished test Library: contact form Success
info: Running test Library: opening hours
info: Finished test Library: opening hours Success
PASS ../../../opt/side/node_modules/selenium-side-runner/dist/main.test.js (20.873 s)
  Running project practical-2
    Running suite Two websites
      [tick] Running test Portal: links (6277 ms)
      [tick] Running test Portal: enquiry errors (3656 ms)
      [tick] Running test Library: search (3043 ms)
      [tick] Running test Library: contact form (2958 ms)
      [tick] Running test Library: opening hours (2887 ms)

Test Suites: 1 passed, 1 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        20.946 s
Ran all test suites within paths "/opt/side/node_modules/selenium-side-runner/dist/main.test.js".
Jest did not exit one second after the test run has completed.

'This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with `--detectOpenHandles` to troubleshoot this issue.

That is an execution log in the ordinary sense: every test, its result, how long it took, and a total. The path on the PASS line is where the runner was installed, and the timings will differ on your machine. Jest, the test framework the runner is built on, marks each passed test with a tick, which these notes print as [tick]. The last lines are a note from Jest that something was still open when the run ended; the results above it are complete, and the note does not change them.

Practical 2: Cross-Website Functional Testing with Selenium IDE

The same project plays in Chrome with one word changed. This is how a Selenium IDE suite reaches Chrome now that the IDE itself cannot be installed there:

$ selenium-side-runner -c "browserName=chrome" -f "Two websites" practical-2.side
info: Running test Portal: links
info: Finished test Portal: links Success
info: Running test Portal: enquiry errors
info: Finished test Portal: enquiry errors Success
info: Running test Library: search
info: Finished test Library: search Success
info: Running test Library: contact form
info: Finished test Library: contact form Success
info: Running test Library: opening hours
info: Finished test Library: opening hours Success
PASS ../../../opt/side/node_modules/selenium-side-runner/dist/main.test.js (10.73 s)
  Running project practical-2
    Running suite Two websites
      [tick] Running test Portal: links (2580 ms)
      [tick] Running test Portal: enquiry errors (1871 ms)
      [tick] Running test Library: search (2294 ms)
      [tick] Running test Library: contact form (2401 ms)
      [tick] Running test Library: opening hours (1105 ms)

Test Suites: 1 passed, 1 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        10.767 s, estimated 21 s
Ran all test suites within paths "/opt/side/node_modules/selenium-side-runner/dist/main.test.js".
Jest did not exit one second after the test run has completed.

'This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with `--detectOpenHandles` to troubleshoot this issue.

The runner also writes its results as a file when asked: its help describes -o as "Write test results as json to file in specified directory. Name will be based on timestamp." That file is the log to keep when a suite is run every night.

Observations

Test case	Website	Operation	Result
Portal: links	Practice Portal	clicking links, verifying titles	passed
Portal: enquiry errors	Practice Portal	filling a form, verifying the error text	passed
Library: search	City Library	filling a form, a dropdown, clicking a result, verifying content	passed
Library: contact form	City Library	filling a form, a dropdown, verifying the reply	passed
Library: opening hours	City Library	clicking a link, verifying a title and a table cell	passed

The deliberate failures: a verify with a wrong expected value failed and let the test continue; an assert on part of a title failed, showing that the IDE compares the whole title. Both were errors in the test, not defects in the websites. Played by the command-line runner, the suite passed in Firefox and in Chrome.

Result

A single Selenium IDE suite of five test cases was developed and executed against two different websites, covering clicking links, filling forms, selecting dropdown options and verifying displayed content. All five passed. Two deliberately failing test cases were analysed and both failures were traced to the tests. Execution logs were produced by the IDE's log panel and by the command-line runner, in Firefox and in Chrome.

Practical 2: Cross-Website Functional Testing with Selenium IDE

Where marks are lost

Two separate projects instead of one suite. MU asks for "a test suite for two different websites". One suite, two sites.

A relative address for the second site. open /contact.php goes to the base URL's site. The second site needs its full address.

Calling every failure a defect. Decide first whether the application, the test or the environment is wrong, and say which in the journal.

Asserting part of a text. The IDE compares the whole text, whatever its Reference tab says.

No log in the journal. "Executed successfully" with no log is not evidence of anything.

Forgetting the second server. If the library tests fail at their first step, check that php -S localhost:8081 is still running in its own window.

For the journal

Aim; tool and version; the two websites and their addresses; the five test cases as command tables, with the operation each performs; the suite; the IDE's execution log; the two deliberate failures, their log and your analysis of each; the command-line runner's log; observations; result.

Quick revision

Functional testing: does each feature behave correctly for the user.
Cross-website in one project: base URL for the first site, full addresses in open for the second.
linkText= clicks a link by its visible text; select with label= picks a dropdown option.
A failed test means the application, the test or the environment is wrong. Decide which.
verify fails and continues; assert fails and stops.
assert title compares the whole title.
The IDE log has no save button; copy it, or use selenium-side-runner.
selenium-side-runner -c "browserName=chrome" project.side plays an IDE project in Chrome.

Questions you must be able to answer

1. How can one Selenium IDE suite test two websites when a project has one base URL? Tests of the base URL's site open relative addresses; tests of the other site open full addresses such as http://localhost:8081/.

2. What makes two websites different to a browser? A different host or port in the address. They share no cookies or login.

3. What are the three things a failed test can mean? The application is wrong, the test's expectation is wrong, or the environment or script could not do the step.

4. In the wrong-expectation test, why did the title check still run? The failing check was a verify, which records the failure and lets the test continue.

5. Why did asserting the title "About" fail on a page titled About - Practice Portal? The IDE compares the whole title, so a part of it does not match.

Practical 2: Cross-Website Functional Testing with Selenium IDE

6. How do you keep a log of a suite run? Copy the IDE's log panel, or run the saved project with selenium-side-runner, which prints every test's result, and with -o writes the results to a JSON file.

7. How can an IDE suite be run in Chrome? Save the project and play it with selenium-side-runner -c "browserName=chrome".

Contents This chapter on its own page

Chapter Five

Practical 3: Manual Test Case Design and Execution

Syllabus topic Module 1, "Manual Test Case Design and Execution", "Tool: MS Excel / Google Sheets", "Prepare a Test Plan, Test Scenarios, and Test Cases for a given web application. Execute test cases manually and prepare a Test Execution Report and Defect Report as per STLC process."

Aim

To prepare a test plan, test scenarios and test cases for a given web application, to execute the test cases manually, and to prepare a test execution report and a defect report, following the software testing life cycle.

What you need to know before you start

This practical has no automation in it at all. Its tool is a spreadsheet, MS Excel or Google Sheets, and its real subject is the documents testing produces. Every later practical automates something; this one teaches what is being automated.

The software testing life cycle (STLC) is the order in which testing work is done. The ISTQB Foundation syllabus, the syllabus most testing certifications in industry follow, describes the test process as groups of activities:

ISTQB activity	What happens	What it produces in this practical
Test planning	objectives and approach are decided	the test plan
Test analysis	the requirements are studied for what to test	the test scenarios
Test design	scenarios become concrete test cases	the test cases
Test implementation	test data and environment are made ready	the valid values, a running site
Test execution	the tests are run and actual results recorded	the actual results, pass or fail
Test completion	results are summarised and handed over	the execution report, the defect report

Test monitoring and control runs alongside all of them: checking progress against the plan and acting on it. The syllabus sums up analysis as answering "what to test?" and design as answering "how to test?".

Many college manuals name the same life cycle in six phases: requirement analysis, test planning, test case development, test environment setup, test execution, and test closure. They are the same work under different headings; use whichever your teacher uses, and say which.

The five documents:

Test plan. What will be tested, how, by whom, with what, and when testing is finished.
Test scenario. One thing to be tested, in a line: "Mobile number must be exactly ten digits."
Test case. One concrete check of a scenario: exact input, exact steps, exact expected result.
Test execution report. What was run and what happened, in figures.
Defect report. One report for each defect found, with enough information for somebody else to reproduce it.

Two design techniques decide which test cases are worth writing. They belong to the theory paper, and here they are used, not studied:

Equivalence partitioning. Inputs that the program should treat the same way form a partition, and one value from each partition is enough. For a ten-digit mobile number the partitions include "ten digits starting 6 to 9" (valid), "nine digits" (invalid) and "starting with 5" (invalid).
Boundary value analysis. Defects collect at the edges of partitions, so test the edge values themselves: for "3 to 40 characters", test 2, 3, 40 and 41.

Practical 3: Manual Test Case Design and Execution

The ISTQB syllabus says of boundary values that when two elements belong to the same partition, all elements between them must also belong to it, which is why the edges are where a wrong comparison shows itself.

The given web application

The application under test is the Student Registration page of the Practice Portal, register.php, which the browser opens at localhost:8080/register.php. Its source is printed at the end of this chapter, for setting the practical up. Do not read it before testing. A tester works from the requirements, which are the promise, and not from the code, which is the attempt.

The requirements given to the tester:

ID	Requirement
R1	Full name is required: letters and spaces only, 3 to 40 characters.
R2	Email is required and must be a valid email address.
R3	Mobile number is required: exactly 10 digits, the first of them 6, 7, 8 or 9.
R4	Date of birth is required, as YYYY-MM-DD, and must be a real date. The student must be at least 17 years old on 1 July 2026.
R5	A course must be chosen: B.Sc. CS, B.Sc. IT or B.Com.
R6	Password is required: at least 8 characters, including at least one digit.
R7	Confirm password must be the same as Password.
R8	The "I agree to the terms" box must be ticked.
R9	On success the page shows "Registration successful. Your registration number is REG-2026-0001.", the number counting up with each registration.
R10	On failure every problem is listed, and the page does not register the student.

Step 1: the test plan

A test plan for a practical is one page. Its headings follow the ISTQB syllabus's list of what a test plan typically contains: the context of testing (scope, objectives, test basis), assumptions and constraints, stakeholders, communication, a risk register, the test approach, and budget and schedule.

Heading	Test plan TP-01
Objective	To find out whether the Student Registration page meets requirements R1 to R10.
Test basis	The requirements table R1 to R10.
In scope	Every field of the registration form, its validation, the success message and the error list.
Out of scope	The rest of the Practice Portal, speed, security, appearance on phones.
Approach	Manual black-box testing. Test cases designed by equivalence partitioning and boundary value analysis.
Test environment	The Practice Portal's `register.php` on `localhost:8080`, served by PHP 8.4's built-in server; Firefox.
Test data	One set of valid values; each test case changes one field.
Entry criteria	The page loads; the requirements are available.
Exit criteria	Every test case executed; every failure reported as a defect.
Deliverables	Test scenarios, test cases, test execution report, defect report.
Roles	Tester: the student. Developer and reviewer: the teacher.
Risks	Requirements may be unclear; the site may not be running during testing.
Schedule	One laboratory session of two hours.

Practical 3: Manual Test Case Design and Execution

Step 2: the test scenarios

One scenario for each thing to be tested, each traced to its requirement:

ID	Scenario	Requirement
TS01	A student with valid details can register.	R9
TS02	Full name length and characters are validated.	R1
TS03	Email format is validated.	R2
TS04	Mobile number length and first digit are validated.	R3
TS05	Date of birth is validated, and age is checked against 1 July 2026.	R4
TS06	A course must be chosen.	R5
TS07	Password rules and confirmation are enforced.	R6, R7
TS08	The terms must be accepted.	R8
TS09	Every problem is listed at once.	R10

Tracing each scenario to a requirement is what makes a gap visible: a requirement with no scenario is a requirement nobody tested.

Step 3: the test cases

Every test case starts from the same valid values and changes only what it is testing. Using one baseline means a failure can have only one cause.

Field	Valid value
Full name	Priya Sharma
Email	priya@example.com
Mobile number	9876543210
Date of birth	2005-04-12
Course	B.Sc. CS
Password	Secret123
Confirm password	Secret123
Terms	ticked

The common steps for every case are: open the registration page, enter the valid values, change the field named in the test data, click Register, and read the page.

The boundary cases, and why each value was chosen:

Full name: 2 and 3 characters are either side of the minimum; a 40-character name and a 41-character name are either side of the maximum.
Mobile number: 9, 10 and 11 digits; a first digit of 5, just outside the allowed 6 to 9, and of 6, just inside.
Date of birth: born on 1 July 2009 is exactly 17 on 1 July 2026, the first valid date; born on 2 July 2009 is one day short.
Password: 7 characters and 8 characters.

Step 4: execute them, and record what happened

Execute each case by hand, in order, on a freshly reset site (localhost:8080/reset.php), and write down what the page actually said. The order matters for one reason only: each accepted registration takes the next registration number.

ID	Test data	Expected result	Actual result	Status
TC01	The valid values	Accepted	Accepted: REG-2026-0001	Pass
TC02	Full name: Pr	Rejected: Name must be at least 3 letters, using letters and spaces only.	Rejected: Name must be at least 3 letters, using letters and spaces only.	Pass
TC03	Full name: Pri	Accepted (exactly 3 characters)	Accepted: REG-2026-0002	Pass
TC04	Full name: Sai Venkata Lakshmi Narasimhan Chowdhary	Accepted (exactly 40 characters)	Accepted: REG-2026-0003	Pass
TC05	Full name: Sri Venkata Lakshmi Subramaniam Chowdhary	Rejected (41 characters)	Accepted: REG-2026-0004	Fail
TC06	Full name: Priya2	Rejected: Name must be at least 3 letters, using letters and spaces only.	Rejected: Name must be at least 3 letters, using letters and spaces only.	Pass
TC07	Email: priya.example.com	Rejected: Please enter a valid email address.	Rejected: Please enter a valid email address.	Pass
TC08	Mobile number: 987654321	Rejected (only 9 digits)	Accepted: REG-2026-0005	Fail
TC09	Mobile number: 98765432101	Rejected: Mobile number must be 10 digits and start with 6, 7, 8 or 9.	Rejected: Mobile number must be 10 digits and start with 6, 7, 8 or 9.	Pass
TC10	Mobile number: 5876543210	Rejected: Mobile number must be 10 digits and start with 6, 7, 8 or 9.	Rejected: Mobile number must be 10 digits and start with 6, 7, 8 or 9.	Pass
TC11	Mobile number: 6000000000	Accepted (lowest allowed first digit)	Accepted: REG-2026-0006	Pass
TC12	Date of birth: 2009-07-01	Accepted (exactly 17 on 1 July 2026)	Accepted: REG-2026-0007	Pass
TC13	Date of birth: 2009-07-02	Rejected (one day short of 17)	Accepted: REG-2026-0008	Fail
TC14	Date of birth: 2010-01-01	Rejected: You must be at least 17 years old on 1 July 2026.	Rejected: You must be at least 17 years old on 1 July 2026.	Pass
TC15	Date of birth: 2009-02-30	Rejected: Please enter a real date of birth as YYYY-MM-DD.	Rejected: Please enter a real date of birth as YYYY-MM-DD.	Pass
TC16	Course: (none)	Rejected: Please choose a course.	Rejected: Please choose a course.	Pass
TC17	Password: Secret1; Confirm password: Secret1	Rejected: Password must be at least 8 characters and contain a digit.	Rejected: Pasword must be at least 8 characters and contain a digit.	Fail
TC18	Password: Secret12; Confirm password: Secret12	Accepted (exactly 8 characters)	Accepted: REG-2026-0009	Pass
TC19	Password: SecretPass; Confirm password: SecretPass	Rejected (no digit)	Rejected: Pasword must be at least 8 characters and contain a digit.	Pass
TC20	Confirm password: Secret999	Rejected (the two passwords differ)	Accepted: REG-2026-0010	Fail
TC21	Terms: not ticked	Rejected: You must accept the terms to register.	Rejected: You must accept the terms to register.	Pass
TC22	Every field left empty	Rejected (every problem listed)	Rejected: Name must be at least 3 letters, using letters and spaces only. / Please enter a valid email address. / Mobile number must be 10 digits and start with 6, 7, 8 or 9. / Please enter a real date of birth as YYYY-MM-DD. / Please choose a course. / Pasword must be at least 8 characters and contain a digit. / You must accept the terms to register.	Pass

Practical 3: Manual Test Case Design and Execution

Three things to notice in how the table was filled:

Practical 3: Manual Test Case Design and Execution

Expected results were written before execution. An expected result written after seeing the page is not an expectation.
A pass or fail is a comparison, not an opinion: TC19 passes because the page did reject a password with no digit, which is all it tested, even though the message is misspelled. The misspelling is TC17's finding, and one defect is reported once.
TC22's list is the page's own, in its own order, and it is where requirement R10 is tested: seven fields wrong, seven problems listed.

Step 5: the test execution report

Measure	Count
Test cases passed	17
Test cases failed	5
Test cases blocked	0
Total	22

All 22 test cases designed were executed. The pass rate is 17 of 22, about 77.3 per cent. Five test cases failed, and each failure is one defect.

Scenario	Cases	Passed	Failed
TS01 Valid registration	1	1	0
TS02 Full name	5	4	1
TS03 Email	1	1	0
TS04 Mobile number	4	3	1
TS05 Date of birth	4	3	1
TS06 Course	1	1	0
TS07 Password	4	2	2
TS08 Terms	1	1	0
TS09 Every problem listed	1	1	0
Total	22	17	5

A report ends with a recommendation. Here: the page does not meet requirements R1, R3, R4, R6 and R7 in full, and should not be released until the five defects below are fixed and their test cases pass on a second run. Running the failed cases again after a fix is called confirmation testing, and running the passed ones again, to check the fix broke nothing, is regression testing.

Step 6: the defect report

The ISTQB syllabus lists what a defect report typically includes: a unique identifier, a title with a short summary, the date, the author, the test object and environment, the context (such as the test case being run), a description that lets somebody reproduce the failure, expected and actual results, severity, priority, status, and references.

Severity is how bad the defect's effect is. Priority is how soon it should be fixed. They are not the same: a spelling mistake on a college's home page is trivial in severity and may still be high in priority.

DEF-01: Mobile number with 9 digits is accepted

Test case: TC08. Requirement: R3.
Environment: Practice Portal, register.php, PHP 8.4 built-in server, Firefox.
Steps: open the registration page; enter the valid values; change Mobile number to 987654321; click Register.
Expected: the registration is rejected, with the mobile number message.
Actual: "Registration successful. Your registration number is REG-2026-0005."
Severity: Major. Priority: High. Status: New.

DEF-02: Mismatched passwords are accepted

Test case: TC20. Requirement: R7.
Steps: valid values; change Confirm password to Secret999; click Register.
Expected: rejected, because Confirm password differs from Password.
Actual: registered as REG-2026-0010. No message about the passwords appears at all.
Severity: Major. Priority: High. Status: New.

Practical 3: Manual Test Case Design and Execution

DEF-03: A student one day under 17 on 1 July 2026 is accepted

Test case: TC13. Requirement: R4.
Steps: valid values; change Date of birth to 2009-07-02; click Register.
Expected: rejected; the student is 16 on 1 July 2026.
Actual: registered as REG-2026-0008. (Born 2010-01-01 is correctly rejected, TC14, so the age check exists but is wrong near the boundary.)
Severity: Major. Priority: Medium. Status: New.

DEF-04: Full name longer than 40 characters is accepted

Test case: TC05. Requirement: R1.
Steps: valid values; change Full name to a 41-character name, Sri Venkata Lakshmi Subramaniam Chowdhary; click Register.
Expected: rejected; the maximum is 40.
Actual: registered as REG-2026-0004.
Severity: Minor. Priority: Low. Status: New.

DEF-05: Spelling mistake in the password message

Test case: TC17. Requirement: R6.
Steps: valid values; change Password and Confirm password to Secret1; click Register.
Expected: the message "Password must be at least 8 characters and contain a digit."
Actual: "Pasword must be at least 8 characters and contain a digit."
Severity: Trivial. Priority: Low. Status: New.

Severity	Defects
Major	3
Minor	1
Trivial	1
Total	5

A good defect title says what is wrong in the user's terms, not what the tester thinks the code does: "Mobile number with 9 digits is accepted", not "validation regex wrong". Practical 20 logs these same five defects in Bugzilla.

Doing it in Excel or Google Sheets

Keep the practical in one workbook with one sheet for each document: Plan, Scenarios, Test cases, Execution report, Defects. On the Test cases sheet, put the columns in this order: ID, Scenario, Test data, Steps, Expected result, Actual result, Status.

Make the Status column a dropdown of Pass, Fail, Blocked and Not run, so that nobody types "passed" in one row and "PASS" in another; both programs call this data validation. Then the execution report can count for itself. Microsoft's own description of COUNTIF is COUNTIF(range, criteria), where the criteria can be a word such as "apples". With the statuses in cells G2 to G23:

Passed:   =COUNTIF(G2:G23,"Pass")
Failed:   =COUNTIF(G2:G23,"Fail")
Blocked:  =COUNTIF(G2:G23,"Blocked")
Total:    =COUNTA(G2:G23)

A report that counts its own figures cannot drift from the table it reports on, which is the whole point of doing it in a spreadsheet rather than typing the numbers.

The page's source, for setting the practical up

Read this only after the test cases are executed. Save it as practice/register.php, and create registrations.json in practice/seed, holding just [], an empty list:

<?php
// register.php: the student registration form tested by hand in Practical 3.
// NOTE: this page carries deliberate defects, so that a tester has something to find.
require 'inc/store.php';
$courses = ['B.Sc. CS', 'B.Sc. IT', 'B.Com.'];
$v = ['name' => '', 'email' => '', 'mobile' => '', 'dob' => '', 'course' => ''];
$errors = [];
$success = '';

if ($_SERVER['REQUEST_METHOD'] === 'POST') {
    foreach ($v as $key => $unused) {
        $v[$key] = trim($_POST[$key] ?? '');
    }
    $password = $_POST['password'] ?? '';
    $confirm = $_POST['confirm'] ?? '';

    if (!preg_match('/^[A-Za-z ]{3,}$/', $v['name'])) {
        $errors[] = 'Name must be at least 3 letters, using letters and spaces only.';
    }
    if (!filter_var($v['email'], FILTER_VALIDATE_EMAIL)) {
        $errors[] = 'Please enter a valid email address.';
    }
    if (!preg_match('/^[6-9][0-9]{8,9}$/', $v['mobile'])) {
        $errors[] = 'Mobile number must be 10 digits and start with 6, 7, 8 or 9.';
    }
    $d = explode('-', $v['dob']);
    if (count($d) !== 3 || !ctype_digit(implode('', $d)) || !checkdate((int) $d[1], (int) $d[2], (int) $d[0])) {
        $errors[] = 'Please enter a real date of birth as YYYY-MM-DD.';
    } elseif (2026 - (int) substr($v['dob'], 0, 4) < 17) {
        $errors[] = 'You must be at least 17 years old on 1 July 2026.';
    }
    if (!in_array($v['course'], $courses, true)) {
        $errors[] = 'Please choose a course.';
    }
    if (strlen($password) < 8 || !preg_match('/\d/', $password)) {
        $errors[] = 'Pasword must be at least 8 characters and contain a digit.';
    }
    if (!isset($_POST['terms'])) {
        $errors[] = 'You must accept the terms to register.';
    }

    if (count($errors) === 0) {
        $all = load_records('registrations');
        $all[] = ['name' => $v['name'], 'email' => $v['email'], 'course' => $v['course']];
        save_records('registrations', $all);
        $success = sprintf('Registration successful. Your registration number is REG-2026-%04d.', count($all));
    }
}
$title = 'Student Registration';
include 'inc/top.php';
?>
<h1>Student Registration</h1>
<?php if ($success !== ''): ?>
  <p id="success" class="success"><?= htmlspecialchars($success) ?></p>
<?php else: ?>
  <?php if ($errors): ?>
    <ul id="errors" class="error">
      <?php foreach ($errors as $e): ?><li><?= htmlspecialchars($e) ?></li><?php endforeach; ?>
    </ul>
  <?php endif; ?>
  <form method="post" action="register.php">
    <label>Full name <input type="text" name="name" id="name" value="<?= htmlspecialchars($v['name']) ?>"></label>
    <label>Email <input type="text" name="email" id="email" value="<?= htmlspecialchars($v['email']) ?>"></label>
    <label>Mobile number <input type="text" name="mobile" id="mobile" value="<?= htmlspecialchars($v['mobile']) ?>"></label>
    <label>Date of birth (YYYY-MM-DD) <input type="text" name="dob" id="dob" value="<?= htmlspecialchars($v['dob']) ?>"></label>
    <label>Course
      <select name="course" id="course">
        <option value="">-- Select --</option>
        <?php foreach ($courses as $c): ?>
          <option<?= $v['course'] === $c ? ' selected' : '' ?>><?= htmlspecialchars($c) ?></option>
        <?php endforeach; ?>
      </select>
    </label>
    <label>Password <input type="password" name="password" id="password"></label>
    <label>Confirm password <input type="password" name="confirm" id="confirm"></label>
    <label><input type="checkbox" name="terms" id="terms"> I agree to the terms</label>
    <p><button type="submit" id="registerBtn">Register</button></p>
  </form>
<?php endif; ?>
<?php include 'inc/bottom.php'; ?>

Practical 3: Manual Test Case Design and Execution

[]

Now read the code against the defect report, which is what a developer will do. Each defect is one line:

DEF-01 is the mobile pattern [0-9]{8,9} after the first digit: eight or nine more digits, so nine digits in all passes. It should be exactly nine more.
DEF-02 is an absence: $confirm is read and never compared with $password.
DEF-03 is 2026 - (int) substr($v['dob'], 0, 4) < 17, which compares only years. Anybody born in 2009 counts as 17, whatever the month.
DEF-04 is the name pattern {3,}: at least three, with no upper limit. It should be {3,40}.
DEF-05 is a typing mistake inside a message.

Practical 3: Manual Test Case Design and Execution

Black-box testing found all five without reading a line of this, which is the point of it: the tests came from the requirements.

Result

A test plan, nine test scenarios and twenty-two test cases were prepared for the Student Registration page and executed manually. Seventeen passed and five failed. The test execution report and a defect report of five defects, three major, one minor and one trivial, were prepared. The page does not meet requirements R1, R3, R4, R6 and R7 in full.

Where marks are lost

Expected results written after execution. Write them first, or the test proves nothing.

Changing two fields in one test case. When it fails, you cannot say which field caused it.

No boundary values. "Mobile number: 12345" tests one partition; the defects are at 9 and 11 digits.

Reporting the same defect twice. The misspelled message appears in three test cases; it is one defect.

A defect report nobody can reproduce. Steps, data, expected and actual, every time.

Mixing up severity and priority. Severity is impact; priority is urgency.

Typed-in report figures. Count them from the Status column.

For the journal

Aim; the application under test and its requirements; the test plan; the test scenarios with their requirements; the valid values; the test case table with actual results and status; the test execution report; the defect report; the result. Attach a printout of the spreadsheet if your college asks for one.

Quick revision

STLC in ISTQB's terms: planning, analysis, design, implementation, execution, completion, with monitoring and control throughout.
Test plan: scope, approach, environment, entry and exit criteria, deliverables, risks.
Scenario: what to test, traced to a requirement. Test case: exact data, steps, expected result.
Equivalence partitioning: one value per partition. Boundary values: the edges, on both sides.
Expected result before execution. Pass or fail is a comparison.
Defect report: ID, title, steps, expected, actual, severity, priority, status, test case.
Severity is impact; priority is urgency.
Here: 22 cases, 17 passed, 5 failed, 5 defects.

Questions you must be able to answer

1. What is the difference between a test scenario and a test case? A scenario says what is to be tested, in a line. A test case is one concrete check of it, with exact data, steps and an expected result.

Practical 3: Manual Test Case Design and Execution

2. Why test a 41-character name when the limit is 40? Boundary value analysis: defects collect at the edges of a range, so the values just inside and just outside it are the ones most likely to expose a wrong comparison. Here, 41 characters exposed DEF-04.

3. Why does every test case change only one field? So that a failure has exactly one possible cause.

4. TC19's message is misspelled. Why is TC19 marked Pass? It tests only that a password with no digit is rejected, and it was. The misspelling is TC17's finding and is reported once, as DEF-05.

5. What is the difference between severity and priority? Severity is how much damage the defect does; priority is how soon it should be fixed.

6. What must a defect report contain for a developer to act on it? An identifier and title, the steps and data to reproduce it, the expected and actual results, the environment, severity, priority, status, and the test case that found it.

7. What is the pass rate here? Seventeen of twenty-two, about 77.3 per cent.

8. What is confirmation testing? Running a failed test case again after the defect it found has been fixed, to confirm the fix.

Contents This chapter on its own page

Chapter Six

Selenium WebDriver from Zero: Java, Maven, the Drivers, Locators and the First Script

Syllabus topic Selenium WebDriver, the tool of Practicals 4 to 20, taught before the first of them. MU's Course Outcome 1: "Design and execute automated test scripts using Selenium IDE and WebDriver for functional validation."

Aim

To set up Java, Eclipse and a Maven project for Selenium WebDriver, to understand how a WebDriver script controls a browser, to write and run a first script, to use every locator strategy, to recognise the exceptions a beginner meets first, and to compile and run the script Selenium IDE exported in Practical 1.

What WebDriver is

Selenium's own documentation says WebDriver "drives a browser natively, as a user would", and that Selenium WebDriver "is a W3C Recommendation": a standard published by the World Wide Web Consortium, the body that also publishes HTML's standards. Because it is a standard, every major browser maker ships a program that obeys it.

Three pieces are involved, and knowing which is which answers most "why is it not working" questions:

Piece	What it is	For Chrome	For Firefox
Your script	Java code using Selenium's library	the same	the same
The driver	a small program that controls one browser	chromedriver	geckodriver
The browser	the browser itself	Chrome	Firefox

In Selenium's own words, "WebDriver talks to a browser through a driver", and the communication "is two-way". Your script asks the driver to click; the driver makes the browser click; the answer comes back the same way. Selenium's documentation also notes that most drivers "are created by the browser vendors themselves".

A script, a driver and a browser, with commands going one way and answers coming back

Figure 6.1 One WebDriver command, from a line of Java to the browser and back.

You do not download the driver. Since Selenium 4.6 every release of Selenium ships a part called Selenium Manager, which, in its documentation's words, "automatically discovers, downloads, and caches the drivers required by Selenium when these drivers are unavailable". The first time a script starts Chrome, Selenium Manager fetches the right chromedriver for the Chrome you have. This is why old tutorials that begin with "download chromedriver.exe and set its path" can be ignored.

What to install

A Java Development Kit (JDK), version 17, 21 or 25. Eclipse Temurin, from adoptium.net, is a free build. Every program in this book was run on all three versions.
Eclipse IDE for Java Developers, from eclipse.org. Its Maven support builds the project inside Eclipse.
Apache Maven, the command-line tool, from maven.apache.org, for the mvn commands this book types in a terminal from Step 6 on, and for Jenkins in Practical 17. Maven's own instructions are to extract the archive and add its bin folder to the PATH.
Chrome and Firefox, already installed in chapter two.
An internet connection the first time the project is built, because Maven downloads the libraries then.

Check Maven in a new terminal:

$ mvn -v
Apache Maven 3.9.9
Maven home: /usr/share/maven
Java version: 21.0.12.1, vendor: Debian, runtime: /usr/lib/jvm/java-21-openjdk-arm64
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "6.8.0-117-generic", arch: "aarch64", family: "unix"

Selenium WebDriver from Zero: Java, Maven, the Drivers, Locators and the First Script

That is the lab's Maven. Yours will show a newer version number, your own folders and your operating system; what matters is that the command is found and names a Java version.

Maven is the tool that downloads libraries and builds Java projects. You list what the project needs in a file called pom.xml (project object model), and Maven fetches each library and everything it depends on. Without it you would download dozens of jar files by hand and add each to Eclipse.

Step 1: the project

In Eclipse: File, New, Maven Project. On the first page tick Create a simple project (skip archetype selection) and click Next. Fill in:

Group Id: stqa
Artifact Id: stqa-practicals

and click Finish. Eclipse creates the project with the standard Maven folders: src/main/java for programs and src/test/java for test classes.

Open pom.xml and replace its contents with this, the one file every Java practical in this book uses:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>stqa</groupId>
  <artifactId>stqa-practicals</artifactId>
  <version>1.0</version>

  <properties>
    <maven.compiler.release>17</maven.compiler.release>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>4.49.0</version>
    </dependency>
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>7.12.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.poi</groupId>
      <artifactId>poi-ooxml</artifactId>
      <version>5.5.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-api</artifactId>
      <version>2.26.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-core</artifactId>
      <version>2.26.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-slf4j2-impl</artifactId>
      <version>2.26.1</version>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.13.2</version>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.16.0</version>
        <configuration>
          <proc>none</proc>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.5.6</version>
        <dependencies>
          <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit4</artifactId>
            <version>3.5.6</version>
          </dependency>
          <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-testng</artifactId>
            <version>3.5.6</version>
          </dependency>
        </dependencies>
      </plugin>
    </plugins>
  </build>
</project>

Save it. Eclipse's Maven support downloads the libraries, which takes a minute or two the first time. What each part is for:

Part	Why it is here
Java release 17	the `release` setting: compiles for Java 17, so the same project runs on 17, 21 and 25
selenium-java	Selenium WebDriver itself, with the Chrome, Firefox and Edge drivers' Java side and Selenium Manager
testng	TestNG, the test framework of Practicals 15 to 17
poi-ooxml	Apache POI, which reads and writes Excel files, for Practical 7
log4j-api	the half of Log4j a program calls. Its version must match log4j-core's: Apache POI brings an older log4j-api, and without this line Maven picks POI's. Practical 14 shows what the mismatch did
log4j-core	Log4j, for logging in Practical 14
log4j-slf4j2-impl	sends TestNG's internal logging to Log4j, so TestNG does not warn that it has nowhere to log
junit	JUnit 4, which the Selenium IDE export from Practical 1 is written in
`proc` none	log4j-core carries an annotation processor that this project never uses; left on, Java 21's compiler prints a four-line note about it on every build
surefire 3.5.6, with two providers	the part of Maven that runs tests, told to run both JUnit and TestNG tests

The last line needs a sentence more. Surefire, the part of Maven that runs tests, looks at the libraries a project has, and when it sees TestNG it runs every test with TestNG's runner. That runner does not see JUnit tests at all: run in the lab without the two providers, Surefire reported "Tests run: 0" for the four-test JUnit class later in this chapter, and "BUILD SUCCESS". A green build that tested nothing is the most dangerous result in testing, and naming both providers is the cure. Surefire 3.6.0, released in September 2026, runs tests differently and has deprecated the setting that Practical 16's testng.xml needs, which is why the version is fixed at 3.5.6.

Selenium WebDriver from Zero: Java, Maven, the Drivers, Locators and the First Script

Step 2: the first script

Right-click src/main/java, choose New, Package, and call it practicals. Every Module 1 program goes in this package. Then New, Class, name it FirstScript, and type:

package practicals;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

public class FirstScript {
    public static void main(String[] args) {
        WebDriver driver = new ChromeDriver();               // start Chrome through chromedriver
        driver.get("http://localhost:8080/about.php");       // open the page and wait for it to load
        System.out.println("Title: " + driver.getTitle());

        String heading = driver.findElement(By.tagName("h1")).getText();
        System.out.println("Heading: " + heading);

        driver.quit();                                       // close the browser, end the session
    }
}

Run it: right-click in the editor, Run As, Java Application. Chrome opens by itself, shows the About page for a moment, and closes. The console shows:

Title: About - Practice Portal
Heading: About the Practice Portal

Sep 30, 2026 5:04:07 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154

The second block is printed in red in Eclipse's console, and it frightens students more than any real error. It is a warning, not an error: the program ran and printed the right answers above it. Selenium carries a definition of Chrome's developer-tools protocol (CDP) for each recent Chrome version, and when your Chrome is newer than the newest one your Selenium knows, it says so and uses the closest. Nothing in this book uses that protocol. The cure is the one the warning names, a newer Selenium version in pom.xml when one is published; the warning disappears when your Selenium is as new as your Chrome, and every other program in this book was run in that state.

The five lines, in order:

new ChromeDriver() starts chromedriver, which starts a fresh Chrome with an empty profile: no history, no saved passwords, no extensions.
driver.get(url) opens the page and returns only when it has loaded.
driver.getTitle() returns the text in the page's <title>.
driver.findElement, given By.tagName("h1"), finds the first h1 element, and .getText() returns its visible text.
driver.quit() closes the browser and ends the driver. Always end with quit(); a program that stops without it leaves a browser and a driver running.

Selenium WebDriver from Zero: Java, Maven, the Drivers, Locators and the First Script

Step 3: locators, all eight

A locator tells WebDriver how to find an element, and By has one method for each strategy. Selenium's documentation lists eight, with these meanings:

Strategy	Java	Finds elements whose...
id	`By.id`	ID attribute matches the search value
name	`By.name`	NAME attribute matches the search value
class name	`By.className`	class name contains the search value (compound class names are not permitted)
tag name	`By.tagName`	tag name matches the search value
link text	`By.linkText`	visible text matches, for links
partial link text	`By.partialLinkText`	visible text contains the search value, for links
css selector	`By.cssSelector`	CSS selector matches
xpath	`By.xpath`	XPath expression matches

This program uses every one of them on the enquiry form from Practical 1:

package practicals;

import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;

public class Locators {
    public static void main(String[] args) {
        WebDriver driver = new ChromeDriver();
        driver.get("http://localhost:8080/enquiry.php");

        WebElement email = driver.findElement(By.id("email"));
        System.out.println("id=email               -> <" + email.getTagName() + "> type=" + email.getDomAttribute("type"));

        List<WebElement> rooms = driver.findElements(By.name("room"));
        System.out.println("name=room              -> " + rooms.size() + " elements");

        List<WebElement> hidden = driver.findElements(By.className("hidden"));
        System.out.println("className=hidden       -> " + hidden.size() + " elements");

        List<WebElement> selects = driver.findElements(By.tagName("select"));
        System.out.println("tagName=select         -> " + selects.size() + " elements");

        WebElement home = driver.findElement(By.linkText("Home"));
        System.out.println("linkText=Home          -> href=" + home.getDomAttribute("href"));

        WebElement partial = driver.findElement(By.partialLinkText("Hom"));
        System.out.println("partialLinkText=Hom    -> " + partial.getText());

        List<WebElement> radios = driver.findElements(By.cssSelector("input[type=radio]"));
        System.out.println("cssSelector=input[type=radio] -> " + radios.size() + " elements");

        List<WebElement> options = driver.findElements(By.xpath("//select[@id='course']/option"));
        System.out.print("xpath=//select[@id='course']/option -> " + options.size() + " options:");
        for (WebElement o : options) {
            System.out.print(" [" + o.getText() + "]");
        }
        System.out.println();

        driver.quit();
    }
}

id=email               -> <input> type=text
name=room              -> 2 elements
className=hidden       -> 2 elements
tagName=select         -> 2 elements
linkText=Home          -> href=index.php
partialLinkText=Hom    -> Home
cssSelector=input[type=radio] -> 2 elements
xpath=//select[@id='course']/option -> 4 options: [-- Select --] [B.Sc. CS] [B.Sc. IT] [B.Com.]

Read the counts against the page's source in Practical 1: two radio buttons share the name room; two elements carry the class hidden (the Year label and the room choices); there are two dropdowns; and the course dropdown has four options, the first of them the "-- Select --" placeholder.

getDomAttribute("href") returns the attribute exactly as written in the HTML, index.php. Which locator to choose, in order of preference: an id when the element has one; then name for form fields; then a short CSS selector; XPath when you must go by position or by text. Link text is for links and nothing else.

Step 4: findElement against findElements

The two methods differ in exactly the way that matters when something is missing:

Selenium WebDriver from Zero: Java, Maven, the Drivers, Locators and the First Script

package practicals;

import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.NoSuchElementException;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;

public class FindOneOrMany {
    public static void main(String[] args) {
        WebDriver driver = new ChromeDriver();
        driver.get("http://localhost:8080/about.php");

        List<WebElement> none = driver.findElements(By.id("missing"));
        System.out.println("findElements found " + none.size() + " element(s), and no exception");

        try {
            driver.findElement(By.id("missing"));
        } catch (NoSuchElementException e) {
            System.out.println("findElement threw " + e.getClass().getSimpleName());
            System.out.println("  " + e.getMessage().lines().findFirst().orElse(""));
        }
        driver.quit();
    }
}

findElements found 0 element(s), and no exception
findElement threw NoSuchElementException
  no such element: Unable to locate element: {"method":"css selector","selector":"#missing"}

So: findElement returns one element or throws NoSuchElementException; findElements returns a list, empty when nothing matches, and never throws for a missing element. Use findElements when "not there" is an allowed answer, for example when counting.

The message's first line is chromedriver's own, and it names the locator it tried, which is the first thing to check: #missing is what By.id("missing") becomes inside the driver.

Step 5: the exceptions a beginner meets first

Each of these was produced on purpose on the Practice Portal. Learn to recognise them; in the examination hall the exception's name is half the diagnosis.

package practicals;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;

public class BeginnerExceptions {
    static void report(String what, WebDriverException e) {
        System.out.println(what + ": " + e.getClass().getSimpleName());
        System.out.println("  " + e.getMessage().lines().findFirst().orElse(""));
    }

    public static void main(String[] args) {
        WebDriver driver = new ChromeDriver();
        driver.get("http://localhost:8080/enquiry.php");

        try {                                   // the room choices are hidden until the box is ticked
            driver.findElement(By.id("shared")).click();
        } catch (WebDriverException e) {
            report("Clicking a hidden radio button", e);
        }

        try {                                   // an XPath with its closing bracket missing
            driver.findElement(By.xpath("//select[@id='course'"));
        } catch (WebDriverException e) {
            report("A broken XPath", e);
        }

        WebElement name = driver.findElement(By.id("name"));
        driver.navigate().refresh();            // the page is rebuilt: the old element is gone
        try {
            name.sendKeys("Priya");
        } catch (WebDriverException e) {
            report("Using an element after a reload", e);
        }
        driver.quit();
    }
}

Clicking a hidden radio button: ElementNotInteractableException
  element not interactable
A broken XPath: InvalidSelectorException
  invalid selector: Unable to locate an element with the xpath expression //select[@id='course' because of the following error:
Using an element after a reload: StaleElementReferenceException
  stale element reference: stale element not found

Exception	What it means	What to do
NoSuchElementException	nothing matches the locator, yet	check the locator against the page source; if the element appears later, wait for it (Practical 6)
ElementNotInteractableException	the element exists but cannot be used, usually because it is hidden	do what makes it visible first, or wait until it is
InvalidSelectorException	the locator itself is broken	fix the CSS or XPath; this one is always a typing mistake
StaleElementReferenceException	the element was found, and then the page changed or reloaded	find it again after the change

Selenium WebDriver from Zero: Java, Maven, the Drivers, Locators and the First Script

Every one of them extends WebDriverException, which is why a single catch (WebDriverException e) catches all three in the program above.

Step 6: running the script Selenium IDE exported

Practical 1 ended with a Java file exported from Selenium IDE, PageformatsTest.java. Put it in src/test/java (it is a test class, so it goes with the tests, in no package) and build:

$ mvn -q test-compile
[ERROR] COMPILATION ERROR :
[ERROR] /home/student/stqa-practicals/src/test/java/PageformatsTest.java:[80,54] incompatible types: int cannot be converted to java.time.Duration
[ERROR] /home/student/stqa-practicals/src/test/java/PageformatsTest.java:[90,54] incompatible types: int cannot be converted to java.time.Duration
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.16.0:testCompile (default-testCompile) on project stqa-practicals: Compilation failure: Compilation failure:
[ERROR] /home/student/stqa-practicals/src/test/java/PageformatsTest.java:[80,54] incompatible types: int cannot be converted to java.time.Duration
[ERROR] /home/student/stqa-practicals/src/test/java/PageformatsTest.java:[90,54] incompatible types: int cannot be converted to java.time.Duration
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

On the student's own machine the paths name their own folder. Two errors, both on the lines where the export makes a wait:

WebDriverWait wait = new WebDriverWait(driver, 5);

Selenium IDE 3 writes Selenium 3's form, a number of seconds. Selenium 4 takes a Duration, and the number form no longer exists. The fix is the same on both lines, and one import:

import java.time.Duration;
...
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(5));

This is the whole file with both lines fixed and the import added; nothing else is changed from what the IDE wrote:

// Generated by Selenium IDE
import org.junit.Test;
import org.junit.Before;
import org.junit.After;
import static org.junit.Assert.*;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.core.IsNot.not;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.Dimension;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.interactions.Actions;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.Alert;
import org.openqa.selenium.Keys;
import java.util.*;
import java.time.Duration;
import java.net.MalformedURLException;
import java.net.URL;
public class PageformatsTest {
  private WebDriver driver;
  private Map<String, Object> vars;
  JavascriptExecutor js;
  @Before
  public void setUp() {
    driver = new FirefoxDriver();
    js = (JavascriptExecutor) driver;
    vars = new HashMap<String, Object>();
  }
  @After
  public void tearDown() {
    driver.quit();
  }
  @Test
  public void hTMLpage() {
    driver.get("http://localhost:8080/about.php");
    assertThat(driver.getTitle(), is("About - Practice Portal"));
    assertThat(driver.findElement(By.cssSelector("h1")).getText(), is("About the Practice Portal"));
    {
      List<WebElement> elements = driver.findElements(By.id("departments"));
      assert(elements.size() > 0);
    }
    assertThat(driver.findElement(By.xpath("//table[@id=\'hours\']//tr[2]/td[2]")).getText(), is("10:00 to 17:00"));
  }
  @Test
  public void xMLresponse() {
    driver.get("http://localhost:8080/index.php");
    vars.put("type", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/notices.php\').then(r => r.headers.get(\'Content-Type\')).then(callback).catch(callback);"));
    assertEquals(vars.get("type").toString(), "application/xml; charset=utf-8");
    vars.put("noticeCount", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/notices.php\').then(r => r.text()).then(t => new DOMParser().parseFromString(t, \'application/xml\').getElementsByTagName(\'notice\').length).then(callback).catch(callback);"));
    assertEquals(vars.get("noticeCount").toString(), "3");
    vars.put("secondTitle", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/notices.php\').then(r => r.text()).then(t => new DOMParser().parseFromString(t, \'application/xml\').querySelector(\'notice[id=\"2\"] title\').textContent).then(callback).catch(callback);"));
    assertEquals(vars.get("secondTitle").toString(), "Journal submission for Practical 1");
  }
  @Test
  public void jSONAPI() {
    driver.get("http://localhost:8080/index.php");
    vars.put("type", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/students.php\').then(r => r.headers.get(\'Content-Type\')).then(callback).catch(callback);"));
    assertEquals(vars.get("type").toString(), "application/json; charset=utf-8");
    vars.put("count", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/students.php\').then(r => r.json()).then(d => d.count).then(callback).catch(callback);"));
    assertEquals(vars.get("count").toString(), "10");
    vars.put("name", js.executeAsyncScript("var callback = arguments[arguments.length - 1];return fetch(\'/api/students.php?roll=103\').then(r => r.json()).then(s => s.name).then(callback).catch(callback);"));
    assertEquals(vars.get("name").toString(), "Kabir Khan");
  }
  @Test
  public void dynamicform() {
    driver.get("http://localhost:8080/enquiry.php");
    driver.findElement(By.id("name")).sendKeys("Priya Sharma");
    driver.findElement(By.id("email")).sendKeys("priya@example.com");
    {
      WebElement dropdown = driver.findElement(By.id("course"));
      dropdown.findElement(By.xpath("//option[. = 'B.Sc. CS']")).click();
    }
    {
      WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(5));
      wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("year")));
    }
    assertThat(driver.findElement(By.id("fee")).getText(), is("Annual fee: Rs. 38,000"));
    {
      WebElement dropdown = driver.findElement(By.id("year"));
      dropdown.findElement(By.xpath("//option[. = 'TY']")).click();
    }
    driver.findElement(By.id("hostel")).click();
    {
      WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(5));
      wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("shared")));
    }
    driver.findElement(By.id("shared")).click();
    driver.findElement(By.id("submit")).click();
    assertThat(driver.findElement(By.id("confirmation")).getText(), is("Thank you, Priya Sharma. Your enquiry for B.Sc. CS (TY) has been recorded. Hostel: Shared room."));
  }
}

Selenium WebDriver from Zero: Java, Maven, the Drivers, Locators and the First Script

Now it compiles. Run its four tests:

$ mvn test -Dtest=PageformatsTest
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------< stqa:stqa-practicals >------------------------
[INFO] Building stqa-practicals 1.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/main/resources
[INFO]
[INFO] --- compiler:3.16.0:compile (default-compile) @ stqa-practicals ---
[INFO] Nothing to compile - all classes are up to date.
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/test/resources
[INFO]
[INFO] --- compiler:3.16.0:testCompile (default-testCompile) @ stqa-practicals ---
[INFO] Recompiling the module because of changed dependency.
[INFO] Compiling 1 source file with javac [debug release 17] to target/test-classes
[INFO] /home/student/stqa-practicals/src/test/java/PageformatsTest.java: /home/student/stqa-practicals/src/test/java/PageformatsTest.java uses or overrides a deprecated API.
[INFO] /home/student/stqa-practicals/src/test/java/PageformatsTest.java: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- surefire:3.5.6:test (default-test) @ stqa-practicals ---
[INFO] Using configured provider org.apache.maven.surefire.junit4.JUnit4Provider
[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running PageformatsTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.19 s -- in PageformatsTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running PageformatsTest
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.734 s -- in PageformatsTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  21.072 s
[INFO] Finished at: 2026-09-30T17:00:01+05:30
[INFO] ------------------------------------------------------------------------

Read the test section twice, because Surefire ran the class twice. The two "Using configured provider" lines are the two runners the pom.xml names. The JUnit runner ran first and reported "Tests run: 4, Failures: 0, Errors: 0, Skipped: 0": the suite built in Selenium IDE in Practical 1, now running as Java through WebDriver. The TestNG runner then looked at the same class, found no TestNG tests in it, and reported "Tests run: 0". That second report is exactly what the whole build would have said with TestNG's runner alone, and it still ends in BUILD SUCCESS, which is why the pom names both.

Selenium WebDriver from Zero: Java, Maven, the Drivers, Locators and the First Script

In Eclipse the same four tests run with a right-click on the class, Run As, JUnit Test, and the JUnit tab shows all four passing.

Two warnings from the build are worth reading rather than ignoring. The compiler notes that the file "uses or overrides a deprecated API": JUnit 4 marks assertThat as deprecated, and the export uses it for every text check. And the export's assert(elements.size() > 0) is Java's own assert statement, which Java skips unless assertions are switched on; Maven's Surefire switches them on for tests by default, which is why it would have caught a missing departments list here.

Observations

Program	Browser	What it showed
FirstScript	Chrome	title and heading read from a real page
Locators	Chrome	all eight strategies, with the counts the page's source predicts
FindOneOrMany	Chrome	findElements returns an empty list; findElement throws
BeginnerExceptions	Chrome	three different exceptions, each from a different mistake
PageformatsTest	Firefox	the Selenium IDE export needed two lines changed, then passed 4 of 4

All five were run on Java 17, 21 and 25 with identical output.

Result

A Maven project with Selenium WebDriver 4.49 was set up, a first script opened a page in Chrome and read its title and heading, every locator strategy was used, the difference between findElement and findElements was shown, three beginner exceptions were produced and explained, and the script exported from Selenium IDE in Practical 1 was corrected for Selenium 4 and passed all four of its tests.

Where marks are lost

Downloading chromedriver by hand and pointing the script at an old one. Let Selenium Manager do it.

No driver.quit(). Browsers and drivers pile up in the background until the machine slows down.

Treating the CDP warning as an error. Read whether the program printed its results.

An absolute XPath, one that spells out every step from the top of the page: /html, then /body, /main, /form, /label[3] and /select. It breaks the moment the page gains one element; use an id or a short relative path.

Using findElement to test that something is absent. It throws. Use findElements and check the size.

Running the IDE export unchanged and concluding Selenium is broken. It was written for Selenium 3.

For the journal

Aim; the software installed and its versions (JDK, Eclipse, Selenium 4.49, Chrome); the pom.xml; the first script and its output; the locator program, its output and the locator table; the exceptions table; the corrected export and its test result.

Selenium WebDriver from Zero: Java, Maven, the Drivers, Locators and the First Script

Quick revision

WebDriver is a W3C standard. Script, driver, browser: chromedriver for Chrome, geckodriver for Firefox.
Selenium Manager, shipped since Selenium 4.6, finds and downloads drivers by itself.
Maven reads pom.xml and downloads every library the project needs.
get, getTitle, findElement, getText, quit.
Eight locators: id, name, className, tagName, linkText, partialLinkText, cssSelector, xpath.
findElement throws NoSuchElementException; findElements returns an empty list.
ElementNotInteractable: hidden. InvalidSelector: broken locator. StaleElementReference: page changed.
Selenium IDE's Java export uses Selenium 3's WebDriverWait(driver, 5); Selenium 4 needs Duration.ofSeconds(5).
With TestNG present, Surefire needs both providers named or it runs zero JUnit tests.

Questions you must be able to answer

1. What are the three pieces involved when a WebDriver script clicks a button? The script, which calls Selenium's library; the driver, such as chromedriver, which controls the browser; and the browser itself.

2. Why do students no longer download chromedriver by hand? Since Selenium 4.6, Selenium Manager discovers, downloads and caches the right driver automatically.

3. What does driver.quit() do, and why must it be called? It closes the browser and ends the driver. Without it both keep running after the program ends.

4. Name the eight locator strategies. id, name, class name, tag name, link text, partial link text, CSS selector and XPath.

5. What is the difference between findElement and findElements when nothing matches? findElement throws NoSuchElementException; findElements returns an empty list.

6. What does StaleElementReferenceException mean? The element was found, and then the page changed or reloaded, so the reference points at something that no longer exists. Find the element again.

7. Why does the Selenium IDE export not compile with Selenium 4? It creates WebDriverWait with a number of seconds, the Selenium 3 form; Selenium 4 requires a Duration.

8. What is the CDP warning, and is it an error? It says the installed Chrome is newer than the developer-tools definitions this Selenium carries. It is a warning; the program still runs.

Contents This chapter on its own page

Chapter Seven

Practical 4: Selenium Grid, Hub and Nodes, and Remote Execution

Syllabus topic Module 1, "Selenium Server (Grid) Configuration and Remote Execution", "Tool: Selenium Server (Grid)", "Install and configure Selenium Server in Hub-Node architecture. Execute a test script on multiple browsers using Remote WebDriver and compare cross-browser results."

Aim

To install and configure Selenium Server in the hub-node architecture, to execute one test script on more than one browser through RemoteWebDriver, and to compare the results the browsers give.

What you need to know before you start

Everything in chapter six ran the browser on the same machine as the script: new ChromeDriver() starts Chrome right there. Selenium Grid separates the two. The script sends its commands to a server, and the server passes them to a browser on whichever machine has the browser asked for.

Selenium's documentation lists what the Hub-and-Node arrangement is for: combining "different machines in a single Grid", machines "with different operating systems and/or browser versions", and "a single entry point" to run tests in different environments. A college laboratory is exactly that: one machine to run the tests from, and several with different browsers.

The two roles:

Hub. The single entry point. Scripts connect to it, at port 4444 by default. It keeps a list of the nodes and what each offers, and sends each new session to a node that has the browser asked for.
Node. A machine that offers browsers. When it starts it finds the browser drivers on its machine and registers with the hub. It runs the browsers the hub sends it.

A script sending commands to a hub, which passes them to a Chrome node and a Firefox node

Figure 7.1 Hub and nodes: the script only ever talks to the hub.

Both roles are the same file, Selenium Server, a single .jar, started with a different word: hub or node. There is also standalone, which is a hub and a node in one process on one machine.

RemoteWebDriver is the Java class that talks to a Grid instead of to a local browser. Where a local script says new ChromeDriver(), a Grid script says new RemoteWebDriver(hubAddress, new ChromeOptions()): the options object says which browser is wanted, and the hub finds it. Everything after that line, get, findElement, click, is the same code as before.

Step 1: install Selenium Server

Java must be installed (chapter six). The Grid documentation asks for Java 11 or higher; every run here used Java 21.
Download selenium-server-4.49.0.jar from the Selenium downloads page, selenium.dev/downloads, or from the Selenium project's GitHub releases.
Put it in a folder of its own, called grid.

There is nothing to install beyond that: the jar is run with java -jar.

Step 2: start the hub

Open a Command Prompt, go into grid, and start the hub. It keeps the window, so this window now belongs to the hub:

$ java -jar selenium-server-4.49.0.jar hub
17:13:20.534 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
17:13:20.539 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
17:13:20.600 INFO [BoundZmqEventBus.<init>] - XPUB binding to [binding to tcp://*:4442, advertising as tcp://172.18.0.2:4442], XSUB binding to [binding to tcp://*:4443, advertising as tcp://172.18.0.2:4443]
17:13:20.611 INFO [ZmqUtils.configureHeartbeat] - ZMQ XPUB socket heartbeat configured: interval=60s, timeout=180s, ttl=360s
17:13:20.620 INFO [ZmqUtils.configureHeartbeat] - ZMQ XSUB socket heartbeat configured: interval=60s, timeout=180s, ttl=360s
17:13:20.645 INFO [UnboundZmqEventBus.<init>] - Connecting to tcp://172.18.0.2:4442 and tcp://172.18.0.2:4443
17:13:20.651 INFO [ZmqUtils.configureHeartbeat] - ZMQ SUB socket heartbeat configured: interval=60s, timeout=180s, ttl=360s
17:13:20.653 INFO [ZmqUtils.configureHeartbeat] - ZMQ PUB socket heartbeat configured: interval=60s, timeout=180s, ttl=360s
17:13:20.653 INFO [UnboundZmqEventBus.<init>] - Sockets created
17:13:21.655 INFO [UnboundZmqEventBus.<init>] - Event bus ready
17:13:22.022 INFO [Hub.execute] - Started Selenium Hub 4.49.0 (revision bedb24f bedb24f6f703876e7e4c5d506f0d697eb358c720): http://172.18.0.2:4444

Practical 4: Selenium Grid, Hub and Nodes, and Remote Execution

The line to look for is the last: Started Selenium Hub, with the address scripts must use. The earlier lines show the hub opening its event bus on ports 4442 and 4443, the channel nodes use to register; the Grid documentation says those two ports must be reachable on the hub machine for nodes on other machines to join.

Step 3: start the nodes

A node needs to know where the hub is. On the same machine as the hub, node finds it by itself. On another machine, the documentation's form is node --hub http://<hub-ip>:4444, with the hub machine's address.

This lab starts two nodes on one machine, one for Chrome and one for Firefox, as two laboratory machines would be. Two nodes on one machine need two ports; the documentation's example is --port 5555 and --port 6666. --driver-implementation names the one browser each node should offer. Open a second window:

$ java -jar selenium-server-4.49.0.jar node --port 5555 --driver-implementation chrome
17:12:16.430 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
17:12:16.434 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
17:12:16.515 INFO [UnboundZmqEventBus.<init>] - Connecting to tcp://0.0.0.0:4442 and tcp://0.0.0.0:4443
17:12:16.546 INFO [ZmqUtils.configureHeartbeat] - ZMQ SUB socket heartbeat configured: interval=60s, timeout=180s, ttl=360s
17:12:16.551 INFO [ZmqUtils.configureHeartbeat] - ZMQ PUB socket heartbeat configured: interval=60s, timeout=180s, ttl=360s
17:12:16.552 INFO [UnboundZmqEventBus.<init>] - Sockets created
17:12:17.554 INFO [UnboundZmqEventBus.<init>] - Event bus ready
17:12:17.646 INFO [NodeServer.createHandlers] - Reporting self as: http://172.18.0.2:5555
17:12:17.658 INFO [NodeOptions.getSessionFactories] - Detected 2 available processors
17:12:17.658 INFO [NodeOptions.discoverDrivers] - Looking for existing drivers on the PATH.
17:12:17.659 INFO [NodeOptions.discoverDrivers] - Add '--selenium-manager true' to the startup command to setup drivers automatically.
17:12:18.119 WARN [SeleniumManager.lambda$runCommand$0] - Unable to discover proper msedgedriver version in offline mode
17:12:18.272 INFO [NodeOptions.report] - Adding Chrome for {"browserName": "chrome","platformName": "linux"} 2 times
17:12:18.308 INFO [Node.<init>] - Binding additional locator mechanisms: relative
17:12:18.504 INFO [NodeServer$2.start] - Starting registration process for Node http://172.18.0.2:5555
17:12:18.504 INFO [NodeServer.execute] - Started Selenium node 4.49.0 (revision bedb24f bedb24f6f703876e7e4c5d506f0d697eb358c720): http://172.18.0.2:5555
17:12:18.510 INFO [NodeServer$2.lambda$start$3] - Sending registration event...
17:12:19.503 INFO [NodeServer.lambda$createHandlers$1] - Node has been added

And a third:

$ java -jar selenium-server-4.49.0.jar node --port 6666 --driver-implementation firefox
17:12:21.049 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
17:12:21.052 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
17:12:21.121 INFO [UnboundZmqEventBus.<init>] - Connecting to tcp://0.0.0.0:4442 and tcp://0.0.0.0:4443
17:12:21.153 INFO [ZmqUtils.configureHeartbeat] - ZMQ SUB socket heartbeat configured: interval=60s, timeout=180s, ttl=360s
17:12:21.161 INFO [ZmqUtils.configureHeartbeat] - ZMQ PUB socket heartbeat configured: interval=60s, timeout=180s, ttl=360s
17:12:21.166 INFO [UnboundZmqEventBus.<init>] - Sockets created
17:12:22.177 INFO [UnboundZmqEventBus.<init>] - Event bus ready
17:12:22.273 INFO [NodeServer.createHandlers] - Reporting self as: http://172.18.0.2:6666
17:12:22.287 INFO [NodeOptions.getSessionFactories] - Detected 2 available processors
17:12:22.288 INFO [NodeOptions.discoverDrivers] - Looking for existing drivers on the PATH.
17:12:22.288 INFO [NodeOptions.discoverDrivers] - Add '--selenium-manager true' to the startup command to setup drivers automatically.
17:12:22.494 WARN [SeleniumManager.lambda$runCommand$0] - Unable to discover proper msedgedriver version in offline mode
17:12:22.589 INFO [NodeOptions.report] - Adding Firefox for {"browserName": "firefox","platformName": "linux"} 2 times
17:12:22.606 INFO [Node.<init>] - Binding additional locator mechanisms: relative
17:12:22.733 INFO [NodeServer$2.start] - Starting registration process for Node http://172.18.0.2:6666
17:12:22.733 INFO [NodeServer.execute] - Started Selenium node 4.49.0 (revision bedb24f bedb24f6f703876e7e4c5d506f0d697eb358c720): http://172.18.0.2:6666
17:12:22.738 INFO [NodeServer$2.lambda$start$3] - Sending registration event...
17:12:22.963 INFO [NodeServer.lambda$createHandlers$1] - Node has been added

Practical 4: Selenium Grid, Hub and Nodes, and Remote Execution

Read a node's log from the middle:

Looking for existing drivers on the PATH. The node finds chromedriver and geckodriver the way a script does.
Adding Chrome for {"browserName": "chrome", ...} 2 times. It offers two Chrome sessions at once, one for each of the lab machine's two processors, as "Detected 2 available processors" said.
Node has been added. The hub accepted it. Only now can a script use it.

The one warning, Unable to discover proper msedgedriver version in offline mode, belongs to the laboratory these notes were checked on: its machine has no internet, and Selenium Manager was told so. On a machine with the internet the line does not appear.

Open http://localhost:4444 in a browser. The Grid's own page lists each node by its address, with the browsers it offers under Stereotypes, its Max. Concurrency (how many sessions it can run at once) and the sessions running now. The left side totals the Grid: four sessions possible, none in use.

The Selenium Grid page with two nodes, one offering Chrome and one Firefox

Figure 7.2 The Grid's page after both nodes registered: Firefox on port 6666 and Chrome on port 5555, each with a concurrency of 2.

Step 4: one script, two browsers, through the hub

The script fills the enquiry form from Practical 1 and reads back what the page shows. It runs the same method twice, once asking the hub for Chrome and once for Firefox, and then compares the two sets of results. Create practicals/GridRun.java:

package practicals;

import java.net.URL;
import java.net.URI;
import java.util.LinkedHashMap;
import java.util.Map;

import org.openqa.selenium.By;
import org.openqa.selenium.Capabilities;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.Select;

public class GridRun {
    static final String HUB = "http://localhost:4444";

    // The same steps on whatever browser the options ask the hub for.
    static String runOn(Capabilities options) throws Exception {
        URL hub = URI.create(HUB).toURL();
        RemoteWebDriver driver = new RemoteWebDriver(hub, options);
        try {
            String browser = driver.getCapabilities().getBrowserName();
            driver.get("http://localhost:8080/enquiry.php");
            driver.findElement(By.id("name")).sendKeys("Priya Sharma");
            driver.findElement(By.id("email")).sendKeys("priya@example.com");
            new Select(driver.findElement(By.id("course"))).selectByVisibleText("B.Sc. IT");
            String fee = driver.findElement(By.id("fee")).getText();
            driver.findElement(By.id("submit")).click();
            String reply = driver.findElement(By.id("confirmation")).getText();

            System.out.println("Browser:      " + browser);
            System.out.println("Title:        " + driver.getTitle());
            System.out.println("Fee shown:    " + fee);
            System.out.println("Confirmation: " + reply);
            System.out.println();
            return driver.getTitle() + " | " + fee + " | " + reply;
        } finally {
            driver.quit();
        }
    }

    public static void main(String[] args) throws Exception {
        Map<String, String> results = new LinkedHashMap<>();
        results.put("chrome", runOn(new ChromeOptions()));
        results.put("firefox", runOn(new FirefoxOptions()));

        boolean same = results.get("chrome").equals(results.get("firefox"));
        System.out.println("Same results on both browsers: " + same);
    }
}

Practical 4: Selenium Grid, Hub and Nodes, and Remote Execution

With the hub and both nodes running, run it:

Browser:      chrome
Title:        Admission Enquiry - Practice Portal
Fee shown:    Annual fee: Rs. 36,000
Confirmation: Thank you, Priya Sharma. Your enquiry for B.Sc. IT (FY) has been recorded.

Browser:      firefox
Title:        Admission Enquiry - Practice Portal
Fee shown:    Annual fee: Rs. 36,000
Confirmation: Thank you, Priya Sharma. Your enquiry for B.Sc. IT (FY) has been recorded.

Same results on both browsers: true

Sep 30, 2026 5:12:26 PM org.openqa.selenium.remote.tracing.opentelemetry.OpenTelemetryTracer createTracer
INFO: Using OpenTelemetry for tracing

The red lines in Eclipse's console are an INFO message, not an error: RemoteWebDriver reports that it traces its requests with OpenTelemetry, a standard way of following one request across several servers, which is what a hub and its nodes are.

Three details in the code matter for the journal:

URI.create(HUB).toURL() makes the hub's address. The older new URL("...") still works but is deprecated from Java 20, and the lab compiles with deprecation warnings treated as errors.
new RemoteWebDriver(hub, new ChromeOptions()) is the whole difference from a local script. The options say what is wanted; the hub decides where it runs.
The page address is still http://localhost:8080. That is right here because the nodes are on the same machine as the site. With nodes on other machines, every address in the script must be one those machines can reach: the site machine's network address, not localhost.

Step 5: what the hub did

Back in the hub's window, the hub logged each session as it happened: "Session request received by the Distributor" with the browser asked for, "Added session to local Session Map" with the address of the node that took it, and, when the script called quit(), the session deleted with the reason "session closed normally (QUIT command)". The script never learned which node ran it and did not need to: it asked for a browser, not for a machine. A script that stops without quit() leaves its session holding a slot on the node until the Grid times it out.

Practical 4: Selenium Grid, Hub and Nodes, and Remote Execution

Standalone is the same thing in one command, java -jar selenium-server-4.49.0.jar standalone, which starts a hub and a node together on one machine. The Grid documentation recommends it for developing tests with RemoteWebDriver locally and for a quick Grid in a CI tool; hub and nodes are for real laboratories of several machines.

Observations

	Chrome node	Firefox node
Port	5555	6666
Sessions offered	2	2
Title	Admission Enquiry - Practice Portal	Admission Enquiry - Practice Portal
Fee shown	Annual fee: Rs. 36,000	Annual fee: Rs. 36,000
Confirmation	identical	identical

The comparison the script printed: the same results on both browsers.

Result

Selenium Server 4.49.0 was configured as a hub and two nodes, one offering Chrome and one Firefox; both nodes registered with the hub. One test script, run through RemoteWebDriver, executed on both browsers through the hub, and the results were compared and found identical.

Where marks are lost

Starting the node before the hub, or in the hub's window. Each role needs its own window, and the hub must be up first.

Pointing the script at a node. Scripts talk to the hub, port 4444, never to a node's port.

localhost in the script with nodes on other machines. The node opens the address, so it must be one the node can reach.

Blocked event bus ports. Nodes on another machine cannot register if ports 4442 and 4443 on the hub machine are closed by a firewall.

Stopping at "it opened a browser". MU asks for the results to be compared. Print them, and state whether they agree.

For the journal

Aim; the Selenium Server version and Java version; the three commands and the key line of each log (Started Selenium Hub, Adding Chrome, Node has been added); a sketch of your hub and nodes with their ports; the program; its output; the observations table; the result.

Quick revision

Grid: scripts talk to a hub; the hub sends each session to a node with the right browser.
One jar, three roles: hub, node, standalone.
Hub: port 4444 for scripts; event bus on 4442 and 4443 for nodes.
Node on another machine: node --hub http://<hub-ip>:4444. Two nodes on one machine: different --port.
A node finds drivers on the PATH; --selenium-manager true lets Selenium Manager set them up.
Script: a RemoteWebDriver made from the hub's address, http://localhost:4444, and the browser's options, such as ChromeOptions.
The node opens the page, so addresses must be reachable from the node.

Questions you must be able to answer

1. What is the difference between the hub and a node? The hub is the single entry point that scripts connect to and that keeps the list of nodes. A node is a machine offering browsers, which registers with the hub and runs the sessions it is sent.

Practical 4: Selenium Grid, Hub and Nodes, and Remote Execution

2. How does a script choose the browser on a Grid? By the options object passed to RemoteWebDriver: ChromeOptions asks for Chrome, FirefoxOptions for Firefox. The hub picks a node that has it.

3. Which port does the script connect to? The hub's, 4444 by default.

4. What does "Adding Chrome ... 2 times" in a node's log mean? The node offers two Chrome sessions at the same time, here one for each processor it detected.

5. What is standalone mode? A hub and a node in one process on one machine, started with standalone.

6. The nodes are on other machines and the test fails at the first page. What is the likely cause? The script uses localhost for the site, which on the node machine means the node itself.

7. Why use URI.create(...).toURL() instead of new URL(...)? The URL constructor is deprecated from Java 20; the URI route builds the same URL without the warning.

Contents This chapter on its own page

Chapter Eight

Practical 5: Login Automation and Validation

Syllabus topic Module 1, "Login Automation and Validation", "Tool: Selenium WebDriver (Java/PHP)", "Write an automation script to perform login on a specified web page. Verify successful login using assertions such as URL validation, welcome message validation, and logout visibility check."

Aim

To write an automation script that logs in to a specified web page, and to verify the login with three assertions: the URL, the welcome message and the visibility of the logout link; and to check the failed logins a login test must also try.

What you need to know before you start

A login is the most tested page on the web, and a good login test checks more than "the page changed". MU names three checks, and each catches a different failure:

URL validation. After a successful login the browser should be at the page that follows a login. A login that "succeeds" but leaves you on the login page has failed.
Welcome message validation. The page should greet the right person. A site that logs everybody in as the first user in its database passes the URL check and fails this one.
Logout visibility. A logged-in user must be able to log out. If the link is missing or hidden, the user is trapped in the session.

A login test is not finished with the one successful case. It must also show that the wrong logins fail: a wrong password, an empty form. Testers call the successful path the positive test and the failures the negative tests, and a login page that lets a wrong password in passes every positive test ever written.

An assertion is a check that stops or fails the test when it is false. This chapter's program uses a small helper of its own that prints PASS or FAIL for each check with the expected and actual values, so that the output is the evidence. Practicals 15 and 16 move the same checks into TestNG, which is where assertions belong in a real framework.

Waiting for the next page. Clicking the Login button sends the form, and the browser loads the next page. The script must not read the URL in the moment before that page has arrived, so it waits, for at most ten seconds, until the URL contains dashboard.php. Practical 6 teaches waits properly; here one line is enough.

The pages under test

The accounts the site knows are in practice/inc/users.php. The passwords are stored as hashes, as on any real site; you log in with Asha@2026 or Ravi@2026, never with the hash:

<?php
// inc/users.php: the two accounts the practice site knows.
// Passwords are stored as hashes, as on any real site. You log in with the
// password (Asha@2026 and Ravi@2026), never with the hash.
return [
    'asha' => ['name' => 'Asha Patil', 'hash' => '$2y$12$Nvtfc//zLdTpJNyGgKjv9etOreEQlZncS1Mxcq1bUnaSP1HoXmENa'],
    'ravi' => ['name' => 'Ravi Kumar', 'hash' => '$2y$12$MlF30iBSW50GFh1fMKFeVe2BqdDuG/11P5xZaetIWS2tECdSp7i82'],
];

The login page, practice/login.php. It checks for an empty form first, then the password, and on success stores the user's name in the session and sends the browser to the dashboard:

Practical 5: Login Automation and Validation

<?php
session_start();
$users = require 'inc/users.php';
$error = '';
if ($_SERVER['REQUEST_METHOD'] === 'POST') {
    $username = trim($_POST['username'] ?? '');
    $password = $_POST['password'] ?? '';
    if ($username === '' || $password === '') {
        $error = 'Please enter both username and password.';
    } elseif (isset($users[$username]) && password_verify($password, $users[$username]['hash'])) {
        $_SESSION['user'] = $users[$username]['name'];
        header('Location: dashboard.php');
        exit;
    } else {
        $error = 'Invalid username or password.';
    }
}
$title = 'Login';
include 'inc/top.php';
?>
<h1>Student Login</h1>
<?php if (isset($_GET['loggedout'])): ?>
  <p id="message" class="success">You have been logged out.</p>
<?php endif; ?>
<?php if ($error !== ''): ?>
  <p id="error" class="error"><?= htmlspecialchars($error) ?></p>
<?php endif; ?>
<form method="post" action="login.php">
  <label>Username <input type="text" name="username" id="username"></label>
  <label>Password <input type="password" name="password" id="password"></label>
  <p><button type="submit" id="loginBtn">Login</button></p>
</form>
<?php include 'inc/bottom.php'; ?>

The page after login, practice/dashboard.php. It refuses anybody without a session, greets the user by name, and offers the logout link:

<?php
session_start();
if (!isset($_SESSION['user'])) {
    header('Location: login.php');
    exit;
}
$title = 'Dashboard';
include 'inc/top.php';
?>
<h1 id="welcome">Welcome, <?= htmlspecialchars($_SESSION['user']) ?></h1>
<p>You are signed in to the Practice Portal.</p>
<ul>
  <li><a href="students.php">Student records</a></li>
  <li><a href="marks.php">Marks table</a></li>
</ul>
<p><a href="logout.php" id="logout">Logout</a></p>
<?php include 'inc/bottom.php'; ?>

And practice/logout.php, which ends the session and returns to the login page with a message:

<?php
session_start();
session_destroy();
header('Location: login.php?loggedout=1');
exit;

What the script needs from the HTML: the username box is id="username", the password box id="password", the button id="loginBtn"; the greeting is id="welcome", the logout link id="logout"; an error is id="error" and the logged-out note id="message".

The script in Java

Create practicals/LoginTest.java:

package practicals;

import java.time.Duration;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class LoginTest {
    static final String SITE = "http://localhost:8080/";
    static int checks = 0, failed = 0;

    // One assertion: prints the result with what was expected and what was found.
    static void check(String what, String expected, String actual) {
        checks++;
        boolean ok = expected.equals(actual);
        if (!ok) failed++;
        System.out.println("  " + (ok ? "PASS" : "FAIL") + "  " + what + ": \"" + actual + "\""
                + (ok ? "" : "  (expected \"" + expected + "\")"));
    }

    static void login(WebDriver driver, String user, String password) {
        driver.get(SITE + "login.php");
        driver.findElement(By.id("username")).sendKeys(user);
        driver.findElement(By.id("password")).sendKeys(password);
        WebElement button = driver.findElement(By.id("loginBtn"));
        button.click();
        // The answer is a new page, sometimes the login page again: wait until the old one is gone.
        new WebDriverWait(driver, Duration.ofSeconds(10))
                .ignoring(WebDriverException.class)
                .until(ExpectedConditions.stalenessOf(button));
    }

    public static void main(String[] args) {
        WebDriver driver = new ChromeDriver();
        WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10));
        try {
            System.out.println("Test 1: valid login as asha");
            login(driver, "asha", "Asha@2026");
            wait.until(ExpectedConditions.urlContains("dashboard.php"));
            check("URL", SITE + "dashboard.php", driver.getCurrentUrl());
            check("welcome message", "Welcome, Asha Patil", driver.findElement(By.id("welcome")).getText());
            check("logout link visible", "true", String.valueOf(driver.findElement(By.id("logout")).isDisplayed()));

            System.out.println("Test 2: logout");
            driver.findElement(By.id("logout")).click();
            wait.until(ExpectedConditions.urlContains("loggedout"));
            check("URL", SITE + "login.php?loggedout=1", driver.getCurrentUrl());
            check("message", "You have been logged out.", driver.findElement(By.id("message")).getText());

            System.out.println("Test 3: the dashboard after logout");
            driver.get(SITE + "dashboard.php");
            check("URL", SITE + "login.php", driver.getCurrentUrl());

            System.out.println("Test 4: wrong password");
            login(driver, "asha", "wrong-password");
            check("error", "Invalid username or password.", driver.findElement(By.id("error")).getText());
            check("URL", SITE + "login.php", driver.getCurrentUrl());

            System.out.println("Test 5: empty form");
            login(driver, "", "");
            check("error", "Please enter both username and password.", driver.findElement(By.id("error")).getText());

            System.out.println("Result: " + checks + " checks, " + (checks - failed) + " passed, " + failed + " failed");
        } finally {
            driver.quit();
        }
    }
}

Practical 5: Login Automation and Validation

Run it:

Test 1: valid login as asha
  PASS  URL: "http://localhost:8080/dashboard.php"
  PASS  welcome message: "Welcome, Asha Patil"
  PASS  logout link visible: "true"
Test 2: logout
  PASS  URL: "http://localhost:8080/login.php?loggedout=1"
  PASS  message: "You have been logged out."
Test 3: the dashboard after logout
  PASS  URL: "http://localhost:8080/login.php"
Test 4: wrong password
  PASS  error: "Invalid username or password."
  PASS  URL: "http://localhost:8080/login.php"
Test 5: empty form
  PASS  error: "Please enter both username and password."
Result: 9 checks, 9 passed, 0 failed

Why login() waits for the Login button to go stale. A wrong password or an empty form brings back the login page itself, with an error on it, and the new page looks just like the old one. An earlier version of this program read #error straight after the click; it passed when the chapter was written, and then on the book's full re-run, on Java 25, Test 5 failed with NoSuchElementException for #error: it had read the old page, which has no error on it, before the new one arrived. An element "goes stale" when its page is replaced, so once the clicked button is stale, the page being read is the new one. The wait ignores WebDriverException because Chrome, caught in the middle of replacing a page, once answered with a plain "unknown error" instead of saying the element was stale (Practical 15 met it); the wait then simply asks again. Practical 15 builds this wait into its page objects.

What each test proves:

Valid login. All three of MU's assertions: the URL is exactly the dashboard's, the greeting names Asha Patil, and the logout link is displayed. isDisplayed() returns true only for an element a user can see, not merely one that exists in the page.
Logout. Clicking it returns to the login page with the logged-out message. A logout test belongs beside a login test because MU's third check, "logout visibility", is only half a check until the link is shown to work.
The dashboard after logout. Typing the dashboard's address after logging out must not show the dashboard. The page sent the browser back to login.php, which proves the session really ended. Security testers call this checking that a protected page is protected.
Wrong password. The error appears and the browser stays on the login page.
Empty form. A different error, from the page's first check.

Practical 5: Login Automation and Validation

Why getCurrentUrl() is compared with the whole address rather than with "contains dashboard": the whole address is the stricter test. A page at dashboard.php?error=1 would pass a "contains" check.

The same script in PHP

MU allows Java or PHP for this practical. PHP's WebDriver client is php-webdriver, installed with Composer, PHP's package manager, in the folder where the test will live:

composer require php-webdriver/webdriver

Composer creates a vendor folder holding the library, and vendor/autoload.php, the one file a script includes to use it.

One difference from Java matters on day one: php-webdriver has no Selenium Manager. Its README says to install Chrome and chromedriver and to "make sure to have a compatible version of Chromedriver and Chrome". Download chromedriver for your Chrome's version from Chrome for Testing, and either put it on the Path or set the environment variable WEBDRIVER_CHROME_DRIVER to its full path, which is what php-webdriver's ChromeDriver::start() looks for. Alternatively, point the script at the Grid from Practical 4, whose nodes already manage the drivers.

Save this as login-test.php:

<?php
// login-test.php: Practical 5 in PHP, with php-webdriver.
require 'vendor/autoload.php';

use Facebook\WebDriver\Chrome\ChromeDriver;
use Facebook\WebDriver\WebDriverBy;
use Facebook\WebDriver\WebDriverExpectedCondition;

const SITE = 'http://localhost:8080/';
$checks = 0;
$failed = 0;

function check($what, $expected, $actual) {
    global $checks, $failed;
    $checks++;
    $ok = ($expected === $actual);
    if (!$ok) { $failed++; }
    echo '  ' . ($ok ? 'PASS' : 'FAIL') . "  $what: \"$actual\"" . ($ok ? '' : "  (expected \"$expected\")") . "\n";
}

$driver = ChromeDriver::start();
try {
    echo "Test 1: valid login as asha\n";
    $driver->get(SITE . 'login.php');
    $driver->findElement(WebDriverBy::id('username'))->sendKeys('asha');
    $driver->findElement(WebDriverBy::id('password'))->sendKeys('Asha@2026');
    $driver->findElement(WebDriverBy::id('loginBtn'))->click();
    $driver->wait(10)->until(WebDriverExpectedCondition::urlContains('dashboard.php'));

    check('URL', SITE . 'dashboard.php', $driver->getCurrentURL());
    check('welcome message', 'Welcome, Asha Patil', $driver->findElement(WebDriverBy::id('welcome'))->getText());
    check('logout link visible', 'true', $driver->findElement(WebDriverBy::id('logout'))->isDisplayed() ? 'true' : 'false');

    echo "Result: $checks checks, " . ($checks - $failed) . " passed, $failed failed\n";
} finally {
    $driver->quit();
}

Run it from the folder with php login-test.php:

Test 1: valid login as asha
  PASS  URL: "http://localhost:8080/dashboard.php"
  PASS  welcome message: "Welcome, Asha Patil"
  PASS  logout link visible: "true"
Result: 3 checks, 3 passed, 0 failed

The same three assertions and the same results. The shapes of the two languages line up line for line: WebDriverBy::id for By.id, -> for ., $driver->wait(10)->until(...) for a WebDriverWait, and getCurrentURL() with a capital URL where Java writes getCurrentUrl().

Observations

Test	Checks	Result
Valid login: URL, welcome message, logout visible	3	passed
Logout: URL, message	2	passed
Dashboard after logout: sent back to login	1	passed
Wrong password: error, still on login page	2	passed
Empty form: error	1	passed

Practical 5: Login Automation and Validation

Nine checks in Java, all passed; the PHP script's three login checks passed too.

Result

A WebDriver script logged in to the Practice Portal and verified the login by URL validation, welcome message validation and logout visibility; it also verified logout, the protection of the dashboard after logout, and the rejection of a wrong password and an empty form. All nine checks passed. The same login and its three assertions were automated in PHP with php-webdriver and passed.

Where marks are lost

Only the positive test. A login test that never tries a wrong password proves nothing about security.

Checking the URL too early. Without the wait, the URL can still be the login page's in the instant after the click.

Reading the answer before it arrives. When the answer is the same page again, only a wait for the old page to go stale makes sure you read the new one.

findElement(...) in place of isDisplayed() for "logout visible". Finding proves the element exists; it may still be hidden.

A "contains" check where an exact one was possible. Compare the whole URL and the whole message.

Typing the password hash. You log in with the password; the hash is what the site stores.

PHP with no chromedriver on the Path. php-webdriver does not fetch drivers. Download the one matching your Chrome, or use the Grid.

For the journal

Aim; tool (Selenium WebDriver 4.49 in Java, or php-webdriver 1.16 in PHP); the login page's locators; the test cases, positive and negative, each with expected results; the script; its output; the observations table; the result.

Quick revision

Three assertions MU names: URL, welcome message, logout visibility.
Positive test: the right login succeeds. Negative tests: wrong password and empty form fail.
Wait for the next page: urlContains("dashboard.php"), from ExpectedConditions, inside wait.until(...).
getCurrentUrl() for the address, getText() for a message, isDisplayed() for visibility.
Also test logout, and that the protected page refuses you after it.
PHP: composer require php-webdriver/webdriver; ChromeDriver::start(); no Selenium Manager, so chromedriver must be installed.

Questions you must be able to answer

1. Why check the welcome message as well as the URL? The URL shows the login led to the right page; the message shows it logged in the right person.

2. What is the difference between finding the logout link and checking that it is visible? findElement succeeds for an element that exists but is hidden; isDisplayed returns true only when a user can see it.

3. Why does the script wait after clicking Login? The next page takes a moment to load, and reading the URL in that moment would read the login page's.

Practical 5: Login Automation and Validation

4. What is a negative test? Give two for a login page. A test that checks the system correctly refuses bad input: a wrong password, and an empty form.

5. Why open the dashboard's address after logging out? To check the session really ended: a protected page must send a logged-out user back to the login page.

6. What must a PHP student install that a Java student does not? A chromedriver matching their Chrome, because php-webdriver has no Selenium Manager to fetch one.

Contents This chapter on its own page

Chapter Nine

Practical 6: Web Elements, Alerts, Frames and Waits

Syllabus topic Module 1, "Web Element Interaction and Synchronization Handling", "Tool: Selenium WebDriver (Java)", "Develop a script to automate interaction with various web elements (textbox, radio button, checkbox, dropdown, alert, and frames). Implement implicit and explicit waits for synchronization handling."

Aim

To automate interaction with a text box, radio buttons, checkboxes, a dropdown, the three kinds of JavaScript alert and a frame, and to handle synchronisation with implicit and explicit waits.

What you need to know before you start

Each kind of element has its own verbs:

Element	Do	Check
Text box	`clear()`, `sendKeys(text)`	`getDomProperty("value")`
Radio button, checkbox	`click()`	`isSelected()`
Dropdown (a `select`)	the `Select` class: `selectByVisibleText`, `selectByValue`, `selectByIndex`	`getFirstSelectedOption()`
Alert, confirm, prompt	`switchTo().alert()`, then `accept()`, `dismiss()`, `sendKeys(text)`	`getText()`
Frame	`switchTo().frame(...)`, and back with `switchTo().defaultContent()`	as on any page

Why a text box is read with getDomProperty("value") and not getText(): what you type into a box becomes its value property, not text inside the element, so getText() on a filled text box returns nothing.

Alerts are not part of the page. A JavaScript alert, confirm or prompt is drawn by the browser itself, above the page, and while it is open the page cannot be touched. WebDriver reaches it with switchTo().alert(). The three kinds differ in their buttons: an alert has only OK; a confirm has OK and Cancel; a prompt has a text box as well.

A frame is a page inside a page. An iframe holds a whole separate document, and WebDriver looks inside only one document at a time. Elements inside a frame cannot be found until the script switches into the frame, and elements outside it cannot be found until it switches back.

Synchronisation is keeping the script in step with the page. Selenium's documentation calls it "perhaps the most common challenge for browser automation": the script and the browser run separately, so sometimes the browser gets into the right state first and sometimes the script acts first, "one of the primary causes of flaky tests". A flaky test passes on one run and fails on the next with nothing changed.

The two mechanisms Selenium provides, in its documentation's words:

Implicit wait: "a global setting that applies to every element location call for the entire session". When set, findElement keeps looking for up to that long before it gives up.
Explicit wait: a loop that repeats a chosen condition, such as "this element is visible" or "this text is present", until it is true or a timeout expires. It is written where it is needed, for the one step that needs it.

And one rule, from the same page, printed as a warning: "Do not mix implicit and explicit waits. Doing so can cause unpredictable wait times." Its example: an implicit wait of 10 seconds with an explicit wait of 15 "could cause a timeout to occur after 20 seconds".

Practical 6: Web Elements, Alerts, Frames and Waits

The page under test

practice/elements.php has one of everything, and at the bottom a button that adds a new paragraph to the page three seconds after it is pressed, which is what the waits are for:

<?php $title = 'Form Elements'; include 'inc/top.php'; ?>
<h1>Form Elements</h1>
<form id="profile">
  <label>Full name <input type="text" id="fullname"></label>

  <p>Year of study</p>
  <label><input type="radio" name="year" id="fy" value="FY"> FY</label>
  <label><input type="radio" name="year" id="sy" value="SY"> SY</label>
  <label><input type="radio" name="year" id="ty" value="TY"> TY</label>

  <p>Skills</p>
  <label><input type="checkbox" name="skills" id="java" value="Java"> Java</label>
  <label><input type="checkbox" name="skills" id="python" value="Python"> Python</label>
  <label><input type="checkbox" name="skills" id="sql" value="SQL"> SQL</label>

  <label>City
    <select id="city">
      <option value="">-- Select --</option>
      <option value="MUM">Mumbai</option>
      <option value="THN">Thane</option>
      <option value="PUN">Pune</option>
    </select>
  </label>
</form>

<h2>Alerts</h2>
<button id="alertBtn">Save profile</button>
<button id="confirmBtn">Delete profile</button>
<button id="promptBtn">Enter roll number</button>
<p id="alertResult"></p>

<h2>A frame</h2>
<iframe id="noticeFrame" name="noticeFrame" src="frame-notice.php" width="420" height="150"></iframe>

<h2>Slow content</h2>
<button id="loadBtn">Load marks</button>
<div id="marksArea"></div>

<script>
document.getElementById('alertBtn').onclick = function () {
  alert('Profile saved');
  document.getElementById('alertResult').textContent = 'Saved';
};
document.getElementById('confirmBtn').onclick = function () {
  const ok = confirm('Delete this profile?');
  document.getElementById('alertResult').textContent = ok ? 'You pressed OK' : 'You pressed Cancel';
};
document.getElementById('promptBtn').onclick = function () {
  const roll = prompt('Enter your roll number');
  document.getElementById('alertResult').textContent = 'Roll number: ' + roll;
};
// Three seconds after the button is pressed, a new paragraph is added to the
// page, as data from a slow server would be. Until then it does not exist at all.
// This is what a wait is for.
document.getElementById('loadBtn').onclick = function () {
  setTimeout(function () {
    const p = document.createElement('p');
    p.id = 'marksResult';
    p.textContent = 'Marks loaded: 78 out of 100';
    document.getElementById('marksArea').appendChild(p);
  }, 3000);
};
</script>
<?php include 'inc/bottom.php'; ?>

The frame shows practice/frame-notice.php, a separate page with its own box and button:

<!DOCTYPE html>
<html lang="en">
<head><meta charset="utf-8"><title>Feedback</title></head>
<body>
<p>Tell us what you think of the portal.</p>
<input type="text" id="feedback">
<button id="send" onclick="document.getElementById('thanks').textContent = 'Feedback received';">Send</button>
<p id="thanks"></p>
</body>
</html>

Program 1: every element, every alert, the frame

Create practicals/ElementsTest.java:

package practicals;

import org.openqa.selenium.Alert;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.Select;

public class ElementsTest {
    public static void main(String[] args) {
        WebDriver driver = new ChromeDriver();
        try {
            driver.get("http://localhost:8080/elements.php");

            // Text box: clear it, type, read back what the box holds.
            WebElement name = driver.findElement(By.id("fullname"));
            name.clear();
            name.sendKeys("Priya Sharma");
            System.out.println("Text box value: " + name.getDomProperty("value"));
            System.out.println("Text box getText(): [" + name.getText() + "]");

            // Radio buttons: choosing one unchooses the others in its group.
            driver.findElement(By.id("fy")).click();
            driver.findElement(By.id("ty")).click();
            System.out.println("Radio FY selected: " + driver.findElement(By.id("fy")).isSelected());
            System.out.println("Radio TY selected: " + driver.findElement(By.id("ty")).isSelected());

            // Checkboxes: each one is independent.
            driver.findElement(By.id("java")).click();
            driver.findElement(By.id("sql")).click();
            int ticked = 0;
            for (WebElement box : driver.findElements(By.name("skills"))) {
                if (box.isSelected()) ticked++;
            }
            System.out.println("Checkboxes ticked: " + ticked + " of " + driver.findElements(By.name("skills")).size());

            // Dropdown: the Select class, by visible text, by value and by index.
            Select city = new Select(driver.findElement(By.id("city")));
            city.selectByVisibleText("Pune");
            System.out.println("Dropdown by text:  " + city.getFirstSelectedOption().getText());
            city.selectByValue("THN");
            System.out.println("Dropdown by value: " + city.getFirstSelectedOption().getText());
            city.selectByIndex(1);
            System.out.println("Dropdown by index: " + city.getFirstSelectedOption().getText());

            // Alert: only OK.
            driver.findElement(By.id("alertBtn")).click();
            Alert alert = driver.switchTo().alert();
            System.out.println("Alert says: " + alert.getText());
            alert.accept();
            System.out.println("Page then shows: " + driver.findElement(By.id("alertResult")).getText());

            // Confirm: OK or Cancel. dismiss() presses Cancel.
            driver.findElement(By.id("confirmBtn")).click();
            Alert confirm = driver.switchTo().alert();
            System.out.println("Confirm says: " + confirm.getText());
            confirm.dismiss();
            System.out.println("Page then shows: " + driver.findElement(By.id("alertResult")).getText());

            // Prompt: type into it, then OK.
            driver.findElement(By.id("promptBtn")).click();
            Alert prompt = driver.switchTo().alert();
            prompt.sendKeys("42");
            prompt.accept();
            System.out.println("Page then shows: " + driver.findElement(By.id("alertResult")).getText());

            // Frame: switch in, work, switch back.
            driver.switchTo().frame("noticeFrame");
            driver.findElement(By.id("feedback")).sendKeys("Very useful");
            driver.findElement(By.id("send")).click();
            System.out.println("Inside the frame: " + driver.findElement(By.id("thanks")).getText());
            driver.switchTo().defaultContent();
            System.out.println("Back outside, the heading is: " + driver.findElement(By.tagName("h1")).getText());
        } finally {
            driver.quit();
        }
    }
}

Practical 6: Web Elements, Alerts, Frames and Waits

Text box value: Priya Sharma
Text box getText(): []
Radio FY selected: false
Radio TY selected: true
Checkboxes ticked: 2 of 3
Dropdown by text:  Pune
Dropdown by value: Thane
Dropdown by index: Mumbai
Alert says: Profile saved
Page then shows: Saved
Confirm says: Delete this profile?
Page then shows: You pressed Cancel
Page then shows: Roll number: 42
Inside the frame: Feedback received
Back outside, the heading is: Form Elements

Line by line, the results that matter:

The text box holds "Priya Sharma" as its value, and getText() on it is empty, exactly as expected.
Clicking TY after FY left only TY selected: the two radios share name="year", which is what makes them one group.
Two of the three checkboxes are ticked; a checkbox does not untick its neighbours.
selectByIndex(1) chose Mumbai, not "-- Select --", because indexes count from 0 and the placeholder is option 0.
The alert's text was read before it was accepted. dismiss() on the confirm pressed Cancel, and the page's own code recorded that.
switchTo().frame found the frame by its name, "noticeFrame". Inside, the frame's own elements could be used; after defaultContent(), the outer page's heading could be read again.

What goes wrong if you forget to switch: looking for id=feedback without switching into the frame gives NoSuchElementException, because the element is in another document. And touching the page while an alert is open gives UnhandledAlertException; handle the alert first.

Program 2: synchronisation, measured

The Load marks button adds a paragraph id="marksResult" three seconds after it is pressed. Before that moment the paragraph does not exist at all. This program meets that delay four ways: with no wait, with an implicit wait, with an explicit wait, and with an explicit wait that is too short. Create practicals/WaitsTest.java:

package practicals;

import java.time.Duration;

import org.openqa.selenium.By;
import org.openqa.selenium.NoSuchElementException;
import org.openqa.selenium.TimeoutException;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class WaitsTest {
    static WebDriver driver;

    static void pressLoad() {
        driver.get("http://localhost:8080/elements.php");
        driver.findElement(By.id("loadBtn")).click();
    }

    static String waited(long start) {
        long ms = System.currentTimeMillis() - start;
        return (ms >= 2500 && ms < 5000) ? "between 2.5 and 5 seconds" : "outside 2.5 to 5 seconds";
    }

    public static void main(String[] args) {
        driver = new ChromeDriver();
        try {
            System.out.println("1. No wait at all");
            pressLoad();
            try {
                driver.findElement(By.id("marksResult"));
            } catch (NoSuchElementException e) {
                System.out.println("   NoSuchElementException: the paragraph did not exist yet");
            }

            System.out.println("2. Implicit wait of 5 seconds");
            driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(5));
            pressLoad();
            long start = System.currentTimeMillis();
            WebElement found = driver.findElement(By.id("marksResult"));
            System.out.println("   found: " + found.getText());
            System.out.println("   findElement waited " + waited(start));
            driver.manage().timeouts().implicitlyWait(Duration.ZERO);   // never mix the two kinds

            System.out.println("3. Explicit wait of 5 seconds for visibility");
            pressLoad();
            start = System.currentTimeMillis();
            WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(5));
            WebElement shown = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("marksResult")));
            System.out.println("   found: " + shown.getText());
            System.out.println("   the wait took " + waited(start));

            System.out.println("4. Explicit wait of only 2 seconds");
            pressLoad();
            try {
                new WebDriverWait(driver, Duration.ofSeconds(2))
                    .until(ExpectedConditions.visibilityOfElementLocated(By.id("marksResult")));
            } catch (TimeoutException e) {
                System.out.println("   TimeoutException, and the first two lines of its message:");
                e.getMessage().lines().limit(2).forEach(line -> System.out.println("   " + line));
            }
        } finally {
            driver.quit();
        }
    }
}

Practical 6: Web Elements, Alerts, Frames and Waits

1. No wait at all
   NoSuchElementException: the paragraph did not exist yet
2. Implicit wait of 5 seconds
   found: Marks loaded: 78 out of 100
   findElement waited between 2.5 and 5 seconds
3. Explicit wait of 5 seconds for visibility
   found: Marks loaded: 78 out of 100
   the wait took between 2.5 and 5 seconds
4. Explicit wait of only 2 seconds
   TimeoutException, and the first two lines of its message:
   Expected condition failed: waiting for visibility of element found by By.id: marksResult, but... org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"#marksResult"}.
   (tried for 2 seconds with 500 milliseconds interval)

What the four runs show:

No wait. The script asked for the paragraph a moment after the click, and it did not exist. This is the failure a flaky test is made of: on a slow machine or a slow day it fails, on a fast one the same code might not.
Implicit wait. With a five-second implicit wait set, the same findElement kept looking until the paragraph appeared, about three seconds later, and returned it. It did not wait the full five: an implicit wait ends the moment the element is found. The program then sets the implicit wait back to zero before using an explicit wait, as the documentation's rule requires.
Explicit wait. WebDriverWait with visibilityOfElementLocated did the same for this one step only, and returned the element once it was both present and visible.
Too short. Two seconds was not enough for a three-second delay, and the wait gave up with a TimeoutException. Its first line says what it was waiting for, and what it last found (no such element); its second line says how long it tried and how often it checked. It is the most useful error message in Selenium.

Practical 6: Web Elements, Alerts, Frames and Waits

Implicit or explicit? The implicit wait is one line and covers every findElement, but it can only wait for an element to exist. An explicit wait can wait for any condition: visible, clickable, containing some text, a URL, an alert. Most frameworks use explicit waits where they are needed and no implicit wait at all, which is also the only way to be certain of never mixing them.

A few of the ready-made conditions in ExpectedConditions, each used in some chapter of this book:

Condition	Waits until	Takes
`visibilityOfElementLocated`	the element exists and is visible	a locator
`elementToBeClickable`	the element is visible and enabled	a locator or an element
`textToBePresentInElementLocated`	the element contains the text	a locator and the text
`urlContains`	the browser's address contains the text	the text
`alertIsPresent`	an alert is open	nothing

Never Thread.sleep() for synchronisation. A sleep always waits its full time, so it is either too short on a slow day or wastes seconds on every fast one. Selenium's documentation makes the same point: code cannot know exactly how long it needs to wait.

Observations

Element or wait	Action	Result
Text box	typed a name	value read back; getText() empty
Radio buttons	FY then TY	only TY selected
Checkboxes	Java and SQL	2 of 3 ticked
Dropdown	by text, value, index	Pune, Thane, Mumbai
Alert	accepted	text read, page shows Saved
Confirm	dismissed	page shows You pressed Cancel
Prompt	typed 42, accepted	page shows Roll number: 42
Frame	switched in and out	feedback sent; outer heading read
No wait	find at once	NoSuchElementException
Implicit 5 s	find	found after about 3 s
Explicit 5 s	visibility	found after about 3 s
Explicit 2 s	visibility	TimeoutException

Result

A script automated a text box, radio buttons, checkboxes, a dropdown, an alert, a confirm, a prompt and a frame, and every interaction gave the expected result. A delayed element was not found with no wait, was found with an implicit wait of five seconds and with an explicit wait of five seconds, each after about three, and an explicit wait of two seconds timed out with a message naming its condition.

Where marks are lost

getText() on a text box. It is empty; read the value.

Index 1 expected to be the first real option. Indexes start at 0, and the placeholder takes it.

Forgetting defaultContent() after working in a frame, and then failing to find anything on the main page.

Clicking something while an alert is open. Accept or dismiss it first.

Practical 6: Web Elements, Alerts, Frames and Waits

Mixing an implicit and an explicit wait. Selenium's documentation warns it can cause unpredictable wait times.

Thread.sleep(5000) instead of a wait. Slow when it works, flaky when it does not.

For the journal

Aim; the page under test; Program 1 and its output with the table of elements; Program 2 and its output; the difference between implicit and explicit waits in your own words; the observations table; the result.

Quick revision

Text box: sendKeys, read with getDomProperty("value"). Radio and checkbox: click, isSelected.
Dropdown: new Select(element), selectByVisibleText, selectByValue, selectByIndex (from 0).
Alerts: switchTo().alert(), getText, accept (OK), dismiss (Cancel), sendKeys (prompt).
Frames: switchTo().frame(name or id or element), back with switchTo().defaultContent().
Implicit wait: global, only for finding elements. Explicit wait: one step, any condition.
Never mix them. Never use Thread.sleep to synchronise.
TimeoutException's message names the condition, the time and the polling interval.

Questions you must be able to answer

1. Why is a text box read with getDomProperty("value")? Typed text becomes the box's value property, not text inside the element, so getText() returns nothing.

2. What is the difference between accept() and dismiss()? accept() presses OK; dismiss() presses Cancel, or closes a plain alert.

3. Why can an element inside an iframe not be found directly? A frame holds a separate document, and WebDriver searches only the current one. Switch into the frame first.

4. What is the difference between an implicit and an explicit wait? An implicit wait is a global setting that makes every findElement keep looking for an element to exist; an explicit wait waits, for one step, for a chosen condition such as visibility.

5. Why must the two not be mixed? Selenium's documentation warns it can cause unpredictable wait times, for example a timeout after 20 seconds from waits of 10 and 15.

6. The implicit wait was 5 seconds. Why did findElement return after about 3? An implicit wait ends as soon as the element is found; the paragraph appeared after three seconds.

7. Why not use Thread.sleep()? It always waits its full time: too short on a slow run, wasted time on every fast one.

Contents This chapter on its own page

Chapter Ten

Practical 7: Data-Driven Testing with Apache POI

Syllabus topic Module 1, "Data-Driven Testing Using Excel Integration", "Tool: Selenium WebDriver + Apache POI Write a program to update 10 student records in an Excel file and validate changes on the web application. Perform data reading, writing, and verification using Apache POI library."

Aim

To keep the data for a test in an Excel file, to update ten student records on the web application from that file, to validate every change on the application, and to write each result back into the file, using Apache POI for the reading and the writing.

What you need to know before you start

Data-driven testing separates the test's steps from the test's data. The steps are written once, in code: open the record, change it, save it, check it. The data, which record and what to change it to, lives in a table outside the code. Ten records, or a thousand, are the same program run over more rows. A tester can add a case without touching Java at all.

Apache POI is the Java library that reads and writes Microsoft Office files. For Excel's modern .xlsx format its classes start with XSSF: XSSFWorkbook is a whole file. POI's quick guide shows the interface names that work for both old and new formats: a Workbook holds Sheets, a Sheet holds Rows, and a Row holds Cells. Rows and cells are numbered from 0, so row 0 is the heading row and Excel's row 2 is POI's row 1.

A cell has a type. A cell holding 75 is numeric, and POI returns numbers as double, so reading it naively gives 75.0. POI's quick guide offers DataFormatter, which returns the text exactly as Excel would display it, 75. That one class prevents the commonest mistake in this practical.

Reading, writing and verification are MU's three words:

Read each row of updates from the workbook.
Apply it on the web page, and verify on the page that the change took, or that the page refused it when it should.
Write the result into the same row, and save the workbook.

The pages under test

practice/students.php lists the ten records. Each row has an Edit link, and the table rows carry ids of the form row-101, which makes each record easy to find:

<?php
require 'inc/store.php';
$students = load_records('students');
$title = 'Student Records';
include 'inc/top.php';
?>
<h1>Student Records</h1>
<?php if (isset($_GET['updated'])): ?>
  <p id="message" class="success">Record <?= (int) $_GET['updated'] ?> updated.</p>
<?php endif; ?>
<table id="students">
  <tr><th>Roll</th><th>Name</th><th>Course</th><th>City</th><th>Marks</th><th></th></tr>
<?php foreach ($students as $s): ?>
  <tr id="row-<?= $s['roll'] ?>">
    <td><?= $s['roll'] ?></td>
    <td><?= htmlspecialchars($s['name']) ?></td>
    <td><?= htmlspecialchars($s['course']) ?></td>
    <td><?= htmlspecialchars($s['city']) ?></td>
    <td><?= $s['marks'] ?></td>
    <td><a href="edit-student.php?roll=<?= $s['roll'] ?>">Edit</a></td>
  </tr>
<?php endforeach; ?>
</table>
<?php include 'inc/bottom.php'; ?>

practice/edit-student.php edits one record's city and marks, and refuses marks that are not a whole number from 0 to 100:

<?php
require 'inc/store.php';
$students = load_records('students');
$roll = (int) ($_GET['roll'] ?? 0);
$index = null;
foreach ($students as $i => $s) {
    if ($s['roll'] === $roll) { $index = $i; }
}
if ($index === null) {
    http_response_code(404);
    exit('No student with roll number ' . $roll);
}
$error = '';
if ($_SERVER['REQUEST_METHOD'] === 'POST') {
    $city = trim($_POST['city'] ?? '');
    $marks = $_POST['marks'] ?? '';
    if ($city === '') {
        $error = 'City cannot be empty.';
    } elseif (!ctype_digit($marks) || (int) $marks > 100) {
        $error = 'Marks must be a whole number from 0 to 100.';
    } else {
        $students[$index]['city'] = $city;
        $students[$index]['marks'] = (int) $marks;
        save_records('students', $students);
        header('Location: students.php?updated=' . $roll);
        exit;
    }
}
$s = $students[$index];
$title = 'Edit Student';
include 'inc/top.php';
?>
<h1>Edit Student <?= $s['roll'] ?></h1>
<p>Name: <span id="name"><?= htmlspecialchars($s['name']) ?></span></p>
<?php if ($error !== ''): ?>
  <p id="error" class="error"><?= htmlspecialchars($error) ?></p>
<?php endif; ?>
<form method="post">
  <label>City <input type="text" name="city" id="city" value="<?= htmlspecialchars($s['city']) ?>"></label>
  <label>Marks <input type="text" name="marks" id="marks" value="<?= $s['marks'] ?>"></label>
  <p><button type="submit" id="save">Save</button></p>
</form>
<?php include 'inc/bottom.php'; ?>

Practical 7: Data-Driven Testing with Apache POI

The records start as the ten in seed/students.json, printed in Practical 1. After this practical changes them, localhost:8080/reset.php puts them back.

Step 1: the workbook of updates

The workbook has one row per update. Expected says what the application should do with it: nine updates should be saved, and the last, marks of 105, should be refused. A data-driven test must include the rows that should fail. Result and Remark are left empty for the program to fill.

Roll	New city	New marks	Expected
101	Pune	75	updated
102	Mumbai	68	updated
103	Thane	61	updated
104	Nashik	85	updated
105	Pune	52	updated
106	Thane	79	updated
107	Mumbai	93	updated
108	Nashik	66	updated
109	Pune	57	updated
110	Mumbai	105	rejected

You can type this into Excel or Google Sheets and save it as student-updates.xlsx in the project folder. Or let POI make it, which is the writing half of the practical in its simplest form. Create practicals/MakeUpdates.java:

package practicals;

import java.io.FileOutputStream;

import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;

public class MakeUpdates {
    public static void main(String[] args) throws Exception {
        String[] heading = {"Roll", "New city", "New marks", "Expected", "Result", "Remark"};
        Object[][] updates = {
            {101, "Pune", 75, "updated"}, {102, "Mumbai", 68, "updated"},
            {103, "Thane", 61, "updated"}, {104, "Nashik", 85, "updated"},
            {105, "Pune", 52, "updated"}, {106, "Thane", 79, "updated"},
            {107, "Mumbai", 93, "updated"}, {108, "Nashik", 66, "updated"},
            {109, "Pune", 57, "updated"}, {110, "Mumbai", 105, "rejected"},
        };
        try (Workbook book = new XSSFWorkbook()) {
            Sheet sheet = book.createSheet("Updates");
            Row top = sheet.createRow(0);
            for (int c = 0; c < heading.length; c++) {
                top.createCell(c).setCellValue(heading[c]);
            }
            for (int r = 0; r < updates.length; r++) {
                Row row = sheet.createRow(r + 1);
                row.createCell(0).setCellValue((Integer) updates[r][0]);   // a number cell
                row.createCell(1).setCellValue((String) updates[r][1]);    // a text cell
                row.createCell(2).setCellValue((Integer) updates[r][2]);
                row.createCell(3).setCellValue((String) updates[r][3]);
            }
            try (FileOutputStream out = new FileOutputStream("student-updates.xlsx")) {
                book.write(out);
            }
            System.out.println("Wrote student-updates.xlsx: " + updates.length + " updates on sheet "
                    + sheet.getSheetName());
        }
    }
}

Practical 7: Data-Driven Testing with Apache POI

Wrote student-updates.xlsx: 10 updates on sheet Updates

try (...) closes the workbook and the file even if something goes wrong halfway, and a workbook that is never closed can leave the file incomplete.

Why every cell is read through DataFormatter

Two small facts decide whether a POI program works on the first try, and this program shows both on the workbook just made. Create practicals/CellTypes.java:

package practicals;

import java.io.FileInputStream;

import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.DataFormatter;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;

public class CellTypes {
    public static void main(String[] args) throws Exception {
        try (Workbook book = new XSSFWorkbook(new FileInputStream("student-updates.xlsx"))) {
            Row first = book.getSheet("Updates").getRow(1);
            Cell marks = first.getCell(2);
            System.out.println("Cell type of New marks: " + marks.getCellType());
            System.out.println("getNumericCellValue():  " + marks.getNumericCellValue());
            System.out.println("DataFormatter:          " + new DataFormatter().formatCellValue(marks));

            Cell result = first.getCell(4);            // nobody has written a Result yet
            System.out.println("Result cell object:     " + result);
            System.out.println("DataFormatter on it:    [" + new DataFormatter().formatCellValue(result) + "]");
        }
    }
}

Cell type of New marks: NUMERIC
getNumericCellValue():  75.0
DataFormatter:          75
Result cell object:     null
DataFormatter on it:    []

The marks were written as a number, so the cell's type is NUMERIC, and POI hands back every number as a double: 75.0. Put that into the web page's marks box and the page, which wants a whole number, refuses it. DataFormatter returns 75, exactly what Excel shows. And the Result cell, which nothing has written yet, does not exist: getCell(4) returned null. DataFormatter turns that into an empty string where calling any method on null would have thrown NullPointerException.

Step 2: read, update, verify, write

The main program. For every row it reads the update, makes it through the edit page, checks the students page, and writes what happened into the Result and Remark cells. Create UpdateStudents.java, in the same practicals folder:

package practicals;

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.time.Duration;

import org.apache.poi.ss.usermodel.DataFormatter;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class UpdateStudents {
    static final String SITE = "http://localhost:8080/";
    static final String FILE = "student-updates.xlsx";

    public static void main(String[] args) throws Exception {
        DataFormatter text = new DataFormatter();   // cells as Excel shows them: 75, not 75.0
        WebDriver driver = new ChromeDriver();
        WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10));
        int passed = 0, failed = 0;
        try (Workbook book = new XSSFWorkbook(new FileInputStream(FILE))) {
            Sheet sheet = book.getSheet("Updates");
            for (int r = 1; r <= sheet.getLastRowNum(); r++) {
                Row row = sheet.getRow(r);
                String roll = text.formatCellValue(row.getCell(0));
                String city = text.formatCellValue(row.getCell(1));
                String marks = text.formatCellValue(row.getCell(2));
                String expected = text.formatCellValue(row.getCell(3));

                // Apply the update on the web page.
                driver.get(SITE + "edit-student.php?roll=" + roll);
                WebElement cityBox = driver.findElement(By.id("city"));
                cityBox.clear();
                cityBox.sendKeys(city);
                WebElement marksBox = driver.findElement(By.id("marks"));
                marksBox.clear();
                marksBox.sendKeys(marks);
                driver.findElement(By.id("save")).click();

                // Verify on the page what actually happened. A saved record sends the
                // browser to students.php?updated=<roll>: wait for that page first.
                String result, remark;
                if (expected.equals("updated")) {
                    wait.until(ExpectedConditions.urlContains("updated=" + roll));
                    WebElement tr = driver.findElement(By.id("row-" + roll));
                    String shownCity = tr.findElement(By.xpath("td[4]")).getText();
                    String shownMarks = tr.findElement(By.xpath("td[5]")).getText();
                    boolean ok = shownCity.equals(city) && shownMarks.equals(marks);
                    result = ok ? "PASS" : "FAIL";
                    remark = "page shows " + shownCity + ", " + shownMarks;
                } else {
                    String error = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("error"))).getText();
                    result = error.startsWith("Marks must be") ? "PASS" : "FAIL";
                    remark = "page refused: " + error;
                }
                if (result.equals("PASS")) passed++; else failed++;

                // Write the result back into the same row.
                row.createCell(4).setCellValue(result);
                row.createCell(5).setCellValue(remark);
                System.out.println(roll + "  " + city + ", " + marks + "  expected " + expected + "  " + result);
            }
            try (FileOutputStream out = new FileOutputStream(FILE)) {
                book.write(out);
            }
        } finally {
            driver.quit();
        }
        System.out.println("Records processed: " + (passed + failed) + ", PASS: " + passed + ", FAIL: " + failed);
    }
}

Practical 7: Data-Driven Testing with Apache POI

101  Pune, 75  expected updated  PASS
102  Mumbai, 68  expected updated  PASS
103  Thane, 61  expected updated  PASS
104  Nashik, 85  expected updated  PASS
105  Pune, 52  expected updated  PASS
106  Thane, 79  expected updated  PASS
107  Mumbai, 93  expected updated  PASS
108  Nashik, 66  expected updated  PASS
109  Pune, 57  expected updated  PASS
110  Mumbai, 105  expected rejected  PASS
Records processed: 10, PASS: 10, FAIL: 0

Wait for the page the save leads to. A saved record sends the browser on to students.php?updated=101, and the program waits for exactly that address before it reads anything. The first version of this program did not: it clicked Save and at once opened the students page itself. Run three times in the laboratory, it failed a different row each time, 105 on one run, 103 and 106 on another, and on the third it crashed looking for an element, because it sometimes read the list before the save had reached the server. That is the flaky test Practical 6 described, met for real, and the cure is the same: wait for the page's own sign that the step is done.

How the verification finds the right values: By.id("row-" + roll) finds the table row for that student, and td[4] and td[5], searched from that row, are its fourth and fifth cells, city and marks. An XPath that starts without // searches from the element it is called on, which is what keeps the check to the right student.

Practical 7: Data-Driven Testing with Apache POI

The tenth row is the one worth pointing at in the journal. The page refused marks of 105 with its own message, which is what the row expected, so the row passes. A data-driven test that only ever feeds good data tests half the page.

Step 3: read back what was written

The last step proves the writing: open the workbook again and print it. Create practicals/ReadResults.java:

package practicals;

import java.io.FileInputStream;

import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.DataFormatter;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;

public class ReadResults {
    public static void main(String[] args) throws Exception {
        DataFormatter text = new DataFormatter();
        try (Workbook book = new XSSFWorkbook(new FileInputStream("student-updates.xlsx"))) {
            Sheet sheet = book.getSheet("Updates");
            for (Row row : sheet) {
                StringBuilder line = new StringBuilder();
                for (Cell cell : row) {
                    line.append(String.format("%-10s", text.formatCellValue(cell)));
                }
                System.out.println(line.toString().stripTrailing());
            }
        }
    }
}

Roll      New city  New marks Expected  Result    Remark
101       Pune      75        updated   PASS      page shows Pune, 75
102       Mumbai    68        updated   PASS      page shows Mumbai, 68
103       Thane     61        updated   PASS      page shows Thane, 61
104       Nashik    85        updated   PASS      page shows Nashik, 85
105       Pune      52        updated   PASS      page shows Pune, 52
106       Thane     79        updated   PASS      page shows Thane, 79
107       Mumbai    93        updated   PASS      page shows Mumbai, 93
108       Nashik    66        updated   PASS      page shows Nashik, 66
109       Pune      57        updated   PASS      page shows Pune, 57
110       Mumbai    105       rejected  PASS      page refused: Marks must be a whole number from 0 to 100.

The Result and Remark columns are now filled, in the same file the test read its data from: that file is both the test's input and its report.

Step 4: the same data, a different run

Change a row in the workbook, for example 103's city to Nashik, and run UpdateStudents again without touching the code. That is the point of data-driven testing: the test changed, the program did not. Reset the site first (reset.php) if you want every row to start from the seed.

Observations

Rows read	Updates saved and verified	Refusals verified	FAIL
10	9	1	0

The workbook's Result column holds PASS for all ten rows, and its Remark column records what the page showed for each.

Result

Ten student updates were read from an Excel workbook with Apache POI and applied on the Practice Portal by Selenium WebDriver. Nine were saved, and each was verified on the students page; the tenth, marks of 105, was refused by the page as expected. The result of every row was written back into the workbook and read back from it.

Where marks are lost

Reading a number cell as a number. You get 75.0. Use DataFormatter.

Practical 7: Data-Driven Testing with Apache POI

Forgetting row 0 is the heading. Start the loop at row 1.

Checking the page before saving, or the wrong row. Find the student's own table row and read its cells from there.

No failing data. Include at least one row the application should refuse, with the expectation written in the file.

Not closing the workbook. Use try-with-resources, or call close(). On Windows, close the file in Excel too before the program writes to it.

Hard-coding the ten records in Java. Then it is not data-driven. The data belongs in the file.

For the journal

Aim; tool (Selenium WebDriver 4.49 and Apache POI 5.5.1); the workbook's layout with its Expected column; the three programs; their output; the final contents of the workbook; the observations table; the result.

Quick revision

Data-driven testing: steps in code, data in a file; add rows, not code.
POI: Workbook, Sheet, Row, Cell. .xlsx is XSSF. Rows and cells count from 0.
Read cells with a DataFormatter and its formatCellValue(cell): it gives what Excel shows and handles null.
Write with createCell(c) on the row and setCellValue(v) on the cell, then book.write(out).
Verify on the page: find the student's row by id, then its cells.
Put expected failures in the data.
try-with-resources closes the workbook and the file.

Questions you must be able to answer

1. What is data-driven testing? Running the same test steps over many sets of data kept outside the code, so that cases are added by adding rows, not code.

2. Why does reading marks of 75 give 75.0? POI returns every numeric cell as a double. DataFormatter returns the value as Excel displays it.

3. What are Workbook, Sheet, Row and Cell? The file, one tab in it, one row of a tab, and one cell of a row. POI numbers rows and cells from 0.

4. How does the program know it is reading the right student's marks on the page? It finds the table row whose id is row- followed by the roll number, and reads the cells from that row.

5. Why is the row with marks of 105 marked PASS? Its expected result was that the page would refuse it, and the page did, with its own error message.

6. What happens if you call a method on a cell that was never created? getCell returns null, and calling a method on null throws NullPointerException. DataFormatter accepts null.

Contents This chapter on its own page

Chapter Eleven

Practical 8: Extracting Marks from a Dynamic Web Table

Syllabus topic Module 1, "Data Extraction and Score Analysis from Web Table", "Tool: Selenium WebDriver", "Automate extraction of student marks from a dynamic web table and determine the number of students scoring above 60 in at least one subject. Compute total count and percentage."

Aim

To extract student marks from a dynamic web table, to determine the number of students who scored above 60 in at least one subject, and to compute that count and its percentage of all students.

What you need to know before you start

A dynamic web table is one whose rows are not in the page the server sends. The page arrives with an empty table, and the page's JavaScript fetches the data and adds the rows a moment later. A script that reads the table the instant the page loads finds no rows, and a count of students scoring above 60 comes out as zero, which is a wrong answer that looks like a right one. So the script must wait until the table is full.

What to wait for. Wait for something the page does when it has finished, not for a number of seconds. This page replaces its "Loading marks..." line with "12 students loaded." once every row is in, so the script waits for the words "students loaded" to appear in that line.

Reading a table. A table is rows (tr) of cells (td). findElements with the CSS selector #marksBody tr gives every data row, and on each row findElements with By.tagName("td") gives its cells in order. Searching from a row, rather than from the whole page, keeps each student's marks together.

"Above 60" means more than 60. A mark of exactly 60 is not above 60. This one word is where most answers to this practical go wrong, and the table below contains four marks of exactly 60 to find them out.

Absent is not zero. A cell showing AB means the student was absent. It is not a mark, and a program that tries to turn it into a number with Integer.parseInt crashes with NumberFormatException. The student may still have scored above 60 in another subject.

The page under test

practice/marks.php sends an empty table and fills it from api/marks.php after one and a half seconds:

<?php $title = 'Marks'; include 'inc/top.php'; ?>
<h1>Semester Marks</h1>
<p id="status">Loading marks...</p>
<table id="marks">
  <thead>
    <tr><th>Roll</th><th>Name</th><th>Maths</th><th>Physics</th><th>Chemistry</th><th>English</th></tr>
  </thead>
  <tbody id="marksBody"></tbody>
</table>
<script>
// The rows are not in the page the server sends. They are fetched and drawn
// a moment later, which is what makes this table "dynamic".
setTimeout(function () {
  fetch('api/marks.php')
    .then(function (response) { return response.json(); })
    .then(function (rows) {
      const body = document.getElementById('marksBody');
      rows.forEach(function (r) {
        const tr = document.createElement('tr');
        [r.roll, r.name, r.maths, r.physics, r.chemistry, r.english].forEach(function (value) {
          const td = document.createElement('td');
          td.textContent = value;
          tr.appendChild(td);
        });
        body.appendChild(tr);
      });
      document.getElementById('status').textContent = rows.length + ' students loaded.';
    });
}, 1500);
</script>
<?php include 'inc/bottom.php'; ?>

practice/api/marks.php serves the records in practice/seed/marks.json:

Practical 8: Extracting Marks from a Dynamic Web Table

<?php
// api/marks.php: the marks the marks table loads. "AB" means absent.
require __DIR__ . '/../inc/store.php';
header('Content-Type: application/json; charset=utf-8');
echo json_encode(load_records('marks'));

[
    {"roll": 1, "name": "Aditi Kulkarni", "maths": "78", "physics": "65", "chemistry": "59", "english": "70"},
    {"roll": 2, "name": "Bhavesh Mehta", "maths": "55", "physics": "60", "chemistry": "48", "english": "52"},
    {"roll": 3, "name": "Chitra Menon", "maths": "61", "physics": "45", "chemistry": "50", "english": "58"},
    {"roll": 4, "name": "Dev Malhotra", "maths": "60", "physics": "60", "chemistry": "60", "english": "60"},
    {"roll": 5, "name": "Esha Pillai", "maths": "AB", "physics": "72", "chemistry": "66", "english": "80"},
    {"roll": 6, "name": "Farhan Qureshi", "maths": "40", "physics": "38", "chemistry": "55", "english": "59"},
    {"roll": 7, "name": "Gauri Deshpande", "maths": "92", "physics": "88", "chemistry": "95", "english": "90"},
    {"roll": 8, "name": "Harsh Vora", "maths": "59", "physics": "61", "chemistry": "57", "english": "44"},
    {"roll": 9, "name": "Isha Banerjee", "maths": "50", "physics": "AB", "chemistry": "49", "english": "58"},
    {"roll": 10, "name": "Jay Thakur", "maths": "35", "physics": "42", "chemistry": "60", "english": "61"},
    {"roll": 11, "name": "Kavya Reddy", "maths": "58", "physics": "57", "chemistry": "56", "english": "55"},
    {"roll": 12, "name": "Laksh Chopra", "maths": "100", "physics": "20", "chemistry": "30", "english": "40"}
]

The expected answer, worked by hand first

A tester works out the expected result before running anything. For each student, the subjects with a mark above 60:

Roll	Name	Maths	Physics	Chemistry	English	Above 60 in	Counts
1	Aditi Kulkarni	78	65	59	70	Maths, Physics, English	yes
2	Bhavesh Mehta	55	60	48	52	none (60 is not above 60)	no
3	Chitra Menon	61	45	50	58	Maths	yes
4	Dev Malhotra	60	60	60	60	none	no
5	Esha Pillai	AB	72	66	80	Physics, Chemistry, English	yes
6	Farhan Qureshi	40	38	55	59	none	no
7	Gauri Deshpande	92	88	95	90	all four	yes
8	Harsh Vora	59	61	57	44	Physics	yes
9	Isha Banerjee	50	AB	49	58	none	no
10	Jay Thakur	35	42	60	61	English	yes
11	Kavya Reddy	58	57	56	55	none	no
12	Laksh Chopra	100	20	30	40	Maths	yes

Seven of the twelve count. The percentage is 7 out of 12 times 100, about 58.33 per cent. The cases that catch a careless program are Bhavesh and Dev (60 is not above 60), Esha (absent in one subject, above 60 in three), Isha (absent, and below 60 everywhere else), and Jay (60 in one subject, 61 in another).

The program

Create practicals/MarksAnalysis.java:

package practicals;

import java.time.Duration;
import java.util.ArrayList;
import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class MarksAnalysis {
    public static void main(String[] args) {
        WebDriver driver = new ChromeDriver();
        try {
            driver.get("http://localhost:8080/marks.php");

            // Wait for the page's own sign that every row has arrived.
            new WebDriverWait(driver, Duration.ofSeconds(10))
                .until(ExpectedConditions.textToBePresentInElementLocated(By.id("status"), "students loaded"));

            List<String> subjects = new ArrayList<>();
            List<WebElement> head = driver.findElements(By.cssSelector("#marks thead th"));
            for (int c = 2; c < head.size(); c++) {          // columns 0 and 1 are Roll and Name
                subjects.add(head.get(c).getText());
            }

            List<WebElement> rows = driver.findElements(By.cssSelector("#marksBody tr"));
            int total = rows.size(), qualifying = 0;
            for (WebElement row : rows) {
                List<WebElement> cells = row.findElements(By.tagName("td"));
                String name = cells.get(1).getText();
                List<String> above = new ArrayList<>();
                for (int c = 2; c < cells.size(); c++) {
                    String mark = cells.get(c).getText();
                    if (mark.equals("AB")) continue;          // absent: not a mark
                    if (Integer.parseInt(mark) > 60) {        // above 60 means more than 60
                        above.add(subjects.get(c - 2));
                    }
                }
                if (!above.isEmpty()) qualifying++;
                System.out.printf("%-16s %s%n", name, above.isEmpty() ? "-" : String.join(", ", above));
            }

            double percentage = Math.round(qualifying * 100.0 / total * 100) / 100.0;
            System.out.println();
            System.out.println("Students in the table: " + total);
            System.out.println("Scoring above 60 in at least one subject: " + qualifying);
            System.out.println("Percentage: " + percentage + "%");
        } finally {
            driver.quit();
        }
    }
}

Practical 8: Extracting Marks from a Dynamic Web Table

Run it:

Aditi Kulkarni   Maths, Physics, English
Bhavesh Mehta    -
Chitra Menon     Maths
Dev Malhotra     -
Esha Pillai      Physics, Chemistry, English
Farhan Qureshi   -
Gauri Deshpande  Maths, Physics, Chemistry, English
Harsh Vora       Physics
Isha Banerjee    -
Jay Thakur       English
Kavya Reddy      -
Laksh Chopra     Maths

Students in the table: 12
Scoring above 60 in at least one subject: 7
Percentage: 58.33%

The program's answer, 7 of 12 and 58.33 per cent, is the answer worked by hand, and each student's list of subjects matches the table above row for row. That agreement is the test: a count that merely looks plausible proves nothing, and an expected result worked out in advance is what makes the program's output checkable.

Four lines of the program carry the logic of the exercise:

The wait for the text "students loaded" in By.id("status") holds the program until the page says it has finished.
continue on AB skips an absent subject without crashing and without counting it as zero.
> 60, not >= 60.
The percentage is computed in double (qualifying 100.0 / total). Written as qualifying 100 / total it would be integer division, which gives 58 and silently loses the fraction.

Math.round(value * 100) / 100.0 rounds to two decimal places.

What happens without the wait. This short program opens the same page and counts the rows at once, and then again after waiting:

package practicals;

import java.time.Duration;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class MarksNoWait {
    public static void main(String[] args) {
        WebDriver driver = new ChromeDriver();
        try {
            driver.get("http://localhost:8080/marks.php");
            System.out.println("Rows at once:        " + driver.findElements(By.cssSelector("#marksBody tr")).size());
            System.out.println("Status line at once: " + driver.findElement(By.id("status")).getText());
            new WebDriverWait(driver, Duration.ofSeconds(10))
                .until(ExpectedConditions.textToBePresentInElementLocated(By.id("status"), "students loaded"));
            System.out.println("Rows after the wait: " + driver.findElements(By.cssSelector("#marksBody tr")).size());
        } finally {
            driver.quit();
        }
    }
}

Practical 8: Extracting Marks from a Dynamic Web Table

Rows at once:        0
Status line at once: Loading marks...
Rows after the wait: 12

With no rows, the analysis program would report 0 students and 0 qualifying, and no error at all. That is the danger of a missing wait here: not a crash, but a confident wrong answer.

Observations

Measure	By hand	By the program
Students in the table	12	12
Scoring above 60 in at least one subject	7	7
Percentage	58.33	58.33

Result

The marks of all 12 students were extracted from the dynamic table after waiting for it to load. Seven students scored above 60 in at least one subject, 58.33 per cent of the class. The result agrees with the expected answer worked out by hand from the source data.

Where marks are lost

No wait. The table is empty for the first moment and every count is zero.

>= 60. Dev Malhotra, with 60 in everything, is not above 60 in anything.

Integer.parseInt("AB"). NumberFormatException. Skip absent subjects.

Integer division for the percentage. 7 * 100 / 12 is 58 in Java's integer arithmetic.

Counting subjects instead of students. The question is how many students, and a student with three marks above 60 counts once.

No expected result. Work the answer out by hand first, so the program's answer can be checked.

For the journal

Aim; the page under test and why it is dynamic; the hand-worked table of expected results; the program; its output; the comparison of hand and program; the result.

Quick revision

Dynamic table: rows added by JavaScript after the page loads. Wait for the page's own "done" sign.
Rows: #marksBody tr; cells of one row: findElements on the row, with By.tagName("td").
Above 60 is > 60. Absent (AB) is skipped, not zero.
Count students, not subjects.
Percentage in double: count * 100.0 / total.
Work the expected answer out by hand before running.

Questions you must be able to answer

1. What makes this table dynamic? Its rows are not in the page the server sends; JavaScript fetches them and adds them after the page has loaded.

2. What does the program wait for, and why not a fixed number of seconds? For the status line to say "students loaded", the page's own sign that every row is in. A fixed time is too short on a slow run and wasted on a fast one.

3. Does Dev Malhotra, with 60 in every subject, count? Why? No. Sixty is not above 60.

4. Esha Pillai was absent for Maths. Does she count? Yes. Absence is skipped, and she scored above 60 in Physics, Chemistry and English.

Practical 8: Extracting Marks from a Dynamic Web Table

5. Why is the percentage computed with 100.0 and not 100? With 100 the expression is integer division and gives 58; with 100.0 it is computed in double and gives 58.33 after rounding.

6. What would the program find if the wait were removed? No rows at all: the page still says "Loading marks...". The analysis would report 0 students without any error, a wrong answer that looks like a result.

Contents This chapter on its own page

Chapter Twelve

Practical 9: Identifying and Counting the Objects on a Page

Syllabus topic Module 1, "Object Identification and Counting on Web Page", "Tool: Selenium WebDriver", "Write a program to identify and count the total number of objects present on a web page including links, images, buttons, and input elements. Display categorized counts."

Aim

To identify and count the objects on a web page, links, images, buttons and input elements, and to display the counts by category.

What you need to know before you start

The exercise sounds like counting and is really about definitions. "How many buttons are on this page?" has more than one right answer, depending on what you mean by a button, and a program only counts what its locator describes. A good answer to this practical states each definition and shows how the count changes with it.

The questions that change the counts:

Is an a tag with no href a link? It looks like a tag for a link, but it goes nowhere, and a browser does not treat it as a link.
Is a hidden element "present"? It is in the page's HTML, but no user can see it or click it.
Is <input type="submit"> a button or an input? It is an input element that behaves as a button.
Is <input type="hidden"> an input? It is an input element that holds data and shows nothing.
Is an image inside a link a link, an image, or both? Both: it is two elements.

A CSS attribute selector writes such definitions exactly. a[href] means "a elements that have an href attribute". input[type=submit] means "inputs whose type is submit". :not(...) excludes a match. Every count in this chapter is one selector.

The page under test

practice/objects.php is a complete page on its own, with no shared header, so that everything counted is visible in one listing:

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Objects - Practice Portal</title>
<link rel="stylesheet" href="assets/site.css">
</head>
<body>
<main>
<h1>Objects on a Page</h1>
<p>
  <a href="index.php">Home</a>
  <a href="about.php">About</a>
  <a href="login.php">Login</a>
  <a href="mailto:office@example.com">Email the office</a>
  <a href="about.php"><img src="assets/logo.png" alt="Portal logo"></a>
  <a href="results.php" class="hidden">Results</a>
  <a id="notes">Notes (an anchor with no address)</a>
</p>
<h2>Gallery</h2>
<img src="assets/campus.png" alt="Campus">
<img src="assets/library.png" alt="Library">
<img src="assets/lab.png" alt="Laboratory">
<h2>Search</h2>
<form>
  <input type="text" name="q" placeholder="Search">
  <input type="email" name="email" placeholder="Email">
  <input type="password" name="pin" placeholder="PIN">
  <input type="hidden" name="source" value="objects">
  <label><input type="checkbox" name="exact"> Exact match</label>
  <label><input type="radio" name="scope" value="all" checked> All</label>
  <label><input type="radio" name="scope" value="titles"> Titles only</label>
  <select name="dept">
    <option>All departments</option>
    <option>Computer Science</option>
  </select>
  <textarea name="notes" rows="2"></textarea>
  <button type="submit">Search</button>
  <button type="button">Print</button>
  <input type="submit" value="Go">
  <input type="button" value="Clear">
  <input type="reset" value="Reset">
</form>
</main>
</body>
</html>

The four pictures, logo.png, campus.png, library.png and lab.png in practice/assets, can be any small images: rename four pictures you already have. None of the counts depends on what the pictures show, and even a missing picture is still an img element in the page.

The expected counts, from the source

Counted by hand from the listing above, before running anything:

Category	Definition	Selector	Count
Anchor tags	every `a` element	`a`	7
Links	`a` elements with an address	`a[href]`	6
Visible links	links a user can see	`a[href]`, displayed	5
Images	every `img` element	`img`	4
Button elements	`button` tags	`button`	2
Input buttons	inputs of type submit, button or reset	`input[type=submit], input[type=button], input[type=reset]`	3
All buttons	both of the above		5
Input tags	every `input` element	`input`	10
Entry inputs	inputs a user types into or chooses	inputs, not hidden and not buttons	6
Other form fields	`select` and `textarea`	`select, textarea`	2

Practical 9: Identifying and Counting the Objects on a Page

Where each number comes from:

7 anchors, 6 links, 5 visible. The a with id="notes" has no href, so it is not a link. The Results link has the class hidden, which the style sheet sets to display: none, so it is a link nobody can see.
4 images. The logo inside the link is an image as well as being inside a link.
5 buttons. Two button tags, and three inputs of type submit, button and reset.
10 inputs, 6 for entry. The ten input tags include the hidden one and the three button types; the six left are text, email, password, the checkbox and the two radios.

The program

Create practicals/CountObjects.java:

package practicals;

import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;

public class CountObjects {
    static WebDriver driver;

    static int count(String css) {
        return driver.findElements(By.cssSelector(css)).size();
    }

    static int countVisible(String css) {
        int n = 0;
        for (WebElement e : driver.findElements(By.cssSelector(css))) {
            if (e.isDisplayed()) n++;
        }
        return n;
    }

    public static void main(String[] args) {
        driver = new ChromeDriver();
        try {
            driver.get("http://localhost:8080/objects.php");
            String inputButtons = "input[type=submit], input[type=button], input[type=reset]";
            String entryInputs = "input:not([type=hidden]):not([type=submit]):not([type=button]):not([type=reset])";

            System.out.println("LINKS");
            System.out.println("  anchor tags (a)          " + count("a"));
            System.out.println("  links (a with href)      " + count("a[href]"));
            System.out.println("  visible links            " + countVisible("a[href]"));
            System.out.println("IMAGES");
            System.out.println("  img elements             " + count("img"));
            int loaded = 0;
            for (WebElement img : driver.findElements(By.tagName("img"))) {
                Object width = ((JavascriptExecutor) driver).executeScript("return arguments[0].naturalWidth;", img);
                if (((Number) width).intValue() > 0) loaded++;
            }
            System.out.println("  images that loaded       " + loaded);
            System.out.println("BUTTONS");
            System.out.println("  button elements          " + count("button"));
            System.out.println("  input buttons            " + count(inputButtons));
            System.out.println("  all buttons              " + (count("button") + count(inputButtons)));
            System.out.println("INPUT ELEMENTS");
            System.out.println("  input tags               " + count("input"));
            System.out.println("  entry inputs             " + count(entryInputs));
            System.out.println("  select and textarea      " + count("select, textarea"));

            System.out.println();
            System.out.println("The links, with what each points to:");
            List<WebElement> links = driver.findElements(By.cssSelector("a[href]"));
            for (WebElement a : links) {
                String label = a.isDisplayed() ? a.getText() : "(hidden)";
                if (label.isEmpty()) label = "(image: " + a.findElement(By.tagName("img")).getDomAttribute("alt") + ")";
                System.out.println("  " + label + " -> " + a.getDomAttribute("href"));
            }
        } finally {
            driver.quit();
        }
    }
}

LINKS
  anchor tags (a)          7
  links (a with href)      6
  visible links            5
IMAGES
  img elements             4
  images that loaded       4
BUTTONS
  button elements          2
  input buttons            3
  all buttons              5
INPUT ELEMENTS
  input tags               10
  entry inputs             6
  select and textarea      2

The links, with what each points to:
  Home -> index.php
  About -> about.php
  Login -> login.php
  Email the office -> mailto:office@example.com
  (image: Portal logo) -> about.php
  (hidden) -> results.php

Practical 9: Identifying and Counting the Objects on a Page

Every count is the one worked out from the source. Three details of the program:

findElements is used for every count, never findElement: a category with nothing in it should count 0, not throw an exception.
isDisplayed() separates the hidden Results link from the five a user can see. getText() of a hidden element is empty, so the list prints "(hidden)" for it instead.
Images that loaded asks the browser for each picture's naturalWidth, its width in the image file itself, through JavascriptExecutor. A picture that failed to load has a natural width of 0. This is how a test finds broken images, which a count of img tags cannot: a broken image is still an img.

Why the selectors, and not the tag names alone

By.tagName("input") returns ten elements, and a program that reports "10 input elements" is right about the tags and wrong about the form, which has six fields a user fills in. By.tagName("button") returns two, and misses the three input buttons a user would also call buttons. The categorised counts MU asks for are only meaningful when the definition is stated, which is why the output labels each count with its definition and the journal should too.

Observations

Category	Expected	Program
Anchor tags	7	7
Links	6	6
Visible links	5	5
Images	4	4
All buttons	5	5
Input tags	10	10
Entry inputs	6	6

Result

The objects on the page were identified and counted by category: 6 links (5 visible, plus 1 anchor with no address), 4 images, all loaded, 5 buttons (2 button elements and 3 input buttons), and 10 input tags of which 6 are entry fields, with 2 more form fields. Every count agreed with the count made by hand from the page's source.

Where marks are lost

One number per category with no definition. "Buttons: 2" is wrong for a user and right for a tag count; say which.

By.tagName("a") for links. It includes the anchor with no address.

Counting hidden elements as visible. Use isDisplayed() when the question is what a user sees.

findElement to count. It throws when there are none.

Forgetting input buttons, or counting them twice, once as buttons and once as inputs, without saying so.

For the journal

Aim; the page under test; the table of definitions and selectors; the expected counts; the program; its output; the observations table; the result.

Practical 9: Identifying and Counting the Objects on a Page

Quick revision

Counting objects is really stating definitions.
a counts anchors; a[href] counts links; add isDisplayed() for visible links.
Buttons: button tags plus input tags whose type is submit, button or reset.
Inputs: input counts every tag; exclude hidden and button types for entry fields.
findElements(...).size() counts, and returns 0 when there are none.
A broken image is still an img; check naturalWidth to know it loaded.

Questions you must be able to answer

1. Why do a and a[href] give different counts here? One a element has no href, so it is an anchor but not a link.

2. How does the program know the Results link is hidden? isDisplayed() returns false for it, because the page's style sheet hides its class.

3. How many buttons are on the page? Five by what a user sees: two button elements and three inputs of type submit, button and reset. Two if only button tags are counted.

4. Why is By.tagName("input") not the number of form fields? It also counts the hidden input and the three input buttons, which a user does not fill in.

5. How can a test tell that an image failed to load? Its naturalWidth, read through JavascriptExecutor, is 0.

6. Why use findElements for counting? It returns an empty list, size 0, when nothing matches; findElement would throw.

Contents This chapter on its own page

Chapter Thirteen

Practical 10: List and Combo Box Verification

Syllabus topic Module 1, "List and Combo Box Item Verification", "Tool: Selenium WebDriver", "Automate the process of identifying a list or dropdown box on a web page and determine the total number of items. Validate default selected value and perform selection testing."

Aim

To identify a dropdown and a list box on a web page, to determine how many items each holds, to validate the default selected value, and to test every selection.

What you need to know before you start

A combo box or dropdown shows one choice and opens to show the rest. A list box shows several items at once and may allow more than one to be chosen. In HTML both are the same element, <select>, holding <option> elements; the list box simply has a size attribute, and a multiple attribute when more than one may be chosen.

Selenium's Select class wraps a select element and gives it the methods a test needs:

getOptions(): every option, as a list
getFirstSelectedOption(): the selected option (the first, in a multiple list)
getAllSelectedOptions(): every selected option
isMultiple(): whether more than one may be chosen
selectByVisibleText(text): choose by the text the user sees
selectByValue(value): choose by the option's value attribute
selectByIndex(i): choose by position, counting from 0
deselectByVisibleText(text) and deselectAll(): unchoose, in a multiple list only

The default selected value is what the box shows before the user touches it. HTML decides it: the option carrying the selected attribute, or the first option if none does. Validating it matters because a wrong default is a real defect: a form that arrives with "Semester 1" already chosen for a Semester 5 student will be submitted wrongly by everybody who does not look.

Selection testing means choosing each option in turn and checking that the box, and anything the page does in response, follow.

The page under test

practice/lists.php has three boxes: a course dropdown whose first option is a placeholder and whose last is disabled, a semester dropdown whose default is not its first option, and a multiple-choice electives list:

<?php $title = 'Lists and Combo Boxes'; include 'inc/top.php'; ?>
<h1>Lists and Combo Boxes</h1>

<label>Course
  <select id="course">
    <option value="" selected>-- Select a course --</option>
    <option value="BSCCS">B.Sc. Computer Science</option>
    <option value="BSCIT">B.Sc. Information Technology</option>
    <option value="BCOM">B.Com.</option>
    <option value="BA">B.A.</option>
    <option value="BMS">B.M.S.</option>
    <option value="BSCDS" disabled>B.Sc. Data Science (closed)</option>
  </select>
</label>
<p id="courseResult"></p>

<label>Semester
  <select id="semester">
    <option>Semester 1</option>
    <option>Semester 2</option>
    <option>Semester 3</option>
    <option>Semester 4</option>
    <option selected>Semester 5</option>
    <option>Semester 6</option>
  </select>
</label>

<label>Electives (hold Ctrl to choose more than one)
  <select id="electives" multiple size="5">
    <option selected>Software Testing</option>
    <option>Wireless Sensor Networks</option>
    <option>Ethical Hacking</option>
    <option>Cloud Computing</option>
    <option>Data Mining</option>
  </select>
</label>

<script>
document.getElementById('course').addEventListener('change', function () {
  const chosen = this.options[this.selectedIndex].text;
  document.getElementById('courseResult').textContent = this.value === '' ? '' : 'You chose ' + chosen;
});
</script>
<?php include 'inc/bottom.php'; ?>

The program

Create practicals/ListsTest.java:

package practicals;

import java.util.ArrayList;
import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.Select;

public class ListsTest {
    static List<String> texts(List<WebElement> options) {
        List<String> out = new ArrayList<>();
        for (WebElement o : options) out.add(o.getText());
        return out;
    }

    public static void main(String[] args) {
        WebDriver driver = new ChromeDriver();
        try {
            driver.get("http://localhost:8080/lists.php");

            // 1. Identify the boxes on the page.
            List<WebElement> boxes = driver.findElements(By.tagName("select"));
            System.out.println("Select boxes on the page: " + boxes.size());
            for (WebElement b : boxes) {
                Select s = new Select(b);
                System.out.println("  " + b.getDomAttribute("id") + ": " + s.getOptions().size()
                        + " items, multiple=" + s.isMultiple());
            }

            // 2. The course dropdown: items, default, and every selection.
            Select course = new Select(driver.findElement(By.id("course")));
            System.out.println("Course items: " + texts(course.getOptions()));
            System.out.println("Course default: " + course.getFirstSelectedOption().getText());
            int ok = 0, tried = 0;
            for (WebElement option : course.getOptions()) {
                if (option.getDomAttribute("value").isEmpty() || !option.isEnabled()) continue;
                tried++;
                course.selectByVisibleText(option.getText());
                String chosen = course.getFirstSelectedOption().getText();
                String shown = driver.findElement(By.id("courseResult")).getText();
                boolean pass = chosen.equals(option.getText()) && shown.equals("You chose " + option.getText());
                if (pass) ok++;
                System.out.println("  select " + option.getText() + " -> page says \"" + shown + "\" " + (pass ? "PASS" : "FAIL"));
            }
            System.out.println("Selection tests passed: " + ok + " of " + tried);

            // 3. The disabled option cannot be selected.
            try {
                course.selectByVisibleText("B.Sc. Data Science (closed)");
            } catch (UnsupportedOperationException e) {
                System.out.println("Disabled option: " + e.getMessage());
            }

            // 4. The semester dropdown's default is not its first item.
            Select semester = new Select(driver.findElement(By.id("semester")));
            System.out.println("Semester default: " + semester.getFirstSelectedOption().getText()
                    + " (first item is " + semester.getOptions().get(0).getText() + ")");
            semester.selectByIndex(0);
            System.out.println("After selectByIndex(0): " + semester.getFirstSelectedOption().getText());
            try {
                semester.deselectAll();
            } catch (UnsupportedOperationException e) {
                System.out.println("deselectAll on a single-choice box: " + e.getMessage());
            }

            // 5. The electives list allows several choices.
            Select electives = new Select(driver.findElement(By.id("electives")));
            System.out.println("Electives default: " + texts(electives.getAllSelectedOptions()));
            electives.selectByVisibleText("Cloud Computing");
            try {
                electives.selectByValue("Data Mining");       // these options have no value attribute
            } catch (org.openqa.selenium.NoSuchElementException e) {
                System.out.println("selectByValue: " + e.getMessage().lines().findFirst().orElse(""));
            }
            electives.selectByVisibleText("Data Mining");
            System.out.println("After two more: " + texts(electives.getAllSelectedOptions()));
            electives.deselectByVisibleText("Software Testing");
            System.out.println("After deselecting one: " + texts(electives.getAllSelectedOptions()));
            electives.deselectAll();
            System.out.println("After deselectAll: " + electives.getAllSelectedOptions().size() + " selected");
        } finally {
            driver.quit();
        }
    }
}

Practical 10: List and Combo Box Verification

Select boxes on the page: 3
  course: 7 items, multiple=false
  semester: 6 items, multiple=false
  electives: 5 items, multiple=true
Course items: [-- Select a course --, B.Sc. Computer Science, B.Sc. Information Technology, B.Com., B.A., B.M.S., B.Sc. Data Science (closed)]
Course default: -- Select a course --
  select B.Sc. Computer Science -> page says "You chose B.Sc. Computer Science" PASS
  select B.Sc. Information Technology -> page says "You chose B.Sc. Information Technology" PASS
  select B.Com. -> page says "You chose B.Com." PASS
  select B.A. -> page says "You chose B.A." PASS
  select B.M.S. -> page says "You chose B.M.S." PASS
Selection tests passed: 5 of 5
Disabled option: You may not select a disabled option
Semester default: Semester 5 (first item is Semester 1)
After selectByIndex(0): Semester 1
deselectAll on a single-choice box: You may only deselect all options of a multi-select
Electives default: [Software Testing]
selectByValue: Cannot locate option with value: Data Mining
After two more: [Software Testing, Cloud Computing, Data Mining]
After deselecting one: [Cloud Computing, Data Mining]
After deselectAll: 0 selected

Practical 10: List and Combo Box Verification

Reading the output against the page:

Identification. Three select elements: course with 7 items, semester with 6, and electives with 5, the only one where multiple is true.
Total items and the default. The course box holds seven options and starts on its placeholder, "-- Select a course --", because that option carries selected. That is the correct default for a form that must not guess.
Selection testing. The loop skipped the placeholder (its value is empty) and the disabled option, selected each of the other five by its visible text, and checked two things each time: that the box shows it, and that the page's own response line says "You chose" followed by the same course. Five of five passed.
The disabled option. Selecting it raised UnsupportedOperationException with Selenium's message, which is the right behaviour: a user cannot choose it either.
A default that is not first. The semester box starts on Semester 5, its fifth item, because of selected. selectByIndex(0) chose Semester 1: indexes start at 0.
deselectAll on a single-choice box is refused with its own message, since such a box must always show a choice.
selectByValue needs a value attribute. The electives' options are written with no value attribute, and selectByValue("Data Mining") failed with "Cannot locate option with value: Data Mining". Selenium looks for an option whose value attribute is that text, and there is none. (A browser submitting the form would use the option's text as its value, but that is the element's value property, not an attribute written in the page.) The program then chose it by visible text instead.
The multiple list. It started with one choice, took two more, let one be removed, and could be emptied.

getDomAttribute("value"), asked of each option, reads the value attribute exactly as written in the HTML: codes such as BSCCS for the courses, an empty string for the placeholder, which is how the loop recognises it.

Observations

Box	Items	Multiple	Default	Selection tests
course	7	no	-- Select a course --	5 of 5 passed; disabled option refused
semester	6	no	Semester 5	index 0 chose Semester 1
electives	5	yes	Software Testing	added 2, removed 1, cleared all

Result

Three select boxes were identified on the page. The course dropdown holds 7 items with the placeholder as its default; each of its five selectable courses was selected and verified against the page's response, and its disabled option was refused. The semester dropdown's default was validated as Semester 5. The electives list was verified as a multiple-choice list, with selections added, removed and cleared.

Practical 10: List and Combo Box Verification

Where marks are lost

Counting the placeholder as a course. It is an item of the box, so it counts in the total, but it is not a real selection; say both.

Assuming the default is the first item. Check getFirstSelectedOption(); the semester box proves the difference.

selectByIndex(1) expected to be the first option. Index 0 is the first.

Testing only one selection. MU says "perform selection testing": every selectable item.

Not checking what the page does after a selection. The box can change while the page's response is wrong.

Calling deselectAll() on a single-choice box. Selenium refuses it.

For the journal

Aim; the page under test; the table of Select methods; the program; its output; the observations table; the result.

Quick revision

Dropdown and list box are both <select>; multiple allows several choices.
new Select(element), then getOptions().size() for the total.
getFirstSelectedOption() gives the default before anything is chosen.
selectByVisibleText, selectByValue, selectByIndex (from 0).
A disabled option cannot be selected: UnsupportedOperationException.
getAllSelectedOptions(), deselectByVisibleText, deselectAll for a multiple list.
selectByValue matches the value attribute; an option with none cannot be chosen that way.

Questions you must be able to answer

1. How do you find the total number of items in a dropdown? Wrap it in Select and take getOptions().size().

2. How do you validate a dropdown's default value? Read getFirstSelectedOption() before selecting anything and compare its text with the expected default.

3. Why was the semester box's default not its first item? Its fifth option carries the selected attribute, which sets the default.

4. What happens when a test selects a disabled option? Select raises UnsupportedOperationException, as a user could not choose it either.

5. How can you tell a list box allows several choices? isMultiple() returns true, because the select has the multiple attribute.

6. Why did selectByValue("Data Mining") fail? selectByValue looks for an option whose value attribute is that text, and the electives' options have no value attribute. Choose them by visible text.

Contents This chapter on its own page

Module II

Checkboxes, sorting and filtering, uploads and downloads with the Robot class, Log4j and screenshots, the Page Object Model, TestNG, Jenkins, cross-browser Grid runs, JMeter load tests and Bugzilla

Chapter Fourteen

Practical 11: Checkbox Identification and Validation

Syllabus topic Module 2, "Checkbox Identification and Validation", "Tool: Selenium WebDriver", "Write a program to count the total number of checkboxes on a web page, including checked and unchecked counts. Perform dynamic selection and validation."

Aim

To count the checkboxes on a web page, in total and as checked and unchecked, to perform dynamic selection, and to validate the result.

What you need to know before you start

A checkbox is <input type="checkbox">, and three of its properties matter to a test:

Property	Selenium	Meaning
checked	`isSelected()`	ticked now
enabled	`isEnabled()`	a user may change it; a `disabled` checkbox cannot be changed
displayed	`isDisplayed()`	a user can see it

click() toggles. Clicking a ticked box unticks it. A script that wants a box ticked must check first and click only if it is not already selected; clicking blindly works on a fresh page and breaks on any page where the box started ticked.

Dynamic selection means choosing the boxes at run time from data, "tick the facilities named Canteen, Hostel and Medical Room", rather than hard-coding which element to click. The natural locator is the label text, because that is what the data names: find the label whose text is "Canteen", and the checkbox inside it.

Dynamic counts. Some pages add or remove checkboxes while you use them. A count taken once, at the start, goes stale. Count again after anything that can change the page.

The page under test

practice/checkboxes.php lists ten facilities. Three start ticked, one is disabled, a button adds two more boxes, another clears every box that can be cleared, and a line at the bottom counts the ticked boxes:

<?php $title = 'Checkboxes'; include 'inc/top.php'; ?>
<h1>Campus Facilities</h1>
<p>Tick every facility you use.</p>
<div id="facilities">
  <label><input type="checkbox" value="Library" checked> Library</label>
  <label><input type="checkbox" value="Laboratory" checked> Laboratory</label>
  <label><input type="checkbox" value="Canteen"> Canteen</label>
  <label><input type="checkbox" value="Gymkhana"> Gymkhana</label>
  <label><input type="checkbox" value="Hostel"> Hostel</label>
  <label><input type="checkbox" value="Wi-Fi" checked> Wi-Fi</label>
  <label><input type="checkbox" value="Parking" disabled> Parking (full)</label>
  <label><input type="checkbox" value="Sports Ground"> Sports Ground</label>
  <label><input type="checkbox" value="Auditorium"> Auditorium</label>
  <label><input type="checkbox" value="Bus Service"> Bus Service</label>
</div>
<p>
  <button id="more">Show more facilities</button>
  <button id="clearAll">Clear all</button>
</p>
<p id="summary"></p>
<script>
const box = document.getElementById('facilities');
function summarise() {
  const ticked = box.querySelectorAll('input[type=checkbox]:checked').length;
  document.getElementById('summary').textContent = ticked + ' facilities selected';
}
box.addEventListener('change', summarise);
// Two more checkboxes are added to the page only when this button is pressed.
document.getElementById('more').onclick = function () {
  ['Medical Room', 'Placement Cell'].forEach(function (name) {
    const label = document.createElement('label');
    label.innerHTML = '<input type="checkbox" value="' + name + '"> ' + name;
    box.appendChild(label);
  });
  this.disabled = true;
};
document.getElementById('clearAll').onclick = function () {
  box.querySelectorAll('input[type=checkbox]:not(:disabled)').forEach(function (c) { c.checked = false; });
  summarise();
};
summarise();
</script>
<?php include 'inc/bottom.php'; ?>

Expected counts from the source, before running anything: 10 checkboxes; 3 checked (Library, Laboratory, Wi-Fi); 7 unchecked, of which 1 (Parking) is disabled. After "Show more facilities": 12 in all, still 3 checked, 9 unchecked.

The program

Create practicals/CheckboxTest.java:

package practicals;

import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;

public class CheckboxTest {
    static WebDriver driver;

    static void counts(String when) {
        List<WebElement> boxes = driver.findElements(By.cssSelector("input[type=checkbox]"));
        int checked = 0, disabled = 0;
        for (WebElement box : boxes) {
            if (box.isSelected()) checked++;
            if (!box.isEnabled()) disabled++;
        }
        System.out.println(when + ": total " + boxes.size() + ", checked " + checked
                + ", unchecked " + (boxes.size() - checked) + ", disabled " + disabled);
        System.out.println("  page says: " + driver.findElement(By.id("summary")).getText());
    }

    // The checkbox inside the label whose text is the facility's name.
    static WebElement box(String facility) {
        return driver.findElement(By.xpath("//label[normalize-space()='" + facility + "']/input"));
    }

    // Tick a box only if it is not already ticked: click() toggles.
    static void tick(String facility) {
        WebElement b = box(facility);
        if (!b.isSelected()) b.click();
    }

    public static void main(String[] args) {
        driver = new ChromeDriver();
        try {
            driver.get("http://localhost:8080/checkboxes.php");
            counts("At the start");

            System.out.println("Checked at the start:");
            for (WebElement b : driver.findElements(By.cssSelector("input[type=checkbox]:checked"))) {
                System.out.println("  " + b.getDomAttribute("value"));
            }

            driver.findElement(By.id("more")).click();
            counts("After Show more");

            // Dynamic selection: the names come from data, not from the code's locators.
            String[] wanted = {"Canteen", "Hostel", "Medical Room", "Library"};
            for (String name : wanted) {
                tick(name);
            }
            boolean allTicked = true;
            for (String name : wanted) {
                allTicked &= box(name).isSelected();
            }
            System.out.println("Wanted facilities all ticked: " + allTicked);
            counts("After selection");

            WebElement parking = box("Parking (full)");
            parking.click();
            System.out.println("Parking enabled: " + parking.isEnabled() + ", selected after a click: " + parking.isSelected());

            driver.findElement(By.id("clearAll")).click();
            counts("After Clear all");
        } finally {
            driver.quit();
        }
    }
}

Practical 11: Checkbox Identification and Validation

At the start: total 10, checked 3, unchecked 7, disabled 1
  page says: 3 facilities selected
Checked at the start:
  Library
  Laboratory
  Wi-Fi
After Show more: total 12, checked 3, unchecked 9, disabled 1
  page says: 3 facilities selected
Wanted facilities all ticked: true
After selection: total 12, checked 6, unchecked 6, disabled 1
  page says: 6 facilities selected
Parking enabled: false, selected after a click: false
After Clear all: total 12, checked 0, unchecked 12, disabled 1
  page says: 0 facilities selected

What the run proves:

The counts match the source: 10, 3 checked, 7 unchecked, 1 disabled at the start, and 12 after the button added two. The page's own summary agrees each time, which is the validation: the script's count and the application's count are two independent measurements of the same thing.
Dynamic selection. Four facilities were named in data. Library was already ticked, so tick() left it alone; the other three were ticked, and every one of the four was then confirmed selected. Six boxes are checked, and the page says so.
The disabled box. Clicking Parking changed nothing: isEnabled() is false and it is still unticked. A test that expects a click to tick a disabled box has found a mismatch between its expectation and the page, which is the page being right.
Clear all unticked every box that can be unticked, and the page counts none.

Practical 11: Checkbox Identification and Validation

input[type=checkbox]:checked is a CSS selector for the ticked boxes only, a second way to count them without a loop.

//label[normalize-space()='Canteen']/input reads: a label whose text, with spaces tidied, is exactly "Canteen", then the input inside it. Each box on this page sits inside its label, and that is what makes selection by name possible.

Observations

Moment	Total	Checked	Unchecked	Disabled	Page's count
At the start	10	3	7	1	3 facilities selected
After Show more	12	3	9	1	3 facilities selected
After selecting four by name	12	6	6	1	6 facilities selected
After Clear all	12	0	12	1	0 facilities selected

Result

The checkboxes on the page were counted: 10 in total, 3 checked and 7 unchecked, one of them disabled; after the page added two more, 12. Four facilities named in data were selected dynamically by their labels and validated, the disabled checkbox was confirmed unchangeable, and clearing was validated. At every step the program's count agreed with the page's own.

Where marks are lost

Clicking without checking first. click() toggles; ticking an already ticked box unticks it.

Counting once. The page added two boxes; a count taken at the start is wrong after that.

Hard-coded ids for dynamic selection. Select by the label the data names.

Counting a disabled box as selectable. Report it separately.

No validation. Compare your count with something independent: the page's own summary here.

For the journal

Aim; the page under test; the expected counts from the source; the program; its output; the observations table; the result.

Quick revision

Checkbox: input[type=checkbox]. isSelected() ticked, isEnabled() changeable, isDisplayed() visible.
click() toggles: check isSelected() before clicking to tick.
Count checked boxes with input[type=checkbox]:checked, or in a loop.
Dynamic selection by label: //label[normalize-space()='Name']/input.
Recount after anything that changes the page.
A disabled checkbox ignores clicks.

Questions you must be able to answer

1. How do you count the checked checkboxes? Find every input of type checkbox and count those whose isSelected() is true, or count the CSS selector input[type=checkbox]:checked.

2. Why does the tick() method check isSelected() first? Because click() toggles: clicking a box that is already ticked would untick it.

3. How does the program select a facility named in data? It finds the label whose text is that name and clicks the checkbox inside it.

4. What happened when the disabled Parking box was clicked? Nothing: it stayed unticked, and isEnabled() reported false.

5. Why count again after Show more facilities? The button added two checkboxes, so the earlier count of ten was no longer true.

Contents This chapter on its own page

Chapter Fifteen

Practical 12: Dynamic Web Table, Sorting and Filtering

Syllabus topic Module 2, "Dynamic Web Table Handling and Sorting Validation", "Tool: Selenium WebDriver", "Automate extraction of data from a dynamic web table. Validate sorting functionality and verify correctness of filtered results."

Aim

To extract the data from a dynamic web table, to validate the table's sorting for every column in both directions, and to verify that its filter shows exactly the rows it should.

What you need to know before you start

The oracle. A test needs a way to know the right answer, and testers call that source of truth the oracle. For sorting the oracle is simple and strong: take the values the page shows, sort them yourself with Java, and compare. If the page's order and Java's order differ, the page's sort is wrong. For filtering the oracle is the full table: filter it yourself in Java and compare with what the page shows.

Sorting has a type. Numbers and text sort differently. As numbers, 9 comes before 38, which comes before 100. As text, compared character by character, "100" comes first (its first character, 1, is smallest), then "38", and "9" comes last. A table that sorts a numeric column as text is one of the most common defects in web applications, and it is invisible on data where every number has the same number of digits.

Ties. When two rows have the same city, their order after sorting by city depends on the sort. Validate only the column that was sorted, not the order of tied rows, unless the requirement says how ties are broken.

The filter's promise. A filter must show every row that matches and no row that does not. Checking only that each shown row matches misses the rows it wrongly hid, so the check counts too.

The page under test

practice/results.php fetches fifteen rows from api/results.php a second after it loads. Clicking a heading sorts by that column, ascending, and clicking it again reverses it; the heading being sorted carries aria-sort="ascending" or "descending". Typing in the filter box shows only rows whose name or city contains the text, and the line above the table counts them:

<?php $title = 'Results'; include 'inc/top.php'; ?>
<h1>Semester Results</h1>
<label>Filter by name or city <input type="text" id="filter"></label>
<p id="count">Loading...</p>
<table id="results">
  <thead>
    <tr>
      <th data-key="roll">Roll</th>
      <th data-key="name">Name</th>
      <th data-key="city">City</th>
      <th data-key="marks">Marks</th>
    </tr>
  </thead>
  <tbody id="resultsBody"></tbody>
</table>
<script>
let rows = [];
let sortKey = null, ascending = true;

function draw() {
  const text = document.getElementById('filter').value.trim().toLowerCase();
  const shown = rows.filter(function (r) {
    return r.name.toLowerCase().includes(text) || r.city.toLowerCase().includes(text);
  });
  const body = document.getElementById('resultsBody');
  body.innerHTML = '';
  shown.forEach(function (r) {
    const tr = document.createElement('tr');
    [r.roll, r.name, r.city, r.marks].forEach(function (v) {
      const td = document.createElement('td');
      td.textContent = v;
      tr.appendChild(td);
    });
    body.appendChild(tr);
  });
  document.getElementById('count').textContent = 'Showing ' + shown.length + ' of ' + rows.length + ' students';
}

document.querySelectorAll('#results th').forEach(function (th) {
  th.addEventListener('click', function () {
    const key = th.dataset.key;
    ascending = (sortKey === key) ? !ascending : true;
    sortKey = key;
    // Every column is compared as text. Watch what that does to the numbers.
    rows.sort(function (a, b) {
      const order = String(a[key]).localeCompare(String(b[key]));
      return ascending ? order : -order;
    });
    document.querySelectorAll('#results th').forEach(function (h) { h.removeAttribute('aria-sort'); });
    th.setAttribute('aria-sort', ascending ? 'ascending' : 'descending');
    draw();
  });
});
document.getElementById('filter').addEventListener('input', draw);

setTimeout(function () {
  fetch('api/results.php')
    .then(function (response) { return response.json(); })
    .then(function (data) { rows = data; draw(); });
}, 1000);
</script>
<?php include 'inc/bottom.php'; ?>

Practical 12: Dynamic Web Table, Sorting and Filtering

<?php
// api/results.php: the rows the results table loads.
require __DIR__ . '/../inc/store.php';
header('Content-Type: application/json; charset=utf-8');
echo json_encode(load_records('results'));

[
    {"roll": 201, "name": "Neha Sawant", "city": "Mumbai", "marks": 88},
    {"roll": 202, "name": "Omkar Jadhav", "city": "Thane", "marks": 9},
    {"roll": 203, "name": "Pooja Mishra", "city": "Pune", "marks": 72},
    {"roll": 204, "name": "Qasim Ansari", "city": "Mumbai", "marks": 100},
    {"roll": 205, "name": "Rhea Fernandes", "city": "Nashik", "marks": 45},
    {"roll": 206, "name": "Siddharth Rane", "city": "Thane", "marks": 67},
    {"roll": 207, "name": "Tanvi Hegde", "city": "Mumbai", "marks": 91},
    {"roll": 208, "name": "Uday Kamat", "city": "Pune", "marks": 38},
    {"roll": 209, "name": "Vaishnavi More", "city": "Mumbai", "marks": 56},
    {"roll": 210, "name": "Wasim Shaikh", "city": "Nashik", "marks": 84},
    {"roll": 211, "name": "Yash Bhatt", "city": "Thane", "marks": 73},
    {"roll": 212, "name": "Zara Mistry", "city": "Mumbai", "marks": 5},
    {"roll": 213, "name": "Arjun Nadkarni", "city": "Pune", "marks": 99},
    {"roll": 214, "name": "Bela Sinha", "city": "Nashik", "marks": 61},
    {"roll": 215, "name": "Chirag Dalal", "city": "Mumbai", "marks": 50}
]

Expected from the seed file, before running: 15 rows; the filter "Mumbai" should keep 6 (rolls 201, 204, 207, 209, 212, 215), "Pune" 3, and "Delhi" none.

The program

Create TableSortFilter.java, in the same practicals folder as the others:

package practicals;

import java.time.Duration;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class TableSortFilter {
    static WebDriver driver;
    static WebDriverWait wait;
    static final String[] COLUMNS = {"Roll", "Name", "City", "Marks"};

    // Every row of the table as it is shown now, one String[] per row.
    static List<String[]> rows() {
        List<String[]> out = new ArrayList<>();
        for (WebElement tr : driver.findElements(By.cssSelector("#resultsBody tr"))) {
            List<WebElement> td = tr.findElements(By.tagName("td"));
            out.add(new String[]{td.get(0).getText(), td.get(1).getText(), td.get(2).getText(), td.get(3).getText()});
        }
        return out;
    }

    static List<String> column(List<String[]> rows, int c) {
        List<String> values = new ArrayList<>();
        for (String[] r : rows) values.add(r[c]);
        return values;
    }

    // The oracle: Java's own sort of the values the page shows.
    static void checkSort(int c, boolean numeric) {
        WebElement th = driver.findElement(By.cssSelector("#results th[data-key='" + COLUMNS[c].toLowerCase() + "']"));
        Comparator<String> order = numeric ? Comparator.comparingInt(Integer::parseInt) : Comparator.naturalOrder();
        for (String direction : new String[]{"ascending", "descending"}) {
            th.click();
            wait.until(ExpectedConditions.attributeToBe(th, "aria-sort", direction));
            List<String> shown = column(rows(), c);
            List<String> expected = new ArrayList<>(shown);
            expected.sort(direction.equals("ascending") ? order : order.reversed());
            boolean ok = shown.equals(expected);
            System.out.printf("%-6s %-10s %s%n", COLUMNS[c], direction, ok ? "PASS" : "FAIL");
            if (!ok) {
                System.out.println("       page:     " + shown);
                System.out.println("       expected: " + expected);
            }
        }
    }

    static void checkFilter(String text, List<String[]> all) {
        WebElement box = driver.findElement(By.id("filter"));
        box.clear();
        box.sendKeys(text);
        List<String> expected = new ArrayList<>();
        for (String[] r : all) {
            if (r[1].toLowerCase().contains(text.toLowerCase()) || r[2].toLowerCase().contains(text.toLowerCase())) {
                expected.add(r[0]);
            }
        }
        String count = "Showing " + expected.size() + " of " + all.size() + " students";
        wait.until(ExpectedConditions.textToBe(By.id("count"), count));
        List<String> shown = column(rows(), 0);
        // The same rows, in whatever order the table is sorted: compare as sets.
        List<String> a = new ArrayList<>(shown), b = new ArrayList<>(expected);
        Collections.sort(a);
        Collections.sort(b);
        boolean ok = a.equals(b);
        System.out.println("Filter \"" + text + "\": page shows rolls " + shown + "  " + (ok ? "PASS" : "FAIL"));
    }

    public static void main(String[] args) {
        driver = new ChromeDriver();
        wait = new WebDriverWait(driver, Duration.ofSeconds(10));
        try {
            driver.get("http://localhost:8080/results.php");
            wait.until(ExpectedConditions.textToBePresentInElementLocated(By.id("count"), "Showing 15 of 15"));
            List<String[]> all = rows();
            System.out.println("Rows extracted: " + all.size());
            System.out.println("First row: " + String.join(" | ", all.get(0)));

            System.out.println("Sorting");
            checkSort(0, true);
            checkSort(1, false);
            checkSort(2, false);
            checkSort(3, true);

            System.out.println("Filtering");
            checkFilter("Mumbai", all);
            checkFilter("Pune", all);
            checkFilter("Delhi", all);
            System.out.println("After Delhi the page says: " + driver.findElement(By.id("count")).getText());
        } finally {
            driver.quit();
        }
    }
}

Practical 12: Dynamic Web Table, Sorting and Filtering

Rows extracted: 15
First row: 201 | Neha Sawant | Mumbai | 88
Sorting
Roll   ascending  PASS
Roll   descending PASS
Name   ascending  PASS
Name   descending PASS
City   ascending  PASS
City   descending PASS
Marks  ascending  FAIL
       page:     [100, 38, 45, 5, 50, 56, 61, 67, 72, 73, 84, 88, 9, 91, 99]
       expected: [5, 9, 38, 45, 50, 56, 61, 67, 72, 73, 84, 88, 91, 99, 100]
Marks  descending FAIL
       page:     [99, 91, 9, 88, 84, 73, 72, 67, 61, 56, 50, 5, 45, 38, 100]
       expected: [100, 99, 91, 88, 84, 73, 72, 67, 61, 56, 50, 45, 38, 9, 5]
Filtering
Filter "Mumbai": page shows rolls [207, 201, 209, 215, 212, 204]  PASS
Filter "Pune": page shows rolls [213, 203, 208]  PASS
Filter "Delhi": page shows rolls []  PASS
After Delhi the page says: Showing 0 of 15 students

Reading the result: a defect, found by the oracle

Roll, Name and City sort correctly both ways. Marks does not. In ascending order the page put 100 first and 9 near the end, and in descending order the reverse: it compared the marks as text. The program did not know this in advance. It knew only that marks are numbers, sorted the page's own values as numbers, and found that the two orders differ. The page's source confirms it, and says so in a comment: every column is compared with localeCompare, a text comparison.

Why Roll passed although it is numeric too: every roll number has three digits, and for numbers of equal length text order and numeric order agree. The defect shows only when lengths differ, which is why the test data includes marks of 5, 9 and 100. Choosing data that can expose a defect is half of testing.

Practical 12: Dynamic Web Table, Sorting and Filtering

How the defect would be reported, in the form Practical 3 set out:

Title: Sorting by Marks orders the numbers as text (100 before 38, 9 after 88).
Steps: open results.php, wait for the table, click the Marks heading.
Expected: marks in ascending numeric order, 5, 9, 38, and so on up to 100.
Actual: 100, 38, 45, 5, 50, and so on, with 9 after 88.
Severity: Major, since a teacher reading the list by marks is misled. Priority: High.

The waits that make it reliable

After the page loads, the program waits for the count line to say "Showing 15 of 15": the rows arrive a second late.
After each click on a heading it waits for that heading's aria-sort attribute to become "ascending" or "descending", the page's own sign that it has re-drawn the table in that order.
After typing a filter it waits for the count line to show the expected count. If the filter were wrong, that wait would time out, which is itself a failure message.

attributeToBe(element, name, value) and textToBe(locator, text), both from ExpectedConditions, are the two conditions used here that earlier chapters did not need.

Filtering, verified both ways

For each filter the program computed the expected rows itself from the full table, then compared the rolls the page shows with that list. Equal lists mean the page showed every matching row and nothing else.

The comparison is of the set of rows, sorted on both sides before comparing, and the first version of this program got that wrong. It compared the lists in order, and reported "Mumbai" and "Pune" as FAIL, because the table was still sorted by Marks from the step before and the filter, correctly, kept that order. The page was right and the test was wrong: a filter's promise is about which rows are shown, not their order. An oracle must check what the requirement says, and nothing it does not. "Mumbai" kept the six rows the seed file has for Mumbai, "Pune" the three for Pune, and "Delhi", which matches nothing, emptied the table and said so: "Showing 0 of 15 students".

Observations

Check	Result
Rows extracted	15
Sort by Roll, ascending and descending	passed
Sort by Name, ascending and descending	passed
Sort by City, ascending and descending	passed
Sort by Marks, ascending and descending	failed: numbers sorted as text
Filter Mumbai	6 rows, correct
Filter Pune	3 rows, correct
Filter Delhi	0 rows, correct

Practical 12: Dynamic Web Table, Sorting and Filtering

Result

Fifteen rows were extracted from the dynamic table. Sorting was validated for all four columns in both directions against Java's own sort of the same values: Roll, Name and City sort correctly, and Marks sorts numbers as text, a defect. Filtering was verified for three filters against the expected rows computed from the full table, and all three were correct.

Where marks are lost

No oracle. "The table changed after clicking" is not a sorting test.

Test data that cannot fail. Marks all of two digits would hide the defect.

Reading the table right after the click. Wait for the page's sign that it has re-sorted.

Checking only that shown rows match the filter. Also check that no matching row is missing: compare with the expected list.

Reporting "sorting works" after checking one direction. Both directions, every column.

For the journal

Aim; the page under test; the expected results worked from the seed file; the program; its output; the defect report for the Marks sort; the observations table; the result.

Quick revision

Oracle: Java's own sort of the page's values; the full table filtered in Java.
Numbers must be compared as numbers, with Comparator.comparingInt and Integer::parseInt.
Text order and number order agree only for numbers of equal length.
Wait for aria-sort after a sort click, and for the count after a filter.
Filter check: shown rows equal expected rows, both what is there and what is not.
Here: Marks sorts as text, a defect.

Questions you must be able to answer

1. What is a test oracle, and what is it here? The source of the right answer. For sorting, Java's own sort of the values the page shows; for filtering, the full table filtered in Java.

2. Why did the Roll column pass although roll numbers are numbers? Every roll number has three digits, and for numbers of equal length text order and numeric order agree.

3. In what order does a text sort put 5, 9, 38 and 100? 100, 38, 5, 9, because text is compared character by character from the left.

4. How does the program know the table has been re-sorted? It waits for the clicked heading's aria-sort attribute to become ascending or descending.

5. Why is checking that every shown row contains "Mumbai" not enough? A filter that wrongly hid some Mumbai rows would still pass. Comparing with the complete expected list catches missing rows too.

Contents This chapter on its own page

Chapter Sixteen

Practical 13: File Upload and Download with the Robot Class

Syllabus topic Module 2, "File Upload and Download Automation", "Tool: Selenium WebDriver + Robot Class", "Develop a script to automate file upload and download functionality. Validate successful upload and verify file existence after download."

Aim

To automate a file upload and a file download with Selenium WebDriver and the Robot class, to validate that the upload succeeded, and to verify that the downloaded file exists and holds what it should.

What you need to know before you start

A file box is <input type="file">. The browser draws it as a button, "Choose File" in Chrome and "Browse..." in Firefox, beside the name of the chosen file. Clicking the button opens the operating system's own file dialog: a window that belongs to Windows, or on Linux to GTK, and not to the web page. WebDriver drives the page. It cannot see that dialog, and Selenium's own documentation opens its File Upload page by saying so: "Selenium cannot interact with the file upload dialog".

So a script uploads a file in one of two ways:

sendKeys. Send the file's full path to the file box. WebDriver puts the file into the box itself, and no dialog ever opens. This is the way Selenium documents.
The Robot class. Let the page open the dialog, then answer the dialog as a person would, with real key presses. java.awt.Robot is part of Java, not of Selenium. Java's documentation describes it as generating native system input events: key presses and mouse movements made at the level of the operating system, so they reach whichever window has the keyboard, including a dialog WebDriver cannot see.

Three rules of the WebDriver standard decide what works. The W3C WebDriver standard is the rule book that ChromeDriver and geckodriver both follow:

click() on a file box is refused. The standard tells a driver to return an "invalid argument" error when the element to be clicked is a file box. A click would open the dialog, and the dialog is out of WebDriver's reach.
sendKeys() checks that the file exists. Every path is checked before anything is set, and a missing file is an "invalid argument" error too. The path must be the whole path, from the top of the disk.
A file box need not be visible. For any other element, sendKeys first waits until a user could type into it, and fails with "element not interactable" if no one could. For a file box it skips that wait, unless the session asks for "strict file interactability", which starts switched off. That is why sendKeys works even on a hidden file box, the kind a styled button hides.

A download happens when the server marks a file as an attachment, which is what download.php does with its Content-Disposition: attachment header. The browser then saves the file instead of showing it: Chrome normally saves in the user's Downloads folder, and it can be set to ask where. A test cannot answer that question, because it is another operating system dialog, so the test sets two of Chrome's preferences before Chrome starts: download.default_directory, the folder to save in, and download.prompt_for_download, set to false so that Chrome never asks.

Practical 13: File Upload and Download with the Robot Class

click() on a link does not wait for the download. The file arrives a moment later, so a test waits for it, the way Practical 6 waited for elements, and only then checks it.

The pages under test

practice/upload.php offers the same upload two ways: an ordinary file box with an Upload button, and a styled button that hides its file box and sends the form as soon as a file is chosen. The page accepts only .txt, .csv and .pdf files up to 1 MB, saves each file in the uploads folder, and reports the name and size it received:

<?php
$message = '';
$error = '';
if ($_SERVER['REQUEST_METHOD'] === 'POST') {
    $file = $_FILES['assignment'] ?? null;
    if ($file === null || $file['error'] === UPLOAD_ERR_NO_FILE) {
        $error = 'Please choose a file first.';
    } elseif ($file['error'] !== UPLOAD_ERR_OK || $file['size'] > 1048576) {
        $error = 'The file could not be uploaded. The limit is 1 MB.';
    } else {
        $name = preg_replace('/[^A-Za-z0-9._-]/', '_', basename($file['name']));
        $ext = strtolower(pathinfo($name, PATHINFO_EXTENSION));
        if (!in_array($ext, ['txt', 'csv', 'pdf'], true)) {
            $error = 'Only .txt, .csv and .pdf files are accepted.';
        } else {
            move_uploaded_file($file['tmp_name'], __DIR__ . '/uploads/' . $name);
            $message = 'Uploaded ' . $name . ' (' . $file['size'] . ' bytes)';
        }
    }
}
$title = 'Upload';
include 'inc/top.php';
?>
<h1>Upload Your Assignment</h1>
<?php if ($message !== ''): ?><p id="uploadResult" class="success"><?= htmlspecialchars($message) ?></p><?php endif; ?>
<?php if ($error !== ''): ?><p id="uploadError" class="error"><?= htmlspecialchars($error) ?></p><?php endif; ?>

<h2>Method 1: an ordinary file box</h2>
<form method="post" enctype="multipart/form-data">
  <input type="file" name="assignment" id="fileInput">
  <button type="submit" id="uploadBtn">Upload</button>
</form>

<h2>Method 2: a styled button that hides the file box</h2>
<form method="post" enctype="multipart/form-data" id="styledForm">
  <input type="file" name="assignment" id="hiddenInput" style="display:none"
         onchange="document.getElementById('styledForm').submit()">
  <button type="button" id="chooseBtn" onclick="document.getElementById('hiddenInput').click()">Choose a file to upload</button>
</form>
<?php include 'inc/bottom.php'; ?>

practice/downloads.php links to two files, and practice/download.php sends each one as an attachment:

<?php $title = 'Downloads'; include 'inc/top.php'; ?>
<h1>Downloads</h1>
<ul>
  <li><a id="timetable" href="download.php?file=timetable.csv">Practical timetable (CSV)</a></li>
  <li><a id="syllabus" href="download.php?file=syllabus.txt">Practical list (text)</a></li>
</ul>
<?php include 'inc/bottom.php'; ?>

<?php
// download.php: sends a file from files/ as an attachment, so the browser saves
// it instead of showing it. Only the names listed here can be asked for.
$allowed = ['timetable.csv' => 'text/csv', 'syllabus.txt' => 'text/plain'];
$file = $_GET['file'] ?? '';
if (!isset($allowed[$file])) {
    http_response_code(404);
    exit('No such file');
}
header('Content-Type: ' . $allowed[$file]);
header('Content-Disposition: attachment; filename="' . $file . '"');
header('Content-Length: ' . filesize(__DIR__ . '/files/' . $file));
readfile(__DIR__ . '/files/' . $file);

Practical 13: File Upload and Download with the Robot Class

Program 1: upload with sendKeys

package practicals;

import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Duration;

import org.openqa.selenium.By;
import org.openqa.selenium.InvalidArgumentException;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class UploadTest {
    static WebDriver driver;

    // Wait for a message on the page that loads after the upload, and return its text.
    static String message(String id) {
        return new WebDriverWait(driver, Duration.ofSeconds(5))
                .until(ExpectedConditions.visibilityOfElementLocated(By.id(id))).getText();
    }

    static String firstLine(Exception e) {
        return e.getMessage().lines().findFirst().orElse("");
    }

    public static void main(String[] args) throws Exception {
        // The program makes the file it uploads, so it knows exactly what the server should get.
        Path file = Path.of(System.getProperty("user.home"), "assignment-1.txt");
        Files.writeString(file, "Roll 101, Practical 13 assignment\n");
        long size = Files.size(file);
        System.out.println("File to upload: " + file + " (" + size + " bytes)");

        driver = new ChromeDriver();
        try {
            driver.get("http://localhost:8080/upload.php");
            WebElement fileBox = driver.findElement(By.id("fileInput"));

            try {
                fileBox.click();
            } catch (InvalidArgumentException e) {
                System.out.println("click() on the file box: " + firstLine(e));
            }
            try {
                fileBox.sendKeys("/home/student/no-such-file.txt");
            } catch (InvalidArgumentException e) {
                System.out.println("sendKeys() of a file that does not exist: " + firstLine(e));
            }

            // Method 1: send the file's full path to the file box, then press Upload.
            fileBox.sendKeys(file.toString());
            driver.findElement(By.id("uploadBtn")).click();
            String result = message("uploadResult");
            System.out.println("Method 1: " + result);
            System.out.println("  message as expected: " + result.equals("Uploaded assignment-1.txt (" + size + " bytes)"));

            // Validate on the server too: open its copy and compare it with the file sent.
            driver.get("http://localhost:8080/uploads/assignment-1.txt");
            String copy = driver.findElement(By.tagName("body")).getText();
            System.out.println("  server's copy matches the file sent: " + copy.equals(Files.readString(file).strip()));

            // Method 2: the file box is hidden behind a styled button.
            driver.get("http://localhost:8080/upload.php");
            WebElement hidden = driver.findElement(By.id("hiddenInput"));
            System.out.println("Method 2: the hidden file box is displayed: " + hidden.isDisplayed());
            hidden.sendKeys(file.toString());   // the page sends the form itself once a file is chosen
            System.out.println("  " + message("uploadResult"));

            // A negative test: the page accepts only .txt, .csv and .pdf files.
            Path photo = Path.of(System.getProperty("user.home"), "photo.jpg");
            Files.writeString(photo, "not really a photo\n");
            driver.findElement(By.id("fileInput")).sendKeys(photo.toString());
            driver.findElement(By.id("uploadBtn")).click();
            System.out.println("Wrong file type: " + message("uploadError"));
        } finally {
            driver.quit();
        }
    }
}

File to upload: /home/student/assignment-1.txt (34 bytes)
click() on the file box: invalid argument
sendKeys() of a file that does not exist: invalid argument: File not found : /home/student/no-such-file.txt
Method 1: Uploaded assignment-1.txt (34 bytes)
  message as expected: true
  server's copy matches the file sent: true
Method 2: the hidden file box is displayed: false
  Uploaded assignment-1.txt (34 bytes)
Wrong file type: Only .txt, .csv and .pdf files are accepted.

What each line of the output proves:

click() on the file box was refused with "invalid argument", the error the standard names. Firefox refuses it too; its driver's message is "Cannot click <input type=file> elements".
A file that does not exist was refused before anything was set: ChromeDriver checked the path and reported "File not found".
Method 1. The page reported the file's name and its size, 34 bytes, which is the size Java measured before the upload. The program then opened the server's copy in uploads/ and compared its text with the file it sent. The message shows that the page believes it received the file; the copy shows that the file really arrived, unchanged. That is what validating an upload means.
Method 2. The hidden file box is not displayed, and isDisplayed() says so, yet sendKeys worked, by the third rule. Choosing the file made the page send its form, as its onchange attribute tells it to.
The wrong file type was rejected by the page's own check, in the page's own words. This is a negative test, and it passes, because rejecting the file is correct.

Practical 13: File Upload and Download with the Robot Class

Every message is read through a WebDriverWait: pressing Upload loads a new page, and reading before it has loaded is the race Practical 7 met.

Program 2: upload through the file dialog with the Robot class

Program 1 never saw a dialog. Program 2 opens one on purpose, with the page's styled button, and answers it with the Robot class. This is the dialog in the lab, just after the Robot pasted the path:

Firefox's file dialog, with the path typed into its location box

Figure 16.1 The operating system's file dialog, opened by the page's styled button in the lab's Firefox. The path in the box at the top was pasted by the Robot. None of this window is part of the page, so WebDriver cannot find anything in it.

package practicals;

import java.awt.Robot;
import java.awt.Toolkit;
import java.awt.datatransfer.StringSelection;
import java.awt.event.KeyEvent;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Duration;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class RobotUpload {
    // Hold Ctrl, press and release a key, release Ctrl: what a person's fingers do.
    static void ctrl(Robot robot, int key) {
        robot.keyPress(KeyEvent.VK_CONTROL);
        robot.keyPress(key);
        robot.keyRelease(key);
        robot.keyRelease(KeyEvent.VK_CONTROL);
    }

    public static void main(String[] args) throws Exception {
        Path file = Path.of(System.getProperty("user.home"), "assignment-2.txt");
        Files.writeString(file, "Roll 102, uploaded through the file dialog\n");

        WebDriver driver = new FirefoxDriver();
        try {
            driver.get("http://localhost:8080/upload.php");
            // An ordinary button, so click() is allowed. Its script opens the
            // operating system's file dialog, which WebDriver cannot see.
            driver.findElement(By.id("chooseBtn")).click();

            Robot robot = new Robot();
            robot.delay(3000);                          // time for the dialog to open
            // Put the path on the clipboard, so that one key press can paste it.
            Toolkit.getDefaultToolkit().getSystemClipboard()
                   .setContents(new StringSelection(file.toString()), null);
            ctrl(robot, KeyEvent.VK_L);                 // this dialog's key for "type a location"
            robot.delay(500);
            ctrl(robot, KeyEvent.VK_V);                 // paste the path
            robot.delay(500);
            robot.keyPress(KeyEvent.VK_ENTER);          // choose the file
            robot.keyRelease(KeyEvent.VK_ENTER);

            // Choosing a file makes the page send the form; wait for its answer.
            String result = new WebDriverWait(driver, Duration.ofSeconds(10))
                    .until(ExpectedConditions.visibilityOfElementLocated(By.id("uploadResult"))).getText();
            System.out.println(result);
            System.out.println("Upload validated: " + result.equals("Uploaded assignment-2.txt (" + Files.size(file) + " bytes)"));
        } finally {
            driver.quit();
        }
    }
}

Practical 13: File Upload and Download with the Robot Class

Uploaded assignment-2.txt (43 bytes)
Upload validated: true

How the program answers a dialog it cannot see:

click() on the styled button is allowed, because the button is not a file box. Its script clicks the hidden file box, and the operating system opens the dialog.
robot.delay(3000) waits three seconds. The program cannot see the dialog either, so it cannot wait for it; it can only wait long enough. Too short a wait, and the key presses arrive before the dialog does.
The clipboard. Typing a path with Robot takes one keyPress for every character, with Shift held for capitals and for most symbols. Putting the path on the clipboard lets Ctrl+V type all of it at once.
Ctrl+L is this dialog's key for opening the box at the top where a location can be typed. Ctrl+V pastes the path into it, and Enter chooses the file, as it would for a person.
The rest is WebDriver again. Choosing the file made the page send its form, and the program waits for the page's answer and validates it the way Program 1 did.

The keys depend on the dialog. The dialog in the lab is the one Linux's GTK draws, and Ctrl+L is its key. Windows draws a different dialog. Before writing a Robot program, do the job once by hand and write down every key you press: those are the keys the Robot must press, in that order.

Why Firefox. In the lab, Chromium opened this dialog and took the path, then never passed the chosen file to the page, even when Chromium was started by hand without Selenium and the file was double-clicked. Firefox, in the same lab, did pass it. So the program names FirefoxDriver; nothing else in it depends on the browser.

What the Robot class costs:

It needs a real screen. Java's documentation says creating a Robot always throws AWTException when there is no display.
Its keys go to whichever window has the keyboard. Touch the mouse or the keyboard while it runs, and the keys land somewhere else.
It presses keys on the computer that runs the Java program. On a Selenium Grid, as in Practical 4, the browser can be on another computer, and its dialog never gets them.
Its waits are guesses, not conditions.

sendKeys has none of these problems. That is why it is the way to upload whenever the page has a file box, and nearly every upload page has one, even when it is hidden. The Robot class is for the moment when the dialog itself has to be answered.

Practical 13: File Upload and Download with the Robot Class

Program 3: download a file and verify it

package practicals;

import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.Map;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;

public class DownloadTest {
    public static void main(String[] args) throws Exception {
        Path folder = Path.of(System.getProperty("user.home"), "stqa-downloads");
        Files.createDirectories(folder);
        Path file = folder.resolve("timetable.csv");
        Files.deleteIfExists(file);   // a file left by an earlier run would prove nothing
        System.out.println("Before the click, timetable.csv exists: " + Files.exists(file));

        // Tell Chrome, before it starts, which folder to save in and not to ask.
        ChromeOptions options = new ChromeOptions();
        options.setExperimentalOption("prefs", Map.of(
                "download.default_directory", folder.toString(),
                "download.prompt_for_download", false));
        WebDriver driver = new ChromeDriver(options);
        try {
            driver.get("http://localhost:8080/downloads.php");
            driver.findElement(By.id("timetable")).click();

            // click() does not wait for the file to be saved, so wait for it: at most 10 seconds.
            long deadline = System.currentTimeMillis() + 10000;
            while (!Files.exists(file) && System.currentTimeMillis() < deadline) {
                Thread.sleep(200);
            }
            System.out.println("After the click, timetable.csv exists: " + Files.exists(file));

            List<String> lines = Files.readAllLines(file);
            System.out.println("Size: " + Files.size(file) + " bytes, " + lines.size() + " lines");
            System.out.println("First line: " + lines.get(0));
            System.out.println("Header as expected: " + lines.get(0).equals("Day,Batch,Practical"));
            System.out.println("Last line: " + lines.get(lines.size() - 1));
        } finally {
            driver.quit();
        }
    }
}

Before the click, timetable.csv exists: false
After the click, timetable.csv exists: true
Size: 238 bytes, 6 lines
First line: Day,Batch,Practical
Header as expected: true
Last line: Friday,A and B,Journal checking

What the run proves:

Before the click the file did not exist, because the program deleted it. Without that line, a file left by an earlier run would make the test pass even if the download failed: a false pass.
After the click it exists. The loop looked every 200 milliseconds, for at most 10 seconds, and stopped as soon as the file appeared.
Its size and contents are right: 238 bytes in 6 lines, the same as files/timetable.csv on the server, with the header Day,Batch,Practical first and Friday's row last. A check of existence alone would pass an empty or half-written file.

The two preferences go in through ChromeOptions, so Chrome starts with them already set.

Observations

Check	Expected	Actual	Status
click() on the file box	refused	invalid argument	Pass
sendKeys() of a missing file	refused	invalid argument: File not found	Pass
Upload by the file box	Uploaded assignment-1.txt (34 bytes)	Uploaded assignment-1.txt (34 bytes)	Pass
Server's copy	same text as the file sent	same	Pass
Upload by the hidden file box	Uploaded assignment-1.txt (34 bytes)	Uploaded assignment-1.txt (34 bytes)	Pass
A .jpg file	rejected	Only .txt, .csv and .pdf files are accepted.	Pass
Upload through the dialog, by Robot	Uploaded assignment-2.txt (43 bytes)	Uploaded assignment-2.txt (43 bytes)	Pass
Download: the file exists	true	true	Pass
Download: size and lines	238 bytes, 6 lines	238 bytes, 6 lines	Pass
Download: header	Day,Batch,Practical	Day,Batch,Practical	Pass

Result

A file was uploaded by script in three ways: by sendKeys into a visible file box, by sendKeys into a hidden one, and by the Robot class answering the operating system's file dialog. Each upload was validated by the page's message, and the first also by comparing the server's copy with the file sent. A file of the wrong type was rejected, as the page requires. A file was downloaded to a folder the test chose, and its existence, size and contents were verified.

Practical 13: File Upload and Download with the Robot Class

Where marks are lost

click() on the file box. WebDriver refuses it. Use sendKeys.

A relative path, or a file that is not there. sendKeys needs the whole path of a file that exists. Build it with Path.of, starting from the home folder that System.getProperty gives for "user.home", as these programs do.

Validating the message only. A page can say "Uploaded" and have saved nothing. Compare the server's copy when you can.

Not deleting the old download first. The test then passes without downloading anything.

Checking existence only. Check the size and the contents too.

No wait after the download click. The file is not there yet.

A Robot delay that is too short, or touching the machine while the Robot runs. The keys go astray.

For the journal

Aim; the pages under test; the three WebDriver rules; each program with its output; the figure of the dialog, or your own screenshot of it; the observations table; the result.

Quick revision

Upload: sendKeys(full path) into the input type="file". No dialog opens.
click() on a file box throws InvalidArgumentException, by the W3C rule.
A hidden file box still takes sendKeys, because strict file interactability starts switched off.
The Robot class, java.awt.Robot, presses real keys. Use it when the dialog itself must be answered.
Robot upload: click the button that opens the dialog, wait, put the path on the clipboard, press the dialog's keys, press Enter.
Robot needs a screen, the keyboard focus, and the same computer as the browser.
Download: Chrome's preferences download.default_directory and download.prompt_for_download (false).
Delete the old file first, wait for the new one, then check its existence, size and contents.

Questions you must be able to answer

1. Why does click() on a file box throw InvalidArgumentException? The WebDriver standard tells a driver to refuse a click on a file box, because the click would open the operating system's dialog, which WebDriver cannot control.

2. How does sendKeys upload a file without opening the dialog? WebDriver sets the file on the file box itself and fires the box's input and change events, as if a user had chosen the file.

3. Why does sendKeys work on a hidden file box? For a file box WebDriver does not wait for the element to be interactable, unless strict file interactability is switched on, and it starts switched off.

Practical 13: File Upload and Download with the Robot Class

4. When is the Robot class needed? When the operating system's file dialog is open and must be answered. Robot presses real keys, which reach the dialog.

5. Give two limitations of the Robot class. It needs a real display with the keyboard focus on the dialog, and it works only on the computer that runs the program, not on a Grid node. Its delays are also guesses.

6. Why does the download program delete the file before clicking? So that a file from an earlier run cannot make the test pass without any download.

7. How does the program verify the download? It waits until the file exists, then checks its size, its number of lines and its first line.

Contents This chapter on its own page

Chapter Seventeen

Practical 14: Screenshot on Failure and Logging with Log4j

Syllabus topic Module 2, "Screenshot Capture and Logging Mechanism", "Tool: Selenium WebDriver + Log4j", "Implement screenshot capture on test failure and integrate logging mechanism using Log4j. Generate execution logs and analyze error messages."

Aim

To capture a screenshot automatically when a test fails, to log a test run with Log4j to the screen and to a file, and to read the log to find out what went wrong and why.

What you need to know before you start

Why a test logs. A test suite often runs with nobody watching: at night, or on a build server such as the Jenkins of Practical 17. When it fails, the only witnesses are what it wrote down. System.out.println writes to the screen and nowhere else, has no time on it, and cannot be switched off without editing the program. A logging library does all of that from a configuration file.

Log4j 2 is the logging library this practical names. Its parts:

Part	What it is
Logger	the object a program writes to, one for each class, got from `LogManager`
Level	how serious a message is: TRACE, DEBUG, INFO, WARN, ERROR, FATAL, from least to most
Appender	where messages go: `Console`, `File` and many others
Layout	how each message is written: `PatternLayout` with a pattern of fields
log4j2.xml	the configuration file, which Log4j finds by itself on the classpath

Levels work as a threshold. A logger or an appender set to INFO writes INFO, WARN, ERROR and FATAL, and drops DEBUG and TRACE. Changing one word in the configuration makes a run more talkative or quieter, with no change to the program.

{} is a placeholder. log.info("Dashboard heading: {}", welcome) puts the value of welcome where the braces are. When the last argument is an exception, Log4j writes the exception's whole stack trace under the message; its manual warns against logging only the exception's message, because the stack trace, which says where the failure happened, is lost.

A screenshot comes from the TakesScreenshot interface, which every browser's driver class implements. Its method getScreenshotAs, asked for OutputType.FILE, returns a temporary PNG file, which the program must copy somewhere lasting. The W3C standard defines the picture as the browser's visual viewport: the part of the page on the screen, not the whole page.

A test fails in one of two ways, and telling them apart is what analysing error messages means:

A check fails. The page did something other than what the test expected. The program throws an AssertionError with a message saying what was expected and what happened. This is often a defect in the application.
A step fails. The script could not do what it was told: an element was not found, a wait ran out. Selenium throws an exception such as NoSuchElementException. This is often a mistake in the test.

The configuration

Save it as src/main/resources/log4j2.xml. Maven puts everything in src/main/resources on the classpath, which is where Log4j looks for it:

Practical 14: Screenshot on Failure and Logging with Log4j

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout pattern="%-5level %c{1} - %msg%n"/>
    </Console>
    <File name="File" fileName="logs/execution.log" append="false">
      <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level %c{1} - %msg%n"/>
    </File>
  </Appenders>
  <Loggers>
    <Root level="debug">
      <AppenderRef ref="Console" level="info"/>
      <AppenderRef ref="File"/>
    </Root>
  </Loggers>
</Configuration>

Line by line:

status="warn" is about Log4j itself: it reports its own problems, such as a mistake in this file, and stays quiet otherwise.
The Console appender writes to the screen. Its pattern is the level, padded to five characters (%-5level), the logger's short name (%c{1}, the class name without its package), and the message (%msg), then a new line (%n).
The File appender writes to logs/execution.log, and adds the date and time to the millisecond (%d{...}). append="false" starts a fresh file on every run, so the file holds the last run only; Log4j's default, true, keeps adding to the end.
Root level="debug" lets every message from DEBUG up into the logger. The two AppenderRef lines send them on: to the file, all of them; to the console, only INFO and above, because of level="info". The screen stays readable, and the file keeps the detail.

The libraries are already in the project: chapter six's pom.xml names log4j-api and log4j-core.

Log4j's two halves must be the same version. A program calls log4j-api; log4j-core does the work. Apache POI, in the same project for Practical 7, brings its own, older log4j-api, and when two libraries ask for different versions of the same thing, Maven keeps one. This command shows which:

$ mvn dependency:tree -Dincludes=org.apache.logging.log4j
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------< stqa:stqa-practicals >------------------------
[INFO] Building stqa-practicals 1.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- dependency:3.7.0:tree (default-cli) @ stqa-practicals ---
[INFO] stqa:stqa-practicals:jar:1.0
[INFO] +- org.apache.logging.log4j:log4j-api:jar:2.26.1:compile
[INFO] +- org.apache.logging.log4j:log4j-core:jar:2.26.1:compile
[INFO] \- org.apache.logging.log4j:log4j-slf4j2-impl:jar:2.26.1:compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.300 s
[INFO] Finished at: 2026-09-30T18:47:38+05:30
[INFO] ------------------------------------------------------------------------

All three are 2.26.1, because the pom names log4j-api itself. Before that line was added, Maven kept POI's log4j-api 2.24.3 beside log4j-core 2.26.1, and in the lab a program with no configuration file printed, instead of its own error message, an error from inside Log4j: An exception occurred processing Appender DefaultConsole-2, followed by Java's exception for an array index out of range, Index -1 out of bounds for length 0. When a library misbehaves in a way nobody else seems to report, check the versions Maven chose first.

The program

practicals/LoggedSuite.java runs three tests against the Practice Portal. The first should pass. The second tests the Marks sort of Practical 12, which has a real defect. The third contains a mistake of its own, put there on purpose, so that the log shows both kinds of failure:

Practical 14: Screenshot on Failure and Logging with Log4j

package practicals;

import java.io.File;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.time.Duration;
import java.util.List;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.openqa.selenium.By;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class LoggedSuite {
    static final Logger log = LogManager.getLogger(LoggedSuite.class);
    static WebDriver driver;
    static WebDriverWait wait;
    static int passed = 0, failed = 0;

    // A test is any method that may throw, so that run() can catch what it throws.
    interface Test {
        void run() throws Exception;
    }

    // A check that fails throws an AssertionError that says what was expected.
    static void check(boolean ok, String message) {
        if (!ok) {
            throw new AssertionError(message);
        }
    }

    // Save what the browser shows now, named after the test.
    static Path screenshot(String test) throws Exception {
        File shot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
        Path target = Path.of("screenshots", test + ".png");
        Files.createDirectories(target.getParent());
        Files.copy(shot.toPath(), target, StandardCopyOption.REPLACE_EXISTING);
        return target;
    }

    static void loginTest() {
        driver.get("http://localhost:8080/login.php");
        log.debug("Login page opened");
        driver.findElement(By.id("username")).sendKeys("asha");
        driver.findElement(By.id("password")).sendKeys("Asha@2026");
        driver.findElement(By.id("loginBtn")).click();
        String welcome = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("welcome"))).getText();
        log.debug("Dashboard heading: {}", welcome);
        check(welcome.equals("Welcome, Asha Patil"), "expected \"Welcome, Asha Patil\" but the page says \"" + welcome + "\"");
    }

    static void marksSortTest() {
        driver.get("http://localhost:8080/results.php");
        wait.until(ExpectedConditions.textToBePresentInElementLocated(By.id("count"), "Showing"));
        log.debug("Results table loaded");
        driver.findElement(By.cssSelector("th[data-key=marks]")).click();
        List<WebElement> marks = driver.findElements(By.cssSelector("#resultsBody td:nth-child(4)"));
        String first = marks.get(0).getText();
        log.debug("First mark after sorting by Marks: {}", first);
        check(first.equals("5"), "sorted by Marks, lowest first, the first mark should be 5 but is " + first);
    }

    static void searchTest() {
        driver.get("http://localhost:8080/results.php");
        wait.until(ExpectedConditions.textToBePresentInElementLocated(By.id("count"), "Showing"));
        driver.findElement(By.id("search")).sendKeys("Pune");   // the mistake: the box's id is "filter"
        check(driver.findElement(By.id("count")).getText().startsWith("Showing"), "no count shown");
    }

    // Run one test: log its start and its outcome, and on failure save a screenshot.
    static void run(String name, Test test) {
        log.info("START {}", name);
        try {
            test.run();
            passed++;
            log.info("PASS  {}", name);
        } catch (Throwable t) {                // an AssertionError and a Selenium exception alike
            failed++;
            log.error("FAIL  {}: {}", name, t.getMessage().lines().findFirst().orElse(""));
            log.debug("Stack trace of {}", name, t);   // the exception last: Log4j writes its stack trace
            try {
                log.error("Screenshot saved: {}", screenshot(name));
            } catch (Exception e) {
                log.error("No screenshot for {}: {}", name, e.getMessage());
            }
        }
    }

    public static void main(String[] args) {
        log.info("Suite started");
        driver = new ChromeDriver();
        wait = new WebDriverWait(driver, Duration.ofSeconds(5));
        try {
            run("login", LoggedSuite::loginTest);
            run("marks-sort", LoggedSuite::marksSortTest);
            run("student-search", LoggedSuite::searchTest);
        } finally {
            driver.quit();
        }
        log.info("Suite finished: {} passed, {} failed", passed, failed);
    }
}

INFO  LoggedSuite - Suite started
INFO  LoggedSuite - START login
INFO  LoggedSuite - PASS  login
INFO  LoggedSuite - START marks-sort
ERROR LoggedSuite - FAIL  marks-sort: sorted by Marks, lowest first, the first mark should be 5 but is 100
ERROR LoggedSuite - Screenshot saved: screenshots/marks-sort.png
INFO  LoggedSuite - START student-search
ERROR LoggedSuite - FAIL  student-search: no such element: Unable to locate element: {"method":"css selector","selector":"#search"}
ERROR LoggedSuite - Screenshot saved: screenshots/student-search.png
INFO  LoggedSuite - Suite finished: 1 passed, 2 failed

Practical 14: Screenshot on Failure and Logging with Log4j

The console shows the suite at INFO level: each test's start and its outcome, the two failures in one line each, where each screenshot went, and the totals. One test passed and two failed. The program itself ended normally, because run() caught every failure: one failing test must not stop the rest of the suite.

The log file

The console showed INFO and ERROR only. The file has everything:

$ cat logs/execution.log
2026-09-30 18:40:20.479 INFO  LoggedSuite - Suite started
2026-09-30 18:40:24.537 INFO  LoggedSuite - START login
2026-09-30 18:40:25.019 DEBUG LoggedSuite - Login page opened
2026-09-30 18:40:25.683 DEBUG LoggedSuite - Dashboard heading: Welcome, Asha Patil
2026-09-30 18:40:25.684 INFO  LoggedSuite - PASS  login
2026-09-30 18:40:25.693 INFO  LoggedSuite - START marks-sort
2026-09-30 18:40:26.900 DEBUG LoggedSuite - Results table loaded
2026-09-30 18:40:27.022 DEBUG LoggedSuite - First mark after sorting by Marks: 100
2026-09-30 18:40:27.023 ERROR LoggedSuite - FAIL  marks-sort: sorted by Marks, lowest first, the first mark should be 5 but is 100
2026-09-30 18:40:27.024 DEBUG LoggedSuite - Stack trace of marks-sort
java.lang.AssertionError: sorted by Marks, lowest first, the first mark should be 5 but is 100
	at practicals.LoggedSuite.check(LoggedSuite.java:35)
	at practicals.LoggedSuite.marksSortTest(LoggedSuite.java:67)
	at practicals.LoggedSuite.run(LoggedSuite.java:81)
	at practicals.LoggedSuite.main(LoggedSuite.java:102)
2026-09-30 18:40:27.128 ERROR LoggedSuite - Screenshot saved: screenshots/marks-sort.png
2026-09-30 18:40:27.131 INFO  LoggedSuite - START student-search
2026-09-30 18:40:28.312 ERROR LoggedSuite - FAIL  student-search: no such element: Unable to locate element: {"method":"css selector","selector":"#search"}
2026-09-30 18:40:28.313 DEBUG LoggedSuite - Stack trace of student-search
org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"#search"}
  (Session info: chrome=154.0.8037.57)
For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#nosuchelementexception
Build info: version: '4.49.0', revision: 'bedb24f bedb24f6f703876e7e4c5d506f0d697eb358c720'
System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '6.8.0-117-generic', java.version: '25.0.4.1'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [0714880c945eca7cc3c07ebbbda513a3, findElement {using=id, value=search}]
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 154.0.8037.57, chrome: {chromedriverVersion: 154.0.8037.57 (73c14f6228d7..., userDataDir: /tmp/org.chromium.Chromium....}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:41347}, goog:processID: 709, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:cdp: ws://localhost:41347/devtoo..., se:cdpVersion: 154.0.8037.57, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:cmtgKey: true, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
Session ID: 0714880c945eca7cc3c07ebbbda513a3
	at org.openqa.selenium.remote.ErrorCodec.decode(ErrorCodec.java:169)
	at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:143)
	at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:50)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:196)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:217)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:175)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:653)
	at org.openqa.selenium.remote.ElementLocation$ElementFinder$2.findElement(ElementLocation.java:173)
	at org.openqa.selenium.remote.ElementLocation.findElement(ElementLocation.java:60)
	at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:417)
	at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:411)
	at practicals.LoggedSuite.searchTest(LoggedSuite.java:73)
	at practicals.LoggedSuite.run(LoggedSuite.java:81)
	at practicals.LoggedSuite.main(LoggedSuite.java:103)
2026-09-30 18:40:28.428 ERROR LoggedSuite - Screenshot saved: screenshots/student-search.png
2026-09-30 18:40:28.602 INFO  LoggedSuite - Suite finished: 1 passed, 2 failed

What the file has that the screen did not:

The time of every line, to the millisecond. The gaps show where the time went: about four seconds between "Suite started" and the first test, which is Chrome starting.
The DEBUG lines. "First mark after sorting by Marks: 100" was written before the check ran, so the file already holds the value that made the test fail.
Both stack traces, under "Stack trace of marks-sort" and "Stack trace of student-search", because the exception was the last argument of log.debug.

Practical 14: Screenshot on Failure and Logging with Log4j

The Selenium exception is long because Selenium adds everything that might help: the browser's version, Selenium's version, the operating system, the command that failed and the session's settings. On your screen the command's two settings may be the other way round, {value=search, using=id}; Java does not fix their order.

The screenshots

$ ls screenshots
marks-sort.png
student-search.png

One screenshot per failure, each named after its test, so the second failure did not overwrite the first. This is marks-sort.png, exactly as the program saved it:

The results table sorted by Marks: 100, 38, 45, 5, 50 and so on

Figure 17.1 The screenshot LoggedSuite took when marks-sort failed. After a click on Marks the column reads 100, 38, 45, 5, 50: sorted as text, not as numbers. The table goes on below the bottom edge, because a screenshot shows only the visible part of the page.

Analysing the error messages

$ grep ERROR logs/execution.log
2026-09-30 18:40:27.023 ERROR LoggedSuite - FAIL  marks-sort: sorted by Marks, lowest first, the first mark should be 5 but is 100
2026-09-30 18:40:27.128 ERROR LoggedSuite - Screenshot saved: screenshots/marks-sort.png
2026-09-30 18:40:28.312 ERROR LoggedSuite - FAIL  student-search: no such element: Unable to locate element: {"method":"css selector","selector":"#search"}
2026-09-30 18:40:28.428 ERROR LoggedSuite - Screenshot saved: screenshots/student-search.png

$ grep "at practicals" logs/execution.log
	at practicals.LoggedSuite.check(LoggedSuite.java:35)
	at practicals.LoggedSuite.marksSortTest(LoggedSuite.java:67)
	at practicals.LoggedSuite.run(LoggedSuite.java:81)
	at practicals.LoggedSuite.main(LoggedSuite.java:102)
	at practicals.LoggedSuite.searchTest(LoggedSuite.java:73)
	at practicals.LoggedSuite.run(LoggedSuite.java:81)
	at practicals.LoggedSuite.main(LoggedSuite.java:103)

Reading a failed run, in this order:

Find the ERROR lines. grep ERROR in a terminal, or Ctrl+F in any editor, finds two failures, each with a screenshot.
Read each failure's first line. It says what went wrong.
Decide whose mistake it is.

marks-sort failed with the test's own message: "the first mark should be 5 but is 100". The test did everything it was told, and the page gave a wrong answer. The screenshot confirms it: the marks read 100, 38, 45, 5, sorted as text. This is a defect in the application, the one Practical 12 found; it is reported, with the screenshot attached.
student-search failed with no such element: Selenium found nothing with the id search. Something the test asked for is not on the page. Either the page has changed or the test is wrong, and the page's source settles it: the box is <input type="text" id="filter">. This is a mistake in the test. The locator is corrected to By.id("filter"), and nothing is reported against the application.

Find the line. A stack trace lists the methods that were running, the most recent first. The lines naming Selenium's own classes come first; the first line naming your own class is where your program was. For student-search that is the line naming LoggedSuite.searchTest and LoggedSuite.java:73: line 73, the findElement(By.id("search")) call. For marks-sort, line 35 is inside check(), and the line under it, 67, is the check in marksSortTest that failed.

Practical 14: Screenshot on Failure and Logging with Log4j

The screenshot of student-search shows the results page looking perfectly normal. A screenshot shows what the page looked like; when the fault is in the script, the message and the stack trace tell the story.

Observations

Test	Result	Error	Screenshot	Cause
login	Pass	none	none
marks-sort	Fail	AssertionError: the first mark should be 5 but is 100	marks-sort.png	defect in the application: Marks sorted as text
student-search	Fail	NoSuchElementException: no element with id search	student-search.png	mistake in the test: the box's id is filter

Result

Log4j was configured with a console appender at INFO and a file appender at DEBUG. A suite of three tests logged its steps; each of the two failures was logged at ERROR, with its stack trace in the file, and a screenshot was captured and named after the test. Reading the log separated a defect in the application, the Marks column sorting as text, from a mistake in the test, a wrong locator.

Where marks are lost

log4j2.xml in the wrong folder. Without it on the classpath, Log4j falls back to a default: only ERROR messages reach the screen, no file is written, and nothing warns you. In the lab, INFO and DEBUG lines simply vanished.

catch (Exception e) around a test. An AssertionError is not an Exception, so it escapes such a catch: no log line, no screenshot, and the rest of the suite never runs. Catch Throwable, as run() does.

Logging only e.getMessage(). The message says what went wrong but not where. Pass the exception itself as the last argument, and Log4j writes the stack trace.

Every screenshot with the same name. The second failure overwrites the first. Name each after its test.

Leaving the screenshot where getScreenshotAs put it. It is a temporary file; copy it to a folder of your own.

Calling every failure a defect. A NoSuchElementException is usually the test's own mistake. Read the message before reporting anything.

For the journal

Aim; log4j2.xml, with a line on each part; the program; its console output; the log file, or its ERROR lines and stack traces; the screenshot of the failure; the analysis of both failures; the observations table; the result.

Quick revision

Log4j parts: Logger, Level, Appender, Layout, and log4j2.xml in src/main/resources.
Levels, least to most serious: TRACE, DEBUG, INFO, WARN, ERROR, FATAL. A threshold lets through its level and everything above.
One logger, two appenders: the console at INFO, the file at DEBUG.
{} is a placeholder; an exception as the last argument puts its stack trace in the log.
Screenshot: cast the driver to TakesScreenshot, call getScreenshotAs with OutputType.FILE, then copy the file, named after the test.
Catch Throwable around each test, so that assertion failures are logged too.
An AssertionError is a failed check: often a defect. A Selenium exception is a failed step: often the test's own mistake.
In a stack trace, the first line naming your own class is where your program failed.

Practical 14: Screenshot on Failure and Logging with Log4j

Questions you must be able to answer

1. Why did the console show no DEBUG lines while the file did? The console's AppenderRef has level="info", so it lets through only INFO and above; the file's has no level of its own, so it receives everything the Root logger lets in, from DEBUG up.

2. What does append="false" do? It makes the File appender start a new file on every run instead of adding to the old one.

3. How does the program take a screenshot, and when? It casts the driver to TakesScreenshot, calls getScreenshotAs(OutputType.FILE) and copies the file to screenshots/, named after the test. It does this only when a test fails, inside run()'s catch.

4. Why does run() catch Throwable rather than Exception? A failed check throws AssertionError, which is an Error and not an Exception; catching Exception would miss it.

5. marks-sort and student-search both failed. Which one is a defect in the application, and how do you know? marks-sort. Its own check failed with the page's wrong answer, 100 first, and the screenshot shows the marks sorted as text. student-search failed because the test used an id that is not on the page.

6. How do you find the line of your program where a test failed? Read the stack trace from the top and stop at the first line that names your own class; the number after the colon is the line.

Contents This chapter on its own page

Chapter Eighteen

Practical 15: The Page Object Model Framework

Syllabus topic Module 2, "Page Object Model (POM) Framework Implementation", "Tool: Selenium WebDriver + TestNG", "Design and implement automation framework using Page Object Model (POM). Create reusable page classes and execute modular test cases."

Aim

To design an automation framework on the Page Object Model: reusable page classes for the pages of a web application, a base test class, and modular TestNG test cases that use them.

What you need to know before you start

The problem. In every program so far, the locators lived inside the tests: By.id("username") in each test that logs in. When the page changes, say the login box gets a new id, every one of those tests breaks and each must be edited.

The Page Object Model puts each page of the application into a class of its own, a page object, and the tests talk only to the page objects. Selenium's documentation calls a page object an interface to a page: when the page changes, only the page object changes, and the fix is in one place. Its rules, which this chapter follows:

A page object holds the page's locators, and offers what a user can do on the page as methods: loginAs(user, password), not a findElement and a click() on the login button.
A method that leads to another page returns that page's object. loginAs returns a DashboardPage, so a test can go on from there.
A page object makes no assertions. Deciding pass or fail is the test's job. The one check a page object does make is that the browser really is on its page, and it makes it in its constructor.

A framework here means the arrangement that makes the pattern work across many tests:

stqa-practicals/
  pom.xml
  src/main/java/pages/     LoginPage, DashboardPage, StudentRecordsPage, EditStudentPage
  src/test/java/tests/     BaseTest, LoginTest, StudentRecordsTest, WithoutPagesTest

The page classes are ordinary Java classes; the tests use them. This book keeps the pages in src/main/java, as a library the tests draw on; many teams keep both folders under src/test/java, and that works the same way.

TestNG, which MU names for this exercise, runs the tests. Practical 16 covers it in full; this chapter needs four pieces:

Piece	What it does
`@Test`	marks a method as a test
`@BeforeMethod`	runs before every test: here, opening a browser
`@AfterMethod`	runs after every test, even a failed one: here, closing the browser
`assertEquals`	fails the test when the two values it is given differ. TestNG takes the actual value first, then the expected one

Run in the lab with the strings "first-argument" and "second-argument", in that order, assertEquals reported "expected [second-argument] but found [first-argument]".

Modular test cases are tests that each stand alone: each opens its own browser, logs in if it needs to, and depends on no other test having run first. Any one of them can be run by itself, in any order.

The page classes

Each page of the Practice Portal that the tests visit gets a class in the pages package.

Practical 15: The Page Object Model Framework

package pages;

import java.time.Duration;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

// The login page: where its elements are, and what a user can do on it.
public class LoginPage {
    private final WebDriver driver;
    private final By username = By.id("username");
    private final By password = By.id("password");
    private final By loginButton = By.id("loginBtn");
    private final By error = By.id("error");
    private final By message = By.id("message");

    public LoginPage(WebDriver driver) {
        this.driver = driver;
        // The one check a page object makes: that the browser is on the page it models.
        new WebDriverWait(driver, Duration.ofSeconds(5)).until(ExpectedConditions.titleIs("Login - Practice Portal"));
    }

    public static LoginPage open(WebDriver driver) {
        driver.get("http://localhost:8080/login.php");
        return new LoginPage(driver);
    }

    private void submit(String user, String pass) {
        driver.findElement(username).sendKeys(user);
        driver.findElement(password).sendKeys(pass);
        WebElement button = driver.findElement(loginButton);
        button.click();
        // The answer may be this same page again, which looks exactly like the old
        // one: wait until the old page is gone, so nothing reads it by mistake.
        new WebDriverWait(driver, Duration.ofSeconds(5))
                .ignoring(WebDriverException.class)
                .until(ExpectedConditions.stalenessOf(button));
    }

    // A login that should work leads to the dashboard...
    public DashboardPage loginAs(String user, String pass) {
        submit(user, pass);
        return new DashboardPage(driver);
    }

    // ...and one that should fail stays on this page.
    public LoginPage loginExpectingError(String user, String pass) {
        submit(user, pass);
        return new LoginPage(driver);
    }

    public String errorMessage() {
        return driver.findElement(error).getText();
    }

    public String message() {
        return driver.findElement(message).getText();
    }
}

package pages;

import java.time.Duration;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class DashboardPage {
    private final WebDriver driver;
    private final By welcome = By.id("welcome");
    private final By studentRecords = By.linkText("Student records");
    private final By logout = By.id("logout");

    public DashboardPage(WebDriver driver) {
        this.driver = driver;
        new WebDriverWait(driver, Duration.ofSeconds(5)).until(ExpectedConditions.visibilityOfElementLocated(welcome));
    }

    public String welcomeText() {
        return driver.findElement(welcome).getText();
    }

    public StudentRecordsPage openStudentRecords() {
        driver.findElement(studentRecords).click();
        return new StudentRecordsPage(driver);
    }

    public LoginPage logout() {
        driver.findElement(logout).click();
        return new LoginPage(driver);
    }
}

package pages;

import java.time.Duration;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class StudentRecordsPage {
    private final WebDriver driver;
    private final By heading = By.tagName("h1");
    private final By message = By.id("message");

    public StudentRecordsPage(WebDriver driver) {
        this.driver = driver;
        new WebDriverWait(driver, Duration.ofSeconds(5)).until(ExpectedConditions.textToBe(heading, "Student Records"));
    }

    public static StudentRecordsPage open(WebDriver driver) {
        driver.get("http://localhost:8080/students.php");
        return new StudentRecordsPage(driver);
    }

    // One cell of a student's row. Each row's id is "row-" and the roll number.
    private String cell(int roll, int column) {
        return driver.findElement(By.cssSelector("#row-" + roll + " td:nth-child(" + column + ")")).getText();
    }

    public String cityOf(int roll) {
        return cell(roll, 4);
    }

    public String marksOf(int roll) {
        return cell(roll, 5);
    }

    public EditStudentPage edit(int roll) {
        driver.findElement(By.cssSelector("#row-" + roll + " a")).click();
        return new EditStudentPage(driver, roll);
    }

    public String message() {
        return driver.findElement(message).getText();
    }
}

package pages;

import java.time.Duration;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class EditStudentPage {
    private final WebDriver driver;
    private final int roll;
    private final By city = By.id("city");
    private final By marks = By.id("marks");
    private final By save = By.id("save");
    private final By error = By.id("error");

    public EditStudentPage(WebDriver driver, int roll) {
        this.driver = driver;
        this.roll = roll;
        new WebDriverWait(driver, Duration.ofSeconds(5))
                .until(ExpectedConditions.textToBe(By.tagName("h1"), "Edit Student " + roll));
    }

    private void fillAndSave(String newCity, String newMarks) {
        driver.findElement(city).clear();
        driver.findElement(city).sendKeys(newCity);
        driver.findElement(marks).clear();
        driver.findElement(marks).sendKeys(newMarks);
        WebElement button = driver.findElement(save);
        button.click();
        new WebDriverWait(driver, Duration.ofSeconds(5))
                .ignoring(WebDriverException.class)
                .until(ExpectedConditions.stalenessOf(button));
    }

    public StudentRecordsPage saveChanges(String newCity, String newMarks) {
        fillAndSave(newCity, newMarks);
        return new StudentRecordsPage(driver);
    }

    public EditStudentPage saveExpectingError(String newCity, String newMarks) {
        fillAndSave(newCity, newMarks);
        return new EditStudentPage(driver, roll);
    }

    public String errorMessage() {
        return driver.findElement(error).getText();
    }
}

Practical 15: The Page Object Model Framework

What the four classes have in common:

Locators are private fields, named for what they find. Nothing outside the class knows that the login button's id is loginBtn.
The constructor checks the page: the login page by its title, the dashboard by its welcome heading, the records and edit pages by their headings. If a test lands on the wrong page, it fails right there, with a message about the wait that ran out, instead of three steps later with a puzzling NoSuchElementException.
Methods are named for what a user does, and return the page that comes next: loginAs returns a DashboardPage, openStudentRecords a StudentRecordsPage, logout a LoginPage. A method that stays on the page but expects an error, loginExpectingError, returns this page's class.
Reading methods return values (welcomeText, errorMessage, cityOf) for the test to judge. None of them asserts anything.

The race the page objects had to learn about. On the first run, eight tests passed and wrongPasswordIsRefused failed with NoSuchElementException for #error. The wrong password sends the form, and the server answers with the login page again, now carrying the error. But the new login page has the same title as the old one, so the constructor's check passed at once, on the OLD page, and errorMessage() looked for #error before the new page had arrived. The cure is in submit() and fillAndSave(): keep hold of the button that was clicked, and wait with stalenessOf(button), from ExpectedConditions, until that button no longer belongs to the page in the browser. An element goes stale when its page is replaced, so once the old button is stale, the new page is the one being read. With that wait, the nine tests passed on the runs that followed, and then the book's full re-run found one more race: marksAbove100AreRefused failed inside that very wait with WebDriverException: unknown error: ... Node with given id does not belong to the document. Chrome, asked about the button while it was replacing the page, answered with a general error instead of saying the element was stale. ignoring, given WebDriverException.class, makes the wait treat that answer as "not yet" and ask again a moment later, when Chrome does report the button stale.

Practical 15: The Page Object Model Framework

The base test

package tests;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;

// Every test class extends this: a fresh browser before each test, closed after it.
public abstract class BaseTest {
    protected WebDriver driver;

    @BeforeMethod
    public void openBrowser() {
        driver = new ChromeDriver();
    }

    @AfterMethod
    public void closeBrowser() {
        driver.quit();
    }
}

BaseTest is abstract: it holds no tests of its own and is never run by itself. Each test class extends it and inherits driver, which TestNG sets up before each test and closes after it. One browser per test costs time, a few seconds each, and buys independence: nothing one test does to the browser, such as logging in, can leak into the next.

The test cases

package tests;

import static org.testng.Assert.assertEquals;

import org.testng.annotations.Test;

import pages.DashboardPage;
import pages.LoginPage;

public class LoginTest extends BaseTest {

    @Test
    public void validLoginShowsWelcome() {
        DashboardPage dashboard = LoginPage.open(driver).loginAs("asha", "Asha@2026");
        assertEquals(dashboard.welcomeText(), "Welcome, Asha Patil");
    }

    @Test
    public void wrongPasswordIsRefused() {
        LoginPage login = LoginPage.open(driver).loginExpectingError("asha", "wrong");
        assertEquals(login.errorMessage(), "Invalid username or password.");
    }

    @Test
    public void emptyFieldsAreRefused() {
        LoginPage login = LoginPage.open(driver).loginExpectingError("", "");
        assertEquals(login.errorMessage(), "Please enter both username and password.");
    }

    @Test
    public void logoutReturnsToLogin() {
        LoginPage login = LoginPage.open(driver).loginAs("ravi", "Ravi@2026").logout();
        assertEquals(login.message(), "You have been logged out.");
    }
}

package tests;

import static org.testng.Assert.assertEquals;

import org.testng.annotations.Test;

import pages.EditStudentPage;
import pages.LoginPage;
import pages.StudentRecordsPage;

public class StudentRecordsTest extends BaseTest {

    @Test
    public void editCityAndMarks() {
        StudentRecordsPage records = LoginPage.open(driver)
                .loginAs("asha", "Asha@2026")
                .openStudentRecords()
                .edit(103)
                .saveChanges("Navi Mumbai", "61");
        assertEquals(records.message(), "Record 103 updated.");
        assertEquals(records.cityOf(103), "Navi Mumbai");
        assertEquals(records.marksOf(103), "61");
    }

    @Test
    public void marksAbove100AreRefused() {
        EditStudentPage edit = StudentRecordsPage.open(driver).edit(104).saveExpectingError("Mumbai", "101");
        assertEquals(edit.errorMessage(), "Marks must be a whole number from 0 to 100.");
    }

    @Test
    public void emptyCityIsRefused() {
        EditStudentPage edit = StudentRecordsPage.open(driver).edit(105).saveExpectingError("", "49");
        assertEquals(edit.errorMessage(), "City cannot be empty.");
    }
}

package tests;

import static org.testng.Assert.assertEquals;

import java.time.Duration;

import org.openqa.selenium.By;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.testng.annotations.Test;

// Two of the same tests written WITHOUT page objects: every locator and every wait inside the test.
public class WithoutPagesTest extends BaseTest {

    @Test
    public void validLoginShowsWelcome() {
        WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(5));
        driver.get("http://localhost:8080/login.php");
        driver.findElement(By.id("username")).sendKeys("asha");
        driver.findElement(By.id("password")).sendKeys("Asha@2026");
        driver.findElement(By.id("loginBtn")).click();
        String welcome = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("welcome"))).getText();
        assertEquals(welcome, "Welcome, Asha Patil");
    }

    @Test
    public void editCityAndMarks() {
        WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(5));
        driver.get("http://localhost:8080/login.php");
        driver.findElement(By.id("username")).sendKeys("asha");
        driver.findElement(By.id("password")).sendKeys("Asha@2026");
        driver.findElement(By.id("loginBtn")).click();
        wait.until(ExpectedConditions.elementToBeClickable(By.linkText("Student records"))).click();
        wait.until(ExpectedConditions.elementToBeClickable(By.cssSelector("#row-103 a"))).click();
        wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("city"))).clear();
        driver.findElement(By.id("city")).sendKeys("Navi Mumbai");
        driver.findElement(By.id("marks")).clear();
        driver.findElement(By.id("marks")).sendKeys("61");
        driver.findElement(By.id("save")).click();
        String message = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("message"))).getText();
        assertEquals(message, "Record 103 updated.");
    }
}

Practical 15: The Page Object Model Framework

Read editCityAndMarks aloud and it is the user's story: open the login page, log in as Asha, open the student records, edit roll 103, save a new city and marks, then check the message, the city and the marks. Each line returns the page the next line needs, which is why the calls chain. There is not one locator in LoginTest or StudentRecordsTest.

WithoutPagesTest does two of the same jobs the old way, for comparison: the same locators typed out in both tests, and the waits too.

Running the tests

$ mvn test -Dtest='tests/*Test'
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------< stqa:stqa-practicals >------------------------
[INFO] Building stqa-practicals 1.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/main/resources
[INFO]
[INFO] --- compiler:3.16.0:compile (default-compile) @ stqa-practicals ---
[INFO] Recompiling the module because of added or removed source files.
[INFO] Compiling 4 source files with javac [debug release 17] to target/classes
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/test/resources
[INFO]
[INFO] --- compiler:3.16.0:testCompile (default-testCompile) @ stqa-practicals ---
[INFO] Recompiling the module because of changed dependency.
[INFO] Compiling 4 source files with javac [debug release 17] to target/test-classes
[INFO]
[INFO] --- surefire:3.5.6:test (default-test) @ stqa-practicals ---
[INFO] Using configured provider org.apache.maven.surefire.junit4.JUnit4Provider
[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
Sep 30, 2026 7:03:19 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:03:33 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:03:47 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:03:51 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:03:58 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:04:01 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:04:04 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:04:09 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:04:14 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 101.8 s -- in TestSuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:51 min
[INFO] Finished at: 2026-09-30T19:04:17+05:30
[INFO] ------------------------------------------------------------------------

Practical 15: The Page Object Model Framework

Reading the run:

-Dtest='tests/*Test' runs only this practical's classes. Surefire's documentation says each pattern becomes **/pattern.java, so this means every class in a tests folder whose name ends in Test; the project also holds earlier practicals' test classes, which this leaves out. The quotes stop a Linux or macOS terminal from expanding the star itself.
The two runners, as in chapter six: the JUnit runner found nothing to run, and the TestNG runner ran all nine tests together as one TestSuite: four in LoginTest, three in StudentRecordsTest and two in WithoutPagesTest. BaseTest was left out because it is abstract.
The warning about CDP version 154, explained in chapter six, appears once for every browser start: nine tests, nine browsers, nine warnings.
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, and BUILD SUCCESS.

Where the locators live

$ grep -c "By\." src/test/java/tests/*.java src/main/java/pages/*.java
src/test/java/tests/BaseTest.java:0
src/test/java/tests/LoginTest.java:0
src/test/java/tests/StudentRecordsTest.java:0
src/test/java/tests/WithoutPagesTest.java:15
src/main/java/pages/DashboardPage.java:3
src/main/java/pages/EditStudentPage.java:5
src/main/java/pages/LoginPage.java:5
src/main/java/pages/StudentRecordsPage.java:4

$ grep -c 'By.id("username")' src/test/java/tests/WithoutPagesTest.java src/main/java/pages/LoginPage.java
src/test/java/tests/WithoutPagesTest.java:2
src/main/java/pages/LoginPage.java:1

The first count is the whole argument for the pattern. grep -c counts the lines that contain a match. The two page-object test classes have no line with a locator at all, and every locator is in a page class, where it appears once. WithoutPagesTest has 15 such lines for only two tests. The second count shows the same thing for one element: the username box's locator is typed twice in WithoutPagesTest, once for each test that logs in, and exactly once in LoginPage. If that box's id changed, LoginPage would need one edit, and the page-object tests none; the page-less tests would need one edit per test that logs in, however many there were by then.

Practical 15: The Page Object Model Framework

Observations

Test class	Test	Result
LoginTest	validLoginShowsWelcome	Pass
LoginTest	wrongPasswordIsRefused	Pass, after the stale-page wait was added
LoginTest	emptyFieldsAreRefused	Pass
LoginTest	logoutReturnsToLogin	Pass
StudentRecordsTest	editCityAndMarks	Pass
StudentRecordsTest	marksAbove100AreRefused	Pass
StudentRecordsTest	emptyCityIsRefused	Pass
WithoutPagesTest	validLoginShowsWelcome	Pass
WithoutPagesTest	editCityAndMarks	Pass

Result

An automation framework on the Page Object Model was built: four reusable page classes holding every locator and returning the next page, a base test class giving each test its own browser, and two TestNG test classes with seven modular test cases that contain no locators. All nine tests, with the two written without page objects for comparison, passed through Maven. A race in the first version, a page object reading the old page after a form was sent, was found and cured with a wait for the old page to go stale.

Where marks are lost

Assertions inside a page object. The test decides pass or fail; the page object only reports what the page shows.

Locators left in the tests. Then the framework has page classes and none of their benefit.

A method that moves to another page but returns nothing. The test cannot go on from it; return the next page's object.

One browser shared by every test. A test that logs in changes what the next one sees. Open and close the browser around each test.

Reading straight after a click that sends a form. The old page may still be there. Wait for it to go stale, or for something only the new page has.

A test class name that does not end in Test. Surefire's default patterns (Test, Test, Tests, TestCase) never find it, and it silently never runs.

For the journal

Aim; the framework's folders; the four page classes; the base test; the test classes; the Maven run; the two locator counts, with a sentence on what they show; the observations table; the result.

Quick revision

Page object: one class per page, holding its locators and its actions as methods.
A method that leads to another page returns that page's object; tests chain the calls.
No assertions in page objects, except the constructor's check that the page is the right one.
BaseTest: @BeforeMethod opens a browser, @AfterMethod closes it; tests extend it.
Modular tests stand alone and run in any order.
After a click that sends a form, wait for the old page to go stale.
Run one practical's tests: mvn test -Dtest='tests/*Test'.
TestNG's assertEquals(actual, expected): the actual value first.

Questions you must be able to answer

1. What is a page object? A class that stands for one page of the application: it holds that page's locators and offers what a user can do on the page as methods, so that tests never touch a locator.

Practical 15: The Page Object Model Framework

2. Why does loginAs return a DashboardPage? Because logging in leads to the dashboard. Returning its page object lets the test go on to the next step, and the dashboard's constructor checks that the browser really got there.

3. Where are assertions written in this framework, and why there? In the test classes only. A page object reports what the page shows; judging whether that is right is the test's job, which keeps the page objects reusable by any test.

4. What does BaseTest do? It opens a new browser before every test and closes it after, so each test starts fresh, and the test classes inherit this instead of repeating it.

5. Why did wrongPasswordIsRefused fail on the first run? The login page came back with the same title, so the page object's check passed on the old page and read it before the new page arrived. Waiting for the clicked button to go stale fixed it.

6. How do the locator counts show the value of the Page Object Model? The page-object tests contain no locators, and each locator is written once in its page class; the tests without page objects repeat them, so a change to one element means an edit in every test that uses it.

Contents This chapter on its own page

Chapter Nineteen

Practical 16: TestNG Annotations, Groups, Priorities and Reports

Syllabus topic Module 2, "TestNG Framework Implementation and Reporting", "Tool: Selenium WebDriver + TestNG", "Create and execute automated test cases using TestNG annotations. Group test cases, define priorities, and generate HTML test reports."

Aim

To create and run automated test cases with TestNG's annotations, to group them and set their priorities in code and in testng.xml, and to generate and read TestNG's HTML test reports.

What you need to know before you start

TestNG runs test methods and reports on them. Practical 15 used four of its pieces; this practical uses the rest of what MU names: annotations, groups, priorities and reports.

Annotations are the @ words above methods. They tell TestNG what a method is and when to run it:

Annotation	Runs
`@BeforeSuite`, `@AfterSuite`	once, before and after everything in the suite
`@BeforeTest`, `@AfterTest`	before and after each `<test>` in the suite file
`@BeforeClass`, `@AfterClass`	before the first and after the last test of a class
`@BeforeMethod`, `@AfterMethod`	before and after every test method
`@Test`	is a test

@Test takes attributes. Three are used here: priority, a number, where TestNG's documentation says lower priorities are scheduled first; groups, the names of the groups a test belongs to; and description, a sentence that the reports print beside the test.

A suite file, usually testng.xml, says what to run: a <suite> holds one or more <test> blocks, each naming classes and, if wanted, the groups to include or exclude. Maven runs a suite file when asked with -Dsurefire.suiteXmlFiles=, a property Surefire's documentation lists.

Reports. After every run TestNG writes HTML reports. Under Maven they go to target/surefire-reports.

This practical builds on Practical 15: the same pages and tests packages, the same BaseTest, with one new page class and two new test classes.

The annotations, in the order TestNG runs them

LifecycleTest opens no browser. Each of its methods prints its own name, so the run shows the order:

package tests;

import org.testng.annotations.AfterClass;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

// No browser here: each method only says when TestNG ran it.
public class LifecycleTest {

    @BeforeSuite
    public void beforeSuite() {
        System.out.println("@BeforeSuite: once, before everything in the suite");
    }

    @BeforeTest
    public void beforeTest() {
        System.out.println("@BeforeTest: before each <test> of the suite file");
    }

    @BeforeClass
    public void beforeClass() {
        System.out.println("@BeforeClass: once, before this class's first test");
    }

    @BeforeMethod
    public void beforeMethod() {
        System.out.println("  @BeforeMethod");
    }

    @Test(priority = 2)
    public void checkResults() {
        System.out.println("    @Test checkResults (priority 2)");
    }

    @Test(priority = 1)
    public void logIn() {
        System.out.println("    @Test logIn (priority 1)");
    }

    @Test
    public void openHomePage() {
        System.out.println("    @Test openHomePage (no priority given)");
    }

    @AfterMethod
    public void afterMethod() {
        System.out.println("  @AfterMethod");
    }

    @AfterClass
    public void afterClass() {
        System.out.println("@AfterClass");
    }

    @AfterTest
    public void afterTest() {
        System.out.println("@AfterTest");
    }

    @AfterSuite
    public void afterSuite() {
        System.out.println("@AfterSuite");
    }
}

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Lifecycle">
  <test name="Annotations">
    <classes>
      <class name="tests.LifecycleTest"/>
    </classes>
  </test>
</suite>

Practical 16: TestNG Annotations, Groups, Priorities and Reports

$ mvn test -Dsurefire.suiteXmlFiles=lifecycle.xml
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------< stqa:stqa-practicals >------------------------
[INFO] Building stqa-practicals 1.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/main/resources
[INFO]
[INFO] --- compiler:3.16.0:compile (default-compile) @ stqa-practicals ---
[INFO] Recompiling the module because of added or removed source files.
[INFO] Compiling 4 source files with javac [debug release 17] to target/classes
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/test/resources
[INFO]
[INFO] --- compiler:3.16.0:testCompile (default-testCompile) @ stqa-practicals ---
[INFO] Recompiling the module because of changed dependency.
[INFO] Compiling 5 source files with javac [debug release 17] to target/test-classes
[INFO]
[INFO] --- surefire:3.5.6:test (default-test) @ stqa-practicals ---
[INFO] Using configured provider org.apache.maven.surefire.junit4.JUnit4Provider
[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
@BeforeSuite: once, before everything in the suite
@BeforeTest: before each <test> of the suite file
@BeforeClass: once, before this class's first test
  @BeforeMethod
    @Test openHomePage (no priority given)
  @AfterMethod
  @BeforeMethod
    @Test logIn (priority 1)
  @AfterMethod
  @BeforeMethod
    @Test checkResults (priority 2)
  @AfterMethod
@AfterClass
@AfterTest
@AfterSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.667 s -- in TestSuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  5.931 s
[INFO] Finished at: 2026-09-30T19:09:03+05:30
[INFO] ------------------------------------------------------------------------

Between "Running TestSuite" and the result is the order TestNG followed:

Outside in, then back out: the suite, the <test>, the class, then for each test method @BeforeMethod, the test and @AfterMethod; then the class, the <test> and the suite close in reverse order.
Priority decided the test order, not the order in the file: the file lists checkResults (priority 2) first, yet openHomePage ran first, then logIn (1), then checkResults (2). openHomePage has no priority, and TestNG's @Test gives priority the default value 0, so it came before 1.

Maven ran three tests, all passed. Everything before "T E S T S" is Maven compiling, as in Practical 15.

A new page, a new page class

package pages;

import java.time.Duration;
import java.util.ArrayList;
import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class ResultsPage {
    private final WebDriver driver;
    private final By count = By.id("count");
    private final By filter = By.id("filter");
    private final By marksHeading = By.cssSelector("th[data-key=marks]");
    private final By marksCells = By.cssSelector("#resultsBody td:nth-child(4)");

    public ResultsPage(WebDriver driver) {
        this.driver = driver;
        // The table arrives a second after the page does: wait until the count line reports it.
        new WebDriverWait(driver, Duration.ofSeconds(5))
                .until(ExpectedConditions.textToBePresentInElementLocated(count, "Showing"));
    }

    public static ResultsPage open(WebDriver driver) {
        driver.get("http://localhost:8080/results.php");
        return new ResultsPage(driver);
    }

    public ResultsPage filterBy(String text) {
        driver.findElement(filter).sendKeys(text);
        return this;
    }

    public ResultsPage sortByMarks() {
        driver.findElement(marksHeading).click();
        return this;
    }

    public String countText() {
        return driver.findElement(count).getText();
    }

    public List<String> marks() {
        List<String> values = new ArrayList<>();
        for (WebElement cell : driver.findElements(marksCells)) {
            values.add(cell.getText());
        }
        return values;
    }
}

Practical 16: TestNG Annotations, Groups, Priorities and Reports

ResultsPage is a page class like Practical 15's four: the results table's locators, its actions (filterBy, sortByMarks, each returning the page, because the page stays the same), and its readings. Its constructor waits for the count line, because the table is filled a second after the page loads. Adding a page to the framework means adding a class; no test changes.

Test cases with groups and priorities

package tests;

import static org.testng.Assert.assertEquals;

import org.testng.annotations.Test;

import pages.LoginPage;
import pages.ResultsPage;

public class PortalTest extends BaseTest {

    @Test(priority = 1, groups = "smoke", description = "The login page opens")
    public void loginPageOpens() {
        LoginPage.open(driver);
        assertEquals(driver.getTitle(), "Login - Practice Portal");
    }

    @Test(priority = 2, groups = {"smoke", "regression"}, description = "A registered student can log in")
    public void validLogin() {
        assertEquals(LoginPage.open(driver).loginAs("asha", "Asha@2026").welcomeText(), "Welcome, Asha Patil");
    }

    @Test(priority = 3, groups = "regression", description = "A wrong password is refused")
    public void wrongPasswordIsRefused() {
        assertEquals(LoginPage.open(driver).loginExpectingError("asha", "wrong").errorMessage(),
                "Invalid username or password.");
    }

    @Test(priority = 4, groups = "regression", description = "The filter keeps only the matching students")
    public void filterByPune() {
        assertEquals(ResultsPage.open(driver).filterBy("Pune").countText(), "Showing 3 of 15 students");
    }

    @Test(priority = 5, groups = "regression", description = "Sorting by Marks puts the lowest mark first")
    public void marksSortLowestFirst() {
        assertEquals(ResultsPage.open(driver).sortByMarks().marks().get(0), "5");
    }
}

Five tests, each with a priority, a group and a description. Two groups:

smoke: a quick check that the application is alive at all: the login page opens and a student can log in.
regression: the fuller check that what worked before still works.

validLogin belongs to both. marksSortLowestFirst tests the Marks sort that Practical 12 found sorting as text, so it should fail.

The suite file runs the class twice, as two <test> blocks, one per group:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Practice Portal">
  <test name="Smoke">
    <groups>
      <run>
        <include name="smoke"/>
      </run>
    </groups>
    <classes>
      <class name="tests.PortalTest"/>
    </classes>
  </test>
  <test name="Regression">
    <groups>
      <run>
        <include name="regression"/>
      </run>
    </groups>
    <classes>
      <class name="tests.PortalTest"/>
    </classes>
  </test>
</suite>

Each <test> names tests.PortalTest and includes one group, so Smoke runs the class's smoke tests and Regression its regression tests.

Running by groups: one change to BaseTest

The first run of this suite in the lab failed all five tests with the same message, a NullPointerException saying that WebDriver.get(String) could not be called because "driver" is null. No browser had been opened. When a suite runs by groups, TestNG runs only the methods that belong to the included groups, and that rule covers @BeforeMethod too: openBrowser in BaseTest belongs to no group, so it never ran. TestNG's documentation gives the cure. With alwaysRun = true, a before-method runs whatever groups it belongs to, and an after-method runs even when a test before it failed. BaseTest, with that change:

Practical 16: TestNG Annotations, Groups, Priorities and Reports

package tests;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;

// Every test class extends this: a fresh browser before each test, closed after it.
// alwaysRun = true: run these even when the suite selects tests by group.
public abstract class BaseTest {
    protected WebDriver driver;

    @BeforeMethod(alwaysRun = true)
    public void openBrowser() {
        driver = new ChromeDriver();
    }

    @AfterMethod(alwaysRun = true)
    public void closeBrowser() {
        driver.quit();
    }
}

Run the suite:

$ mvn test -Dsurefire.suiteXmlFiles=testng.xml
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------< stqa:stqa-practicals >------------------------
[INFO] Building stqa-practicals 1.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/main/resources
[INFO]
[INFO] --- compiler:3.16.0:compile (default-compile) @ stqa-practicals ---
[INFO] Recompiling the module because of added or removed source files.
[INFO] Compiling 5 source files with javac [debug release 17] to target/classes
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/test/resources
[INFO]
[INFO] --- compiler:3.16.0:testCompile (default-testCompile) @ stqa-practicals ---
[INFO] Recompiling the module because of changed dependency.
[INFO] Compiling 6 source files with javac [debug release 17] to target/test-classes
[INFO]
[INFO] --- surefire:3.5.6:test (default-test) @ stqa-practicals ---
[INFO] Using configured provider org.apache.maven.surefire.junit4.JUnit4Provider
[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
Sep 30, 2026 7:11:04 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:11:06 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:11:08 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:11:10 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:11:11 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:11:14 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
[ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 19.94 s <<< FAILURE! -- in TestSuite
[ERROR] tests.PortalTest.marksSortLowestFirst -- Time elapsed: 1.534 s <<< FAILURE!
java.lang.AssertionError: expected [5] but found [100]
	at org.testng.Assert.fail(Assert.java:111)
	at org.testng.Assert.failNotEquals(Assert.java:1590)
	at org.testng.Assert.assertEqualsImpl(Assert.java:150)
	at org.testng.Assert.assertEquals(Assert.java:132)
	at org.testng.Assert.assertEquals(Assert.java:656)
	at org.testng.Assert.assertEquals(Assert.java:666)
	at tests.PortalTest.marksSortLowestFirst(PortalTest.java:36)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:141)
	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:697)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:230)
	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:63)
	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:1005)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:203)
	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:154)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:134)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at org.testng.TestRunner.privateRun(TestRunner.java:744)
	at org.testng.TestRunner.run(TestRunner.java:616)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:421)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:413)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:373)
	at org.testng.SuiteRunner.run(SuiteRunner.java:312)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1274)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1208)
	at org.testng.TestNG.runSuites(TestNG.java:1112)
	at org.testng.TestNG.run(TestNG.java:1079)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:308)
	at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:71)
	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:113)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   PortalTest.marksSortLowestFirst:36 expected [5] but found [100]
[INFO]
[ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  22.513 s
[INFO] Finished at: 2026-09-30T19:11:16+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.6:test (default-test) on project stqa-practicals: There are test failures.
[ERROR]
[ERROR] See /home/student/stqa-practicals/target/surefire-reports for the individual test results.
[ERROR] See dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Practical 16: TestNG Annotations, Groups, Priorities and Reports

Reading the run:

Tests run: 6, Failures: 1. Smoke ran two tests and Regression four; validLogin, in both groups, ran in both, so six runs from five methods.
The one failure is the defect, not the framework: marksSortLowestFirst expected 5 and found 100, the Marks column sorting as text. TestNG's message, "expected [5] but found [100]", comes from assertEquals(actual, expected).
The lines starting with at are the failure's stack trace, printed by Surefire. Its first line naming your own code, the one for marksSortLowestFirst at PortalTest.java:36, is the failing assertion; the rest is TestNG and Java at work, and differs a little between Java versions.
BUILD FAILURE is correct: a build with a failing test must not look green. It is not a crash; the reports were written all the same.

Practical 16: TestNG Annotations, Groups, Priorities and Reports

The reports

$ ls target/surefire-reports
Lifecycle
Practice Portal
TEST-TestSuite.xml
TestSuite.txt
bullet_point.png
collapseall.gif
emailable-report.html
failed.png
index.html
jquery-3.6.0.min.js
junitreports
navigator-bullet.png
passed.png
skipped.png
testng-failed.xml
testng-reports.css
testng-reports.js
testng-reports1.css
testng-reports2.js
testng-results.xml

The two reports to open in a browser are emailable-report.html and index.html. The rest are what they are built from: testng-results.xml holds the whole run for other programs to read (Jenkins reads such files in Practical 17), testng-failed.xml is a ready-made suite file that runs only the tests that failed, and Lifecycle and Practice Portal are folders named after the two suites run in this project.

emailable-report.html is a summary built to be mailed: one row per <test>, with passed, skipped and failed counts, times and groups, then every method, coloured by result:

TestNG's emailable report: Smoke 2 passed, Regression 3 passed and 1 failed

Figure 19.1 The emailable report of this run. Smoke passed both its tests; Regression passed three and failed one, marksSortLowestFirst, shown in red. The Included Groups column shows what each test ran.

index.html is TestNG's own report, with a menu on the left and the details of any failure on the right, including the test's description:

TestNG's index report with the failed method marksSortLowestFirst

Figure 19.2 TestNG's index.html for the same run: 6 methods, 1 failed, 5 passed. On the right, the failure's message and the start of its stack trace, and below them the test's description in italics.

Observations

`<test>`	Test	Priority	Groups	Result
Smoke	loginPageOpens	1	smoke	Pass
Smoke	validLogin	2	smoke, regression	Pass
Regression	validLogin	2	smoke, regression	Pass
Regression	wrongPasswordIsRefused	3	regression	Pass
Regression	filterByPune	4	regression	Pass
Regression	marksSortLowestFirst	5	regression	Fail: expected [5] but found [100]

Result

Test cases were written with TestNG's annotations and run in the order the annotations define, with priorities deciding the order of the tests. The tests were grouped as smoke and regression and selected by group through testng.xml, which needed alwaysRun = true on the base class's set-up and tear-down. Of six test runs, five passed and one failed, on the Marks sort defect. TestNG generated its HTML reports, emailable-report.html and index.html, which show the same result.

Where marks are lost

Running by groups without alwaysRun = true on the set-up. The browser never opens, and every test fails with a NullPointerException about driver.

Expecting the file's order to be the run order. Priority decides; with no priority, a test counts as 0.

Reading assertEquals backwards. TestNG's is assertEquals(actual, expected); swapped, the failure message says the opposite of the truth.

No descriptions. The report then shows method names only. A sentence per test makes the report readable by someone who never saw the code.

Showing only the console. MU asks for HTML reports; the journal needs the report, printed or as a screenshot.

Calling BUILD FAILURE a broken setup. Here it is a failing test doing its job. Read which test failed and why.

Practical 16: TestNG Annotations, Groups, Priorities and Reports

For the journal

Aim; the table of annotations; LifecycleTest and its output, with the order explained; ResultsPage; PortalTest; testng.xml; the alwaysRun change and why; the Maven run; the two HTML reports; the observations table; the result.

Quick revision

Order: suite, test, class, then method, test, method for each test, then back out.
@Test(priority = n): lower runs first; no priority is 0.
@Test(groups = "smoke"); a test may belong to several groups.
testng.xml: <suite>, <test>, <groups><run><include name="..."/>, <classes>.
mvn test, with -Dsurefire.suiteXmlFiles= and the file's name, runs a suite file, such as testng.xml.
Set-up and tear-down in a base class need alwaysRun = true when running by groups.
Reports: target/surefire-reports/emailable-report.html and index.html.
testng-failed.xml re-runs only the failures.

Questions you must be able to answer

1. In what order do @BeforeSuite, @BeforeTest, @BeforeClass and @BeforeMethod run? @BeforeSuite, then @BeforeTest, then @BeforeClass, then @BeforeMethod before each test method; the @After methods run in the reverse order.

2. The file lists checkResults (priority 2), logIn (priority 1) and openHomePage (no priority). In what order do they run, and why? openHomePage, logIn, checkResults. Lower priorities run first, and a test with no priority has priority 0.

3. How does testng.xml run only the smoke tests? A <groups> element inside the <test>, with <run><include name="smoke"/></run>, selects only the methods in the smoke group.

4. Why did every test fail with a NullPointerException on the first grouped run? BaseTest's @BeforeMethod belonged to no group, so TestNG skipped it when running by group, and no browser was opened. alwaysRun = true makes it run regardless of groups.

5. Why were there six test runs from five test methods? validLogin belongs to both groups, and the suite file runs the class once for each group, so it ran twice.

6. Where does TestNG put its HTML reports under Maven, and what do the two main ones show? In target/surefire-reports. emailable-report.html is a summary per test with counts, times and groups; index.html is TestNG's full report, with details of each failure.

Contents This chapter on its own page

Chapter Twenty

Practical 17: Continuous Integration with Jenkins

Syllabus topic Module 2, "Continuous Integration using Jenkins", "Tool: Jenkins + Selenium WebDriver", "Configure Jenkins to execute Selenium automation scripts automatically. Generate build reports and analyze test execution history."

Aim

To configure Jenkins to run the Selenium test suite automatically, to generate build and test reports, and to analyse the history of test executions across builds.

What you need to know before you start

Continuous integration means that every change to a project is built and tested automatically, soon after it is made, so a change that breaks something is caught while it is still fresh in the mind of whoever made it. Jenkins is the free automation server most often used for it. It runs jobs on a schedule or on events, keeps every result, and shows how the results change over time.

Word	What it means in Jenkins
Job, or project	a saved set of instructions: what to run, how, and when
Build	one run of a job, numbered #1, #2, #3
Workspace	the folder a job works in
Build step	a command the job runs
Post-build action	something done after the steps: here, reading the test results
Trigger	what starts a build without a person, such as a schedule
Console output	everything a build printed
Result	SUCCESS, a green tick; UNSTABLE, an orange mark, when tests failed; FAILURE, red, when a build step itself failed

A Freestyle project is Jenkins's simplest kind of job: a form of settings, no code. It is all this practical needs.

The job runs the same Maven command as Practical 16, with one addition. It sets maven.test.failure.ignore to true, a Surefire setting from its documentation: with it, a failing test no longer makes Maven stop with BUILD FAILURE, so Jenkins goes on to read the results, and the JUnit plugin, as its documentation says, marks the build unstable when it finds at least one test failure. Run in the lab without it, the same failing test made Jenkins report "Build step 'Execute shell' marked build as failure" and a red FAILURE, exactly what a build that could not even compile would show. UNSTABLE says "the build worked and a test found a problem", which is the truth.

Installing Jenkins

Java 21 or 25. Jenkins's own support policy says so for every release since 2.555.1 of April 2026. The lab ran Jenkins 2.580.1 on Java 21.
The war file. Download jenkins.war, the long-term-support release, from jenkins.io, and start it from a terminal in your own user account:

java -jar jenkins.war --httpPort=8090

Jenkins's documentation gives --httpPort for choosing the port. Its default, 8080, is the Practice Portal's, so this book uses 8090. Running it in your own account means the builds use the same Java, Maven and browsers as your own terminal.

The setup wizard. Open http://localhost:8090. Jenkins asks for the password it generated, in a file named initialAdminPassword whose location the page shows; then offers to install the suggested plugins; then asks you to create the first administrator user.
The JUnit plugin reads test results. If Publish JUnit test result report is missing from the Post-build Actions list later, install JUnit from Manage Jenkins, Plugins.

Practical 17: Continuous Integration with Jenkins

The lab's Jenkins skips the wizard and has the JUnit plugin installed already. It also has no users, so the commands below need no password. On your Jenkins, with a user, add -auth yourname:yourtoken to each jenkins-cli.jar command and --user yourname:yourtoken to each curl, with an API token made on your user's Security page; Jenkins's documentation describes both.

A second suite file

smoke.xml runs only the smoke group of Practical 16's tests, for a quick check; testng.xml runs everything:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Smoke">
  <test name="Smoke">
    <groups>
      <run>
        <include name="smoke"/>
      </run>
    </groups>
    <classes>
      <class name="tests.PortalTest"/>
    </classes>
  </test>
</suite>

Creating the job

In Jenkins: New Item, name it stqa-tests, choose Freestyle project, and click OK. Then fill in the form:

Section	Setting	Value
General	Description	Runs the Practice Portal's Selenium tests with Maven and TestNG.
General	This project is parameterized, Add Parameter, Choice Parameter	Name `SUITE`; Choices `testng.xml` and `smoke.xml`, one per line
General, Advanced	Use custom workspace, Directory	your project folder: `/home/student/stqa-practicals` in the lab
Triggers	Build periodically, Schedule	`H/15`
Build Steps	Add build step, Execute shell	`mvn test -Dsurefire.suiteXmlFiles=$SUITE -Dmaven.test.failure.ignore=true`
Post-build Actions	Publish JUnit test result report, Test report XMLs	`target/surefire-reports/TEST-*.xml`

On Windows the build step is Execute Windows batch command, and the parameter is written %SUITE%: Jenkins's help says each parameter becomes an environment variable, ${PARAMETER_NAME}, or %PARAMETER_NAME% on Windows. The custom workspace is your Eclipse project's folder.

The top of the job's form: the description and the SUITE choice parameter

Figure 20.1 General: the description, and SUITE with its two choices. The word Edited beside Advanced shows a setting in there has been changed: the custom workspace.

Triggers with Build periodically set to H/15, and the Execute shell build step

Figure 20.2 Under the schedule, Jenkins shows when the job would last have run and would next run, here 7:31 and 7:46 PM: fifteen minutes apart, at the minutes it chose for this job's name.

The Publish JUnit test result report action and its Test report XMLs pattern

Figure 20.3 The post-build action that reads Surefire's XML results after every build.

Save. The job's page now offers Build with Parameters instead of Build Now, which Jenkins's help says happens to every parameterized job.

The same job as a file

Jenkins keeps every job as a file named config.xml. The form above saves the settings below, among many defaults it fills in itself. The lab creates the job from this file with Jenkins's command-line tool, which does what the form does, and which Jenkins itself hands out at /jnlpJars/jenkins-cli.jar:

<?xml version="1.1" encoding="UTF-8"?>
<project>
  <description>Runs the Practice Portal's Selenium tests with Maven and TestNG.</description>
  <properties>
    <hudson.model.ParametersDefinitionProperty>
      <parameterDefinitions>
        <hudson.model.ChoiceParameterDefinition>
          <name>SUITE</name>
          <description>The TestNG suite file to run</description>
          <choices>
            <string>testng.xml</string>
            <string>smoke.xml</string>
          </choices>
        </hudson.model.ChoiceParameterDefinition>
      </parameterDefinitions>
    </hudson.model.ParametersDefinitionProperty>
  </properties>
  <scm class="hudson.scm.NullSCM"/>
  <triggers>
    <hudson.triggers.TimerTrigger>
      <spec>H/15 * * * *</spec>
    </hudson.triggers.TimerTrigger>
  </triggers>
  <customWorkspace>/home/student/stqa-practicals</customWorkspace>
  <builders>
    <hudson.tasks.Shell>
      <command>mvn test -Dsurefire.suiteXmlFiles=$SUITE -Dmaven.test.failure.ignore=true</command>
    </hudson.tasks.Shell>
  </builders>
  <publishers>
    <hudson.tasks.junit.JUnitResultArchiver>
      <testResults>target/surefire-reports/TEST-*.xml</testResults>
    </hudson.tasks.junit.JUnitResultArchiver>
  </publishers>
</project>

Practical 17: Continuous Integration with Jenkins

$ curl -s -o jenkins-cli.jar http://localhost:8090/jnlpJars/jenkins-cli.jar
$ java -jar jenkins-cli.jar -s http://localhost:8090/ -http create-job stqa-tests < stqa-tests.xml

Both commands print nothing when they work. -http is needed because Jenkins's command-line tool has used WebSocket by default since Jenkins 2.391, and its documentation says HTTP must then be asked for; WebSocket also needs the Jenkins URL set, which the wizard does on your Jenkins and the lab's skipped.

Build 1: the smoke suite

In the browser: Build with Parameters, choose smoke.xml, Build, then open the build and its Console Output. The lab starts the same build from the command line, which waits for it (-s) and prints its console output (-v):

$ java -jar jenkins-cli.jar -s http://localhost:8090/ -http build stqa-tests -p SUITE=smoke.xml -s -v
Started stqa-tests #1
Started from command line by anonymous
Running as SYSTEM
Building in workspace /home/student/stqa-practicals
[stqa-practicals] $ /bin/sh -xe /tmp/jenkins1300623275929803492.sh
+ mvn test -Dsurefire.suiteXmlFiles=smoke.xml -Dmaven.test.failure.ignore=true
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------< stqa:stqa-practicals >------------------------
[INFO] Building stqa-practicals 1.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/main/resources
[INFO]
[INFO] --- compiler:3.16.0:compile (default-compile) @ stqa-practicals ---
[INFO] Recompiling the module because of changed source code.
[INFO] Compiling 5 source files with javac [debug release 17] to target/classes
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/test/resources
[INFO]
[INFO] --- compiler:3.16.0:testCompile (default-testCompile) @ stqa-practicals ---
[INFO] Recompiling the module because of changed dependency.
[INFO] Compiling 6 source files with javac [debug release 17] to target/test-classes
[INFO]
[INFO] --- surefire:3.5.6:test (default-test) @ stqa-practicals ---
[INFO] Using configured provider org.apache.maven.surefire.junit4.JUnit4Provider
[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
Sep 30, 2026 7:29:35 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Sep 30, 2026 7:29:47 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 53.02 s -- in TestSuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:04 min
[INFO] Finished at: 2026-09-30T19:29:50+05:30
[INFO] ------------------------------------------------------------------------
Recording test results
[Checks API] No suitable checks publisher found.
Finished: SUCCESS
Completed stqa-tests #1 : SUCCESS

Practical 17: Continuous Integration with Jenkins

The console output, part by part:

"Started from command line by anonymous" is the cause; a build from the browser says "Started by user" and your name. "Running as SYSTEM" and "Building in workspace" are Jenkins stating who runs the build and where: the custom workspace, the project folder.
The line starting + mvn test is the build step, with $SUITE already replaced by the value chosen, smoke.xml. Jenkins runs the step as a small shell script, whose name it prints first.
The Maven run is Practical 16's: the smoke suite's two tests passed. The CDP warning appears once per browser, as before.
"Recording test results" is the post-build action reading target/surefire-reports/TEST-*.xml. "No suitable checks publisher found" is the JUnit plugin saying it has no code-hosting site to report to, which is right for this job.
"Finished: SUCCESS".

Build 2: the whole suite

This time the command does not print the console output (no -v), only how the build ended:

$ java -jar jenkins-cli.jar -s http://localhost:8090/ -http build stqa-tests -p SUITE=testng.xml -s
Started stqa-tests #2
Completed stqa-tests #2 : UNSTABLE

UNSTABLE: Maven built the project and ran all six tests, and one failed, the Marks sort of Practicals 12 and 16. The command-line tool also exits with a non-zero code when a build does not succeed.

Build 3: started by the schedule

H/15 means every fifteen minutes. The five fields are minute, hour, day of the month, month and day of the week, and * means every value. H is Jenkins's own addition: its help says it stands for a hash of the job's name, so that a hundred jobs scheduled "every fifteen minutes" do not all start at the same second, yet each job's minutes stay fixed. For stqa-tests Jenkins chose minutes 1, 16, 31 and 46, as the preview under the schedule showed.

Nobody waits a quarter of an hour in a practical, so the lab changed the schedule to , every minute, waited until build #3 appeared, and put H/15 * straight back. In the browser that is: Configure, change the Schedule, Save, wait a minute, change it back.

$ sed -i 's|H/15 \* \* \* \*|* * * * *|' stqa-tests.xml
$ java -jar jenkins-cli.jar -s http://localhost:8090/ -http update-job stqa-tests < stqa-tests.xml
$ until curl -sf http://localhost:8090/job/stqa-tests/3/api/json > /dev/null; do sleep 5; done
$ sed -i 's|<spec>\* \* \* \* \*</spec>|<spec>H/15 * * * *</spec>|' stqa-tests.xml
$ java -jar jenkins-cli.jar -s http://localhost:8090/ -http update-job stqa-tests < stqa-tests.xml
$ until curl -s http://localhost:8090/job/stqa-tests/3/api/json | grep -q '"building":false'; do sleep 5; done
$ curl -sg "http://localhost:8090/job/stqa-tests/3/api/json?tree=result,actions[causes[shortDescription]]&pretty=true"
{
  "_class" : "hudson.model.FreeStyleBuild",
  "actions" : [
    {
      "_class" : "hudson.model.CauseAction",
      "causes" : [
        {
          "_class" : "hudson.triggers.TimerTrigger$TimerTriggerCause",
          "shortDescription" : "Started by timer"
        }
      ]
    },
    {
      "_class" : "hudson.model.ParametersAction"
    },
    {
      "_class" : "hudson.tasks.junit.TestResultAction"
    },
    {

    },
    {
      "_class" : "org.jenkinsci.plugins.displayurlapi.actions.RunDisplayAction"
    }
  ],
  "result" : "UNSTABLE"
}

Practical 17: Continuous Integration with Jenkins

The until lines wait: the first until build #3 exists, the second until it has finished. The last command asks Jenkins's remote API, the JSON behind every page, for the build's result and its cause: "Started by timer". Nobody started it. It ran testng.xml, because Jenkins's help says a build started automatically uses each parameter's default, and a choice parameter's default is the value on its first line.

The history

The job's page draws the history for you:

The job page with three builds and the Test Result Trend chart

Figure 20.4 The job page after three builds: #1 green, #2 and #3 unstable. The Test Result Trend chart shows 2 passed in #1, and 5 passed with 1 failed in #2 and #3.

The same history, as data from the remote API, is what the analysis below reads:

$ curl -sg "http://localhost:8090/job/stqa-tests/api/json?tree=builds[number,result]&pretty=true"
{
  "_class" : "hudson.model.FreeStyleProject",
  "builds" : [
    {
      "_class" : "hudson.model.FreeStyleBuild",
      "number" : 3,
      "result" : "UNSTABLE"
    },
    {
      "_class" : "hudson.model.FreeStyleBuild",
      "number" : 2,
      "result" : "UNSTABLE"
    },
    {
      "_class" : "hudson.model.FreeStyleBuild",
      "number" : 1,
      "result" : "SUCCESS"
    }
  ]
}
$ curl -sg "http://localhost:8090/job/stqa-tests/3/testReport/api/json?tree=passCount,failCount,skipCount,suites[cases[className,name,status,age]]&pretty=true"
{
  "_class" : "hudson.tasks.junit.TestResult",
  "failCount" : 1,
  "passCount" : 5,
  "skipCount" : 0,
  "suites" : [
    {
      "cases" : [
        {
          "age" : 0,
          "className" : "tests.PortalTest",
          "name" : "loginPageOpens",
          "status" : "PASSED"
        },
        {
          "age" : 0,
          "className" : "tests.PortalTest",
          "name" : "validLogin",
          "status" : "PASSED"
        },
        {
          "age" : 0,
          "className" : "tests.PortalTest",
          "name" : "validLogin",
          "status" : "PASSED"
        },
        {
          "age" : 0,
          "className" : "tests.PortalTest",
          "name" : "wrongPasswordIsRefused",
          "status" : "PASSED"
        },
        {
          "age" : 0,
          "className" : "tests.PortalTest",
          "name" : "filterByPune",
          "status" : "PASSED"
        },
        {
          "age" : 2,
          "className" : "tests.PortalTest",
          "name" : "marksSortLowestFirst",
          "status" : "FAILED"
        }
      ]
    }
  ]
}

Build #3's test report, in the browser, and the history of its one failing test:

Build 3's tests: 1 failed, 5 passed, marksSortLowestFirst with Age 2

Figure 20.5 Build #3's test result: six tests, one failed. Age 2 means marksSortLowestFirst has failed in two builds in a row.

The history of marksSortLowestFirst: failed in builds 2 and 3

Figure 20.6 The test's own history page: it failed in both builds that ran it, #2 and #3.

Analysing the test execution history

Build #1 ran the smoke suite: two tests, both passed.
Build #2 ran the whole suite for the first time: six test runs, five passed, and marksSortLowestFirst failed. It did not fail in #1 because #1 never ran it.
Build #3, started by the timer, failed the same test with the same result. The report gives it age 2, and its history page shows it failing in both builds that ran it.

Practical 17: Continuous Integration with Jenkins

What the pattern means: a test that fails in every build since it was first run, in the same way, is a steady defect, here the Marks column sorting as text, found in Practical 12. A flaky test looks different: it passes and fails in turn with nothing changed. A test that passed for many builds and fails from one build onwards points to a change made just before that build, which is the question continuous integration exists to answer quickly.

Observations

Build	Started by	Suite	Test runs	Passed	Failed	Result
#1	command line	smoke.xml	2	2	0	SUCCESS
#2	command line	testng.xml	6	5	1	UNSTABLE
#3	timer	testng.xml, the default	6	5	1	UNSTABLE

Result

Jenkins was configured with a parameterized Freestyle job that runs the Selenium suite through Maven in the project's folder, publishes the JUnit results after every build, and starts by itself on a schedule. Three builds were run, the third started by the timer. Jenkins's reports, the console output, the test result pages and the Test Result Trend, showed the smoke suite passing and the whole suite failing one test in every build that ran it; the history identified the failure as a steady defect, not a flaky test.

Where marks are lost

Leaving out the maven.test.failure.ignore setting. A failing test then turns the build red, FAILURE, the same as a build that cannot compile.

The wrong folder for the results. The Test report XMLs pattern is relative to the workspace; without the custom workspace, Jenkins looks in its own empty folder and finds nothing.

left in place. A build every minute buries the useful history. Use H/15 or a daily H H .

$SUITE in a Windows batch step. Windows writes it %SUITE%.

Reading only the latest build. The point of Jenkins is the history: say when a failure started and whether it is steady.

A screenshot-free journal. The job's form, a console output, a test result page and the trend chart are the build reports MU asks for.

For the journal

Aim; the words table; how Jenkins was installed and started; the job's settings table; the console output of a build; the results of the three builds; the trend chart and a test's history; the analysis; the observations table; the result.

Quick revision

CI: every change built and tested automatically. Jenkins runs jobs and keeps the history.
Freestyle job: parameters, triggers, build steps, post-build actions.
Build step: mvn test with two settings, each written -Dname=value: surefire.suiteXmlFiles set to $SUITE, and maven.test.failure.ignore set to true.
Post-build: Publish JUnit test result report, target/surefire-reports/TEST-*.xml.
Results: SUCCESS, UNSTABLE (tests failed), FAILURE (a step failed).
Schedule: minute, hour, day of month, month, day of week; H is a hash of the job name.
An automatic build uses each parameter's default: a choice's first line.
History: a failure's age, its history page and the trend chart show whether it is steady or flaky.

Practical 17: Continuous Integration with Jenkins

Questions you must be able to answer

1. What is continuous integration? Building and testing a project automatically every time it changes, so that a change that breaks something is found at once.

2. Why does the build step include -Dmaven.test.failure.ignore=true? So that a failing test does not stop Maven with an error; Jenkins then records the results and marks the build UNSTABLE instead of FAILURE.

3. What does H/15 mean? Every fifteen minutes, at minutes chosen from a hash of the job's name: minutes 1, 16, 31 and 46 for this job.

4. Build #3 ran testng.xml though nobody chose it. Why? It was started by the timer, and an automatic build uses each parameter's default; a choice parameter's default is its first choice, testng.xml.

5. What is the difference between UNSTABLE and FAILURE? UNSTABLE: the build ran and at least one test failed. FAILURE: a build step itself failed, for example Maven could not compile.

6. marksSortLowestFirst has age 2 in build #3. What does that tell you? It has failed in two builds in a row, every build that ran it, so it is a steady failure, a real defect, not a flaky test.

Contents This chapter on its own page

Chapter Twenty-One

Practical 18: Cross-Browser Testing with Selenium Grid

Syllabus topic Module 2, "Cross-Browser Testing Using Selenium Grid", "Tool: Selenium Grid", "Execute automation scripts across multiple browsers (Chrome, Firefox, Edge) using Selenium Grid. Compare execution time and compatibility issues."

Aim

To run the same automation scripts on Chrome, Firefox and Edge through Selenium Grid, and to compare their execution times and the compatibility issues found.

What you need to know before you start

Cross-browser testing runs the same tests on several browsers. Chrome, Firefox and Edge are separate programs with their own drivers, and a page, or a test, that works in one can behave differently in another. The only way to know is to run the tests in each.

Selenium Grid does the running, as in Practical 4: a hub receives each request for a browser and hands it to a node that offers that browser. A script asks for a browser with an options object, ChromeOptions, FirefoxOptions or EdgeOptions, and RemoteWebDriver sends the request to the hub's address.

TestNG's @Parameters hands a value from the suite file to a method. Here the suite file has three <test> blocks, each with a different browser parameter, so one test class runs three times, once per browser.

Two measures, the ones MU names:

Execution time: how long each browser takes to start and to run the same tests.
Compatibility issues: anything that works in one browser and not in another, whether the fault is the page's, the test's, or the setup's.

Starting the Grid

The hub and the two nodes of Practical 4, started in the background with their logs sent to files, and one addition to the hub's command:

$ nohup java -jar selenium-server-4.49.0.jar hub --reject-unsupported-caps true > hub.log 2>&1 &
$ until grep -q "Started Selenium Hub" hub.log; do sleep 1; done
$ nohup java -jar selenium-server-4.49.0.jar node --port 5555 --driver-implementation chrome > chrome-node.log 2>&1 &
$ nohup java -jar selenium-server-4.49.0.jar node --port 6666 --driver-implementation firefox > firefox-node.log 2>&1 &
$ until grep -q "Node has been added" chrome-node.log && grep -q "Node has been added" firefox-node.log; do sleep 1; done
$ grep -h "Adding" chrome-node.log firefox-node.log | cut -d ' ' -f 5-
Adding Chrome for {"browserName": "chrome","platformName": "linux"} 2 times
Adding Firefox for {"browserName": "firefox","platformName": "linux"} 2 times

--reject-unsupported-caps true. The Grid's documentation describes it as letting the hub reject at once a request for something no node offers. Without it, a request waits in the hub's queue for a node that may never come, up to --session-request-timeout, 300 seconds by default. The lab tried that: asked for Edge with the flag left out, the program waited 181 seconds and then gave up with "Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure", a message that points in the wrong direction.

The nodes offer Chrome and Firefox, two sessions each, and no Edge. Edge is not installed in the lab, and cannot be: the lab runs Linux on an ARM64 processor, and Microsoft's own Linux repository for Edge, checked on 30 September 2026, has packages only for amd64, the usual PC processor; its arm64 list is empty. So the Edge part of this practical shows what a Grid does when asked for a browser it lacks, and how to run it where Edge exists:

Practical 18: Cross-Browser Testing with Selenium Grid

On a Windows PC with Edge, start a third node for it. The Grid's documentation writes the browser as edge: java -jar selenium-server-4.49.0.jar node --port 7777 --driver-implementation edge.
Do not trust a node's list without trying it. Started without --driver-implementation, a node in the lab looked for browsers itself and logged "Adding Edge for {"browserName": "MicrosoftEdge", ...} 2 times", although there is no Edge on that machine; a session asked of it failed with Cannot run program "". Only a session that starts proves a browser is there.

The cross-browser test

CrossBrowserTest uses the page classes of Practicals 15 and 16. What is new:

package tests;

import static org.testng.Assert.assertEquals;

import java.net.MalformedURLException;
import java.net.URI;

import org.openqa.selenium.By;
import org.openqa.selenium.Capabilities;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.edge.EdgeOptions;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.Color;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Parameters;
import org.testng.annotations.Test;

import pages.LoginPage;
import pages.ResultsPage;

// The same three tests on whichever browser the suite file names, through the Grid.
public class CrossBrowserTest {
    private WebDriver driver;
    private String browser;
    private long started;

    @Parameters("browser")
    @BeforeClass(alwaysRun = true)
    public void startBrowser(String browser) throws MalformedURLException {
        this.browser = browser;
        Capabilities options = switch (browser) {
            case "chrome" -> new ChromeOptions();
            case "firefox" -> new FirefoxOptions();
            case "MicrosoftEdge" -> new EdgeOptions();
            default -> throw new IllegalArgumentException("Unknown browser: " + browser);
        };
        started = System.currentTimeMillis();
        driver = new RemoteWebDriver(URI.create("http://localhost:4444").toURL(), options);
        System.out.println(browser + ": session started in " + (System.currentTimeMillis() - started) + " ms");
    }

    @Test
    public void validLogin() {
        assertEquals(LoginPage.open(driver).loginAs("asha", "Asha@2026").welcomeText(), "Welcome, Asha Patil");
    }

    @Test
    public void filterByPune() {
        assertEquals(ResultsPage.open(driver).filterBy("Pune").countText(), "Showing 3 of 15 students");
    }

    @Test
    public void errorMessageIsRed() {
        LoginPage.open(driver).loginExpectingError("asha", "wrong");
        String reported = driver.findElement(By.id("error")).getCssValue("color");
        System.out.println(browser + ": the error's colour is reported as " + reported);
        // Browsers write the same colour differently; Color reads either form.
        assertEquals(Color.fromString(reported).asHex(), "#b00020");
    }

    @AfterClass(alwaysRun = true)
    public void stopBrowser() {
        if (driver != null) {
            driver.quit();
        }
        System.out.println(browser + ": finished in " + (System.currentTimeMillis() - started) + " ms");
    }
}

@Parameters("browser") fills startBrowser's argument from the suite file, and the switch turns the name into the options object for that browser.
@BeforeClass and @AfterClass, not per method: one browser session for the class's three tests, so the times printed are one session's start and one class's whole run.
errorMessageIsRed reads the colour of the login error with getCssValue("color") and prints it exactly as the browser reported it. The practice site's style sheet sets it to #b00020. Selenium's Color class reads a colour in any of the forms browsers use, and asHex() gives it back in one form, so the comparison works in every browser.

Practical 18: Cross-Browser Testing with Selenium Grid

The suite file runs the class once per browser:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Cross-browser">
  <test name="Chrome">
    <parameter name="browser" value="chrome"/>
    <classes>
      <class name="tests.CrossBrowserTest"/>
    </classes>
  </test>
  <test name="Firefox">
    <parameter name="browser" value="firefox"/>
    <classes>
      <class name="tests.CrossBrowserTest"/>
    </classes>
  </test>
  <test name="Edge">
    <parameter name="browser" value="MicrosoftEdge"/>
    <classes>
      <class name="tests.CrossBrowserTest"/>
    </classes>
  </test>
</suite>

$ mvn test -Dsurefire.suiteXmlFiles=crossbrowser.xml
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------< stqa:stqa-practicals >------------------------
[INFO] Building stqa-practicals 1.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/main/resources
[INFO]
[INFO] --- compiler:3.16.0:compile (default-compile) @ stqa-practicals ---
[INFO] Recompiling the module because of added or removed source files.
[INFO] Compiling 5 source files with javac [debug release 17] to target/classes
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/test/resources
[INFO]
[INFO] --- compiler:3.16.0:testCompile (default-testCompile) @ stqa-practicals ---
[INFO] Recompiling the module because of changed dependency.
[INFO] Compiling 7 source files with javac [debug release 17] to target/test-classes
[INFO]
[INFO] --- surefire:3.5.6:test (default-test) @ stqa-practicals ---
[INFO] Using configured provider org.apache.maven.surefire.junit4.JUnit4Provider
[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
Sep 30, 2026 7:51:56 PM org.openqa.selenium.remote.tracing.opentelemetry.OpenTelemetryTracer createTracer
INFO: Using OpenTelemetry for tracing
chrome: session started in 12373 ms
chrome: the error's colour is reported as rgba(176, 0, 32, 1)
chrome: finished in 17674 ms
firefox: session started in 15818 ms
firefox: the error's colour is reported as rgb(176, 0, 32)
firefox: finished in 22648 ms
MicrosoftEdge: finished in 118 ms
[ERROR] Tests run: 10, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 41.83 s <<< FAILURE! -- in TestSuite
[ERROR] tests.CrossBrowserTest.startBrowser[MicrosoftEdge](0) -- Time elapsed: 3.690 s <<< FAILURE!
org.openqa.selenium.SessionNotCreatedException:
Could not start a new session. Response code 500. Message: Could not start a new session. No nodes support the capabilities in the request
Host info: host: 'labpc', ip: '172.18.0.3'
Build info: version: '4.49.0', revision: 'bedb24f bedb24f6f703876e7e4c5d506f0d697eb358c720'
System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '6.8.0-117-generic', java.version: '21.0.12.1'
Driver info: driver.version: unknown
Build info: version: '4.49.0', revision: 'bedb24f bedb24f6f703876e7e4c5d506f0d697eb358c720'
System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '6.8.0-117-generic', java.version: '21.0.12.1'
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: MicrosoftEdge, ms:edgeOptions: {args: [], extensions: []}, se:remoteUrl: http://localhost:4444}]}]
Capabilities {browserName: MicrosoftEdge, ms:edgeOptions: {args: [], extensions: []}}
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:115)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:76)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:62)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:171)
	at org.openqa.selenium.remote.TracedCommandExecutor.execute(TracedCommandExecutor.java:51)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:653)
	at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:279)
	at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:213)
	at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:178)
	at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:174)
	at tests.CrossBrowserTest.startBrowser(CrossBrowserTest.java:41)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:141)
	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:71)
	at org.testng.internal.invokers.ConfigInvoker.invokeConfigurationMethod(ConfigInvoker.java:400)
	at org.testng.internal.invokers.ConfigInvoker.invokeConfigurations(ConfigInvoker.java:333)
	at org.testng.internal.invokers.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:188)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at org.testng.TestRunner.privateRun(TestRunner.java:744)
	at org.testng.TestRunner.run(TestRunner.java:616)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:421)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:413)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:373)
	at org.testng.SuiteRunner.run(SuiteRunner.java:312)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1274)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1208)
	at org.testng.TestNG.runSuites(TestNG.java:1112)
	at org.testng.TestNG.run(TestNG.java:1079)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:308)
	at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:71)
	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:113)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   CrossBrowserTest.startBrowser:41 » SessionNotCreated Could not start a new session. Response code 500. Message: Could not start a new session. No nodes support the capabilities in the request
Host info: host: 'labpc', ip: '172.18.0.3'
Build info: version: '4.49.0', revision: 'bedb24f bedb24f6f703876e7e4c5d506f0d697eb358c720'
System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '6.8.0-117-generic', java.version: '21.0.12.1'
Driver info: driver.version: unknown
Build info: version: '4.49.0', revision: 'bedb24f bedb24f6f703876e7e4c5d506f0d697eb358c720'
System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '6.8.0-117-generic', java.version: '21.0.12.1'
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: MicrosoftEdge, ms:edgeOptions: {args: [], extensions: []}, se:remoteUrl: http://localhost:4444}]}]
Capabilities {browserName: MicrosoftEdge, ms:edgeOptions: {args: [], extensions: []}}
[INFO]
[ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  47.799 s
[INFO] Finished at: 2026-09-30T19:52:37+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.6:test (default-test) on project stqa-practicals: There are test failures.
[ERROR]
[ERROR] See /home/student/stqa-practicals/target/surefire-reports for the individual test results.
[ERROR] See dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Practical 18: Cross-Browser Testing with Selenium Grid

What the run shows:

"Using OpenTelemetry for tracing" is RemoteWebDriver naming the library it uses to trace requests. It is not a problem.
Chrome and Firefox passed all three tests. Each printed when its session started, the colour it reported, and when it finished.
The same colour, two spellings: Chrome reported the error's colour as rgba(176, 0, 32, 1), Firefox as rgb(176, 0, 32). Both are #b00020, and the test passed in both because it compared them through Color. A test that compared the text rgba(176, 0, 32, 1) would pass in Chrome and fail in Firefox: a compatibility issue in the test, not in the page.
Edge never started. startBrowser failed with SessionNotCreatedException: "No nodes support the capabilities in the request", within seconds, because of --reject-unsupported-caps. TestNG then skipped Edge's three tests: when a @BeforeClass fails, the tests that depend on it are not run, and TestNG reports them as skipped, not failed. The stack trace under the error is Surefire printing where the exception came from.
Surefire's two summary lines count differently. TestNG's own report, next, gives the plain count per browser.

Practical 18: Cross-Browser Testing with Selenium Grid

The report, and the times

TestNG's emailable report, from a later run of the same suite in the lab:

TestNG's emailable report: Chrome 3 passed, Firefox 3 passed, Edge 3 skipped

Figure 21.1 Time per browser in this run: Chrome 11,543 ms and Firefox 14,104 ms for the same three tests, and 193 ms for Edge, which only failed to start. Below, each method's time, Edge's failed configuration method startBrowser, and its three skipped tests.

Comparing the execution times:

Measure	Chrome	Firefox	Edge
Session start, the run printed above	12,373 ms	15,818 ms	refused
Whole class, the run printed above	17,674 ms	22,648 ms	118 ms, only the refusal
Whole test, the report's run	11,543 ms	14,104 ms	193 ms, only the refusal

The same browser took different times in the two runs. Timings depend on everything else the machine is doing, and the lab's machine was busy. One run proves little; compare several runs, or their average.
In both runs Firefox took longer than Chrome, and most of the difference was in starting the session: once started, each test took one to two seconds in either browser, as the report's method times show.
Edge's time measures only the refusal, which is why it is by far the smallest number and means nothing about Edge's speed.

The compatibility issues found

Issue	Where it showed	Whose it is	What to do
Edge cannot run on this machine	Edge's `startBrowser` failed: no node offers Edge	the platform: no Edge for Linux on ARM64	run the Edge node on a machine with Edge, such as a Windows PC
The same colour written two ways	`rgba(176, 0, 32, 1)` in Chrome, `rgb(176, 0, 32)` in Firefox	the test, if it compares text	compare through Selenium's `Color`
A node listing a browser it cannot start	the self-configured node's "Adding Edge ... 2 times"	the setup	name the browsers with `--driver-implementation`, and trust only a session that starts
The native file dialog	Practical 13: the lab's Chromium never passed the chosen file to the page; Firefox did	the lab's Chromium	upload with sendKeys, which needs no dialog

Observations

Browser	validLogin	filterByPune	errorMessageIsRed	Colour reported
Chrome	Pass	Pass	Pass	rgba(176, 0, 32, 1)
Firefox	Pass	Pass	Pass	rgb(176, 0, 32)
Edge	Skipped	Skipped	Skipped	never reached: the session was refused

Result

The same TestNG test class was run through Selenium Grid on Chrome and Firefox, where all three tests passed, and on Edge, which the Grid refused at once because no node offers it; Edge cannot be installed on the lab's ARM64 Linux. Execution times were compared: Firefox took longer than Chrome in both runs, mostly in starting its session, and the same browser's time changed between runs. The compatibility issues found were the missing Edge, one colour reported in two different forms, and a node advertising a browser it could not start.

Practical 18: Cross-Browser Testing with Selenium Grid

Where marks are lost

Hard-coding one browser. Pass the browser in from the suite file, so one class serves all three.

Comparing CSS values as text. Browsers spell the same colour differently. Use Color.

Reading a missing browser as a slow one. Without --reject-unsupported-caps, a request for a browser no node offers waits in the queue and ends in a misleading error.

Timing one run and calling the result a fact. Run several times; state the machine and what else it was doing.

Counting skipped tests as failures. A failed set-up skips the tests after it; the fault is the set-up.

Trusting a node's list of browsers. Only a session that starts proves the browser is there.

For the journal

Aim; the hub and node commands and the Adding lines; the test class and the suite file; the run's output; the report; the table of times with the machine it was measured on; the table of compatibility issues; the observations table; the result.

Quick revision

One class, three <test> blocks, a browser parameter each: @Parameters("browser").
RemoteWebDriver + ChromeOptions, FirefoxOptions or EdgeOptions, sent to http://localhost:4444.
Hub: --reject-unsupported-caps true refuses at once what no node offers.
Node for one browser: --driver-implementation chrome, firefox or edge.
getCssValue("color") differs by browser; the same colour passed through Color.fromString and asHex() does not.
A failed @BeforeClass skips its tests.
Times vary from run to run; compare several runs.
Edge for Linux is published for amd64 only: no Edge on the lab's ARM64 machine.

Questions you must be able to answer

1. How does one test class run on three browsers? The suite file has three <test> blocks, each passing a different browser name through @Parameters; the class builds the options object for that browser and asks the Grid for it.

2. Why did Edge's tests show as skipped rather than failed? Their @BeforeClass, which starts the browser, failed, and TestNG skips the tests of a class whose set-up failed.

3. What does --reject-unsupported-caps true do? It makes the hub refuse at once a request for a browser no node offers, instead of queueing it until the request times out.

4. Chrome and Firefox reported the error's colour differently. Is that a defect in the page? No. It is the same colour written two ways; a test that compares the text would be wrong. Comparing through Selenium's Color class works in both.

Practical 18: Cross-Browser Testing with Selenium Grid

5. Which browser was faster in the lab, and how sure can you be? Chrome, in both runs, mostly in starting its session. Two runs on one busy machine show a tendency, not a rule; several runs, or an average, would be needed.

6. Why could the lab not run Edge? The lab runs Linux on an ARM64 processor, and Microsoft publishes Edge for Linux only for amd64 processors.

Contents This chapter on its own page

Chapter Twenty-Two

Practical 19: Load Testing with Apache JMeter

Syllabus topic Module 2, "Load Testing Using Apache JMeter", "Tool: Apache JMeter", "Design and execute load testing scenarios for a web application. Configure thread groups, ramp-up time, and loop count. Analyze response time, throughput, and error percentage using graphical reports."

Aim

To design and run load testing scenarios for a web application in Apache JMeter, configuring the thread group, ramp-up time and loop count, and to analyse response time, throughput and error percentage with JMeter's graphical reports.

What you need to know before you start

Load testing measures how an application behaves when many people use it at once: how long each person waits, how many requests it can serve each second, and how many requests fail. The aim is to find where it starts to slow down or fail, before real users find it.

Apache JMeter does it by playing many users at once. Its parts, in JMeter's own words where its manual defines them:

Part	What it is
Test plan	everything JMeter will do, saved as a `.jmx` file
Thread group	a group of virtual users: each thread is one user
Number of threads	how many users
Ramp-up period	how long JMeter takes to start all the threads. JMeter's manual: with 10 threads and a ramp-up of 100 seconds, each thread starts 10 seconds after the one before
Loop count	how many times each user repeats the steps
Sampler	one request, here an HTTP Request; each result is a sample
Config element	settings the samplers share: HTTP Request Defaults, HTTP Cookie Manager
Assertion	a check on each sample; a sample that fails an assertion counts as an error

The measures, as the manual's glossary defines them:

Response time, JMeter's elapsed time: from just before the request is sent to just after the whole response has arrived.
Throughput: requests per unit of time, counted from the start of the first sample to the end of the last.
Error %: the share of samples that failed: an HTTP error, or an assertion that did not hold.
90th percentile, "90% Line": the value below which 90% of the samples fall. The average hides the slow requests; the percentile shows them.

Two modes. JMeter's manual says GUI mode should be used only for creating the test script, and that CLI mode, the command line, must be used for load testing: the GUI itself uses the machine the test is measuring. JMeter prints the same warning in its terminal every time the GUI starts.

Installing JMeter. Download the binary zip from jmeter.apache.org and unzip it. JMeter's manual says it needs Java 8 or newer. Start the GUI with bin\jmeter.bat on Windows or bin/jmeter elsewhere; the lab ran JMeter 5.6.3 on Java 21.

The application under load

The Practice Portal, started exactly as Practical 1 started it, with PHP's built-in server:

$ nohup php -S localhost:8080 router.php > php.log 2>&1 &
$ until curl -s -o /dev/null http://localhost:8080/; do sleep 1; done
$ curl -s -o /dev/null -w "%{http_code} %{time_total}s\n" http://localhost:8080/results.php
200 0.003990s
$ curl -s -o /dev/null -w "%{http_code} %{time_total}s\n" -d "username=asha&password=Asha@2026" http://localhost:8080/login.php
302 0.304230s

Practical 19: Load Testing with Apache JMeter

PHP's documentation says its built-in server runs a single process that handles one request at a time. Several processes can be asked for with PHP_CLI_SERVER_WORKERS, but not on Windows, so the lab, which usually runs the portal with four, runs it here with one, as a student's is. Everything this chapter measures is that one process.

The two curl lines time one request each, with nobody else using the site: the results page took about 4 milliseconds, and a login about 300. A login checks the password against the stored hash in inc/users.php, and that check is the slow part. Under load, every request waits for the ones in front of it, and the logins make the queue long.

The test plan

Build it in the GUI. Right-click the Test Plan, Add, Threads (Users), Thread Group:

Figure 22.1 Right-click, Add: the thread group is under Threads (Users). The other entries of Add are where config elements, listeners and assertions are found.

Fill in the thread group. Instead of fixed numbers, each field holds ${__P(name,default)}, JMeter's __P function, which reads a value given on the command line with -J, or uses the default. One plan then serves both scenarios:

The Students thread group: threads, ramp-up and loop count as __P properties

Figure 22.2 Number of Threads ${__P(users,5)}, Ramp-up period ${__P(rampup,5)} and Loop Count ${__P(loops,10)}. With no -J values: 5 users, started over 5 seconds, 10 times round each.

Then add, to the thread group:

Add	Element	Settings
Config Element	HTTP Request Defaults	Protocol `http`, Server `localhost`, Port `8080`
Config Element	HTTP Cookie Manager	tick Clear cookies each iteration, so each round starts logged out
Sampler	HTTP Request, "Home page"	GET `/index.php`
Sampler	HTTP Request, "Results page"	GET `/results.php`, with a Response Assertion: the text contains `Semester Results`
Sampler	HTTP Request, "Results data"	GET `/api/results.php`, the data the results page loads
Sampler	HTTP Request, "Log in"	POST `/login.php`, parameters `username` asha and `password` Asha@2026, Follow Redirects unticked, with a Response Assertion: the response code equals `302`
Sampler	HTTP Request, "Dashboard"	GET `/dashboard.php`, with a Response Assertion: the text contains `Welcome, Asha Patil`
Assertions	Duration Assertion	1500 milliseconds: any sample slower than that counts as an error

The Log in request: POST /login.php with the username and password parameters

Figure 22.3 The whole plan in the tree on the left. Log in sends the form with its two parameters. Follow Redirects is off, so the login's own response, the 302 that sends the browser to the dashboard, is the sample; Dashboard is the next request.

Why the login is split into two requests: JMeter's manual says that when a request follows a redirect, the redirect and the page it leads to appear as extra samples under it, and the parent's time includes all of them. Two plain requests keep one sample per page, so every count in the reports means the same thing.

Practical 19: Load Testing with Apache JMeter

Why 1500 milliseconds: it is the frustration threshold JMeter's dashboard uses for its APDEX score by default. A page slower than that counts as a frustrated user in the report's own terms, so the plan counts it as an error.

Save the plan as portal-load.jmx. This is the file, with what the GUI wrote shortened to the settings used:

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.6.3">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Practice Portal load test">
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Students">
        <stringProp name="ThreadGroup.num_threads">${__P(users,5)}</stringProp>
        <stringProp name="ThreadGroup.ramp_time">${__P(rampup,5)}</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller">
          <stringProp name="LoopController.loops">${__P(loops,10)}</stringProp>
          <boolProp name="LoopController.continue_forever">false</boolProp>
        </elementProp>
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
      </ThreadGroup>
      <hashTree>
        <ConfigTestElement guiclass="HttpDefaultsGui" testclass="ConfigTestElement" testname="HTTP Request Defaults">
          <stringProp name="HTTPSampler.protocol">http</stringProp>
          <stringProp name="HTTPSampler.domain">localhost</stringProp>
          <stringProp name="HTTPSampler.port">8080</stringProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
        </ConfigTestElement>
        <hashTree/>
        <CookieManager guiclass="CookiePanel" testclass="CookieManager" testname="HTTP Cookie Manager">
          <collectionProp name="CookieManager.cookies"/>
          <boolProp name="CookieManager.clearEachIteration">true</boolProp>
        </CookieManager>
        <hashTree/>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Home page">
          <stringProp name="HTTPSampler.path">/index.php</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree/>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Results page">
          <stringProp name="HTTPSampler.path">/results.php</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree>
          <ResponseAssertion guiclass="AssertionGui" testclass="ResponseAssertion" testname="Page heading present">
            <collectionProp name="Asserion.test_strings">
              <stringProp name="1">Semester Results</stringProp>
            </collectionProp>
            <stringProp name="Assertion.test_field">Assertion.response_data</stringProp>
            <boolProp name="Assertion.assume_success">false</boolProp>
            <intProp name="Assertion.test_type">16</intProp>
          </ResponseAssertion>
          <hashTree/>
        </hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Results data">
          <stringProp name="HTTPSampler.path">/api/results.php</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree/>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Log in">
          <stringProp name="HTTPSampler.path">/login.php</stringProp>
          <stringProp name="HTTPSampler.method">POST</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">false</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments">
              <elementProp name="username" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">true</boolProp>
                <stringProp name="Argument.name">username</stringProp>
                <stringProp name="Argument.value">asha</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
                <boolProp name="HTTPArgument.use_equals">true</boolProp>
              </elementProp>
              <elementProp name="password" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">true</boolProp>
                <stringProp name="Argument.name">password</stringProp>
                <stringProp name="Argument.value">Asha@2026</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
                <boolProp name="HTTPArgument.use_equals">true</boolProp>
              </elementProp>
            </collectionProp>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree>
          <ResponseAssertion guiclass="AssertionGui" testclass="ResponseAssertion" testname="Sent to the dashboard">
            <collectionProp name="Asserion.test_strings">
              <stringProp name="1">302</stringProp>
            </collectionProp>
            <stringProp name="Assertion.test_field">Assertion.response_code</stringProp>
            <boolProp name="Assertion.assume_success">false</boolProp>
            <intProp name="Assertion.test_type">8</intProp>
          </ResponseAssertion>
          <hashTree/>
        </hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Dashboard">
          <stringProp name="HTTPSampler.path">/dashboard.php</stringProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree>
          <ResponseAssertion guiclass="AssertionGui" testclass="ResponseAssertion" testname="Welcome shown">
            <collectionProp name="Asserion.test_strings">
              <stringProp name="1">Welcome, Asha Patil</stringProp>
            </collectionProp>
            <stringProp name="Assertion.test_field">Assertion.response_data</stringProp>
            <boolProp name="Assertion.assume_success">false</boolProp>
            <intProp name="Assertion.test_type">16</intProp>
          </ResponseAssertion>
          <hashTree/>
        </hashTree>
        <DurationAssertion guiclass="DurationAssertionGui" testclass="DurationAssertion" testname="Answered within 1.5 seconds">
          <stringProp name="DurationAssertion.duration">1500</stringProp>
        </DurationAssertion>
        <hashTree/>
      </hashTree>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

The misspelt Asserion.test_strings is JMeter's own spelling, kept for old plans; type it as it is.

Practical 19: Load Testing with Apache JMeter

Scenario 1: light load

In a terminal, in the project folder: 5 users started over 5 seconds, 10 rounds each, 5 requests a round, so 250 samples:

$ jmeter -n -t portal-load.jmx -Jjmeter.reportgenerator.overall_granularity=2000 -l light.jtl -e -o light-report 2>&1 | tail -3
summary =    250 in 00:00:13 =   19.5/s Avg:   165 Min:     0 Max:   730 Err:     0 (0.00%)
Tidying up ...    @ 2026 Sep 30 20:21:32 IST (1790779892151)
... end of run

The options: -n runs without the GUI, -t names the plan, -l the file every sample is written to, and -e -o builds the HTML dashboard into a new folder when the test ends. The -J setting is a JMeter property from its documentation: the report generator's overall_granularity, one minute by default, is lowered to 2 seconds so the dashboard's charts have a point every two seconds; the manual warns against going below one second. 2>&1 | tail -3 joins JMeter's warnings to its normal output and keeps only the last three lines, which hold the result: the screen stays quiet until the test ends, then shows three lines like these. Without it you also see some start-up lines first, among them four warnings about "package scanning" from the logging library inside JMeter 5.6.3 itself, not from your plan. Then comes a progress line at every 30-second mark of the clock (summariser.interval, 30 seconds by default), so how many you see depends on when a run starts and how long it takes.

The summary line is the result: 250 samples in 13 seconds, 19.5 per second; average 165 ms, minimum 0, maximum 730; 0 errors. Every page came back in well under a second, and every assertion held. The last two lines say the test has ended.

Scenario 2: heavy load

The same plan with 50 users, -Jusers=50, started over the same 5 seconds: 2500 samples.

$ jmeter -n -t portal-load.jmx -Jusers=50 -Jjmeter.reportgenerator.overall_granularity=2000 -l heavy.jtl -e -o heavy-report 2>&1 | tail -3
summary =   2500 in 00:02:08 =   19.6/s Avg:  2378 Min:     1 Max:  4857 Err:  2204 (88.16%)
Tidying up ...    @ 2026 Sep 30 20:37:21 IST (1790780841080)
... end of run

2500 samples in 2 minutes 8 seconds, 19.6 per second; average 2378 ms, maximum 4857 ms; 2204 errors, 88.16%.

The graphical reports

Each run left a dashboard: open heavy-report/index.html or light-report/index.html in a browser. The figures here are from a second pair of runs made in the lab for the pictures, and their numbers are not the ones printed above. That difference is explained below, and it is a lesson in itself.

The dashboard page gives the APDEX score, the pass and fail shares, and the Statistics table: for each request, its samples, failures, error %, average, median and percentiles, and its throughput.

Practical 19: Load Testing with Apache JMeter

JMeter's dashboard for the heavy run: APDEX, the pass and fail pie, and the Statistics table

Figure 22.4 The heavy run of the figure pair: 2500 samples, 78.48% failed, average 2628 ms, 17.71 transactions a second in all. Log in is the worst request: 94.80% failed, average 3797 ms, 90th percentile 5662 ms.

Charts, Over Time, Response Times Over Time plots each request's average response time every two seconds. Heavy against light:

Response times over time under heavy load: rising in the first seconds, then between about 1.5 and 6 seconds

Figure 22.5 50 users. The lines climb while the threads start, during the 5-second ramp-up, then swing between about 1.5 and 6 seconds until the users finish. The top line, purple in the legend, is Log in.

Response times over time under light load: Log in between about 360 and 780 ms, the other pages near zero

Figure 22.6 5 users. Log in, purple, between about 360 and 780 ms; every other page close to zero on this scale.

Analysing the results

From the printed pair of runs:

Measure	Light: 5 users	Heavy: 50 users
Samples	250	2500
Throughput	19.5/s	19.6/s
Average response time	165 ms	2378 ms
Maximum response time	730 ms	4857 ms
Error %	0.00%	88.16%

Throughput did not grow with the users. Ten times the users gave the same 19.5 to 19.6 requests a second. That is the server's ceiling: one PHP process can finish only so many requests a second, however many are waiting.
So the waiting grew instead. The average response time rose from 165 ms to 2378 ms, about fourteen times. With the server at its ceiling, each extra user only lengthens the queue every request waits in.
The errors are late answers, not wrong ones. In the lab's results files every failure carried the Duration Assertion's message, "The operation lasted too long"; the pages themselves were right. At 50 users 88% of the requests were too slow, the logins worst of all, because each carries the slow password check.
The capacity limit lies between the two loads: at 5 users nothing was slow; at 50, nearly everything was. The next step in a real test would be scenarios in between, 10, 20, 30 users, to find where the error % starts to rise.

Why the figure pair differs. Its heavy run served 17.71 requests a second with 78.48% errors, against 19.6 and 88.16% above: close, but not the same. Another pair of runs in the lab went further apart: 39.08 requests a second and 27.28% errors, the same plan and the same site at twice the speed. The lab's machine is shared, and when other work was using its processors, the PHP process had less time. JMeter's own guidance begins with sizing the machine that runs the test. A load test measures the whole machine, not only the application, so results are compared only between runs made on the same, otherwise idle, machine, and every test is repeated before a number is believed. Each pair here was run back to back, and within each pair the picture is the same: the heavy load at the server's ceiling and far slower.

Practical 19: Load Testing with Apache JMeter

Observations

Scenario	Users	Ramp-up	Loops	Samples	Throughput	Average	Error %
Light	5	5 s	10	250	19.5/s	165 ms	0.00%
Heavy	50	5 s	10	2500	19.6/s	2378 ms	88.16%

Result

A load test plan was designed in JMeter with a parameterized thread group, five HTTP requests with assertions and a 1.5-second time limit, and run in CLI mode in two scenarios. With 5 users the Practice Portal answered in 165 ms on average with no errors; with 50 users its throughput stayed at about 19.6 requests a second, its ceiling on one PHP process, while the average response time rose to 2378 ms and 88.16% of the requests missed the time limit. JMeter's HTML dashboards showed the same in their statistics table and response-time charts. A second pair of runs on the busy shared machine gave different numbers with the same pattern, which is why load test results are compared only on one idle machine.

Where marks are lost

Running the load in GUI mode. JMeter's manual says CLI mode must be used for load testing.

No ramp-up. A ramp-up of 0 starts every thread at the same instant, which tests a crowd arriving in one moment, not a load. JMeter's manual suggests starting with the ramp-up equal to the number of threads.

No assertions. Without them a page that says "Error" still counts as a success. Assert something only the right page contains.

Reading only the average. It hides the slowest requests; read the 90th percentile and the maximum too.

Comparing runs made at different times on a busy machine. Their numbers differ for reasons that have nothing to do with the application.

Following redirects without knowing it. The redirect adds hidden samples, and the counts stop meaning one page each.

For the journal

Aim; the table of JMeter's parts; the thread group settings and why ${__P(...)}; the table of elements added; the two command lines and their summary lines; the dashboard's statistics table and the response-time charts; the analysis table and what it shows; a note on the machine the test ran on; the result.

Quick revision

Thread group: number of threads (users), ramp-up period, loop count.
Ramp-up: 10 threads over 100 seconds start 10 seconds apart.
Samples are users × loops × requests per loop: 5 × 10 × 5 = 250, and 50 × 10 × 5 = 2500.
Response time is elapsed time; throughput is requests per second; error % is failed samples over all samples.
jmeter -n -t plan.jmx -l results.jtl -e -o report: CLI run with the HTML dashboard; summary + lines are interim, summary = is the total.
${__P(users,5)} reads -Jusers=50, or uses 5.
At the server's ceiling, more users raise response time, not throughput.
A load test measures the whole machine: repeat it on an idle one.

Practical 19: Load Testing with Apache JMeter

Questions you must be able to answer

1. What do the number of threads, the ramp-up period and the loop count control? How many virtual users run, how long JMeter takes to start all of them, and how many times each repeats the steps.

2. Why is the test run from the command line and not from the GUI? JMeter's manual says GUI mode is only for creating the test script; the GUI uses the machine being measured, so load is run in CLI mode.

3. Throughput was about 19.6 requests a second under both loads. What does that tell you? The server was already at its maximum under the light load's pace; more users could not make it serve more requests a second, so they waited longer instead.

4. Why did the average response time rise so much under heavy load? The single PHP process serves one request at a time, so with 50 users each request waited behind many others.

5. What counted as an error in this plan? A failed assertion or HTTP error, and above all any sample slower than 1.5 seconds, the Duration Assertion's limit.

6. The two heavy runs gave 88.16% and 27.28% errors. Which one is right? Both are true measurements of different conditions: the machine was busier during the first. Load tests must be repeated on an idle machine before their numbers are compared.

Contents This chapter on its own page

Chapter Twenty-Three

Practical 20: Bug Tracking and the Defect Life Cycle in Bugzilla

Syllabus topic Module 2, "Bug Tracking and Defect Lifecycle Management", "Tool: Bugzilla", "Log software defects using Bugzilla. Assign severity and priority levels. Track defect lifecycle stages and generate defect summary reports."

Aim

To log software defects in Bugzilla with severity and priority levels, to track them through the stages of the defect life cycle, and to generate defect summary reports.

What you need to know before you start

A bug tracker keeps one shared record of every defect, from the moment it is reported until it is closed: what is wrong, how bad it is, who is fixing it, what has been done, and when. Practical 3 wrote defect reports in a table; a tracker turns each into a record that the tester, the developer and the manager all work on, and that remembers every change.

Bugzilla is a free, web-based bug tracker. Its main parts, as its user guide describes them:

Field	What it holds
Product and Component	what the bug is in: a product has one or more components
Summary	one line saying what is wrong, in the user's terms
Description	the report itself: steps, expected result, actual result
Severity	how bad the problem is
Priority	how soon it should be fixed
Status and Resolution	where the bug is in its life cycle, and how it ended
Assigned To	the person responsible for fixing it
Comments and History	everything said about the bug, and every change made to it

Severity and priority are different questions. Bugzilla's guide describes severity as running from blocker ("application unusable") to trivial ("minor cosmetic issue"), with an extra value, enhancement, for requests. Priority is set by the assignee, or by someone with authority over their time such as a project manager. A misspelt word on the page every visitor sees is trivial in severity and may still be urgent; a crash in a feature nobody uses yet is severe and can wait.

The values in the lab's Bugzilla 5.2, read from the running installation:

Field	Values, most to least
Severity	blocker, critical, major, normal, minor, trivial, enhancement
Priority	Highest, High, Normal, Low, Lowest, and --- for not yet set

Bugzilla's own guide says the priorities default to P1 to P5; the 5.2 installed here offered the names above. Your college's Bugzilla may differ again, because an administrator can change every list, so use the values your installation shows.

The life cycle. A bug's status says where it is; when it is closed, its resolution says why:

Status	Meaning
UNCONFIRMED	reported, not yet confirmed to be a real bug
CONFIRMED	confirmed; waiting to be worked on
IN_PROGRESS	someone is working on it
RESOLVED	closed by the developer, with a resolution
VERIFIED	a tester has checked the resolution

Resolution	Meaning
FIXED	the defect was corrected
INVALID	not a bug
WONTFIX	a real bug that will not be fixed
DUPLICATE	the same as another bug, whose number is recorded
WORKSFORME	the developer could not make it happen

Practical 20: Bug Tracking and the Defect Life Cycle in Bugzilla

Which moves are allowed is Bugzilla's workflow, set by the administrator. This is the lab's, as Bugzilla's Administration, Bug Status Workflow page shows it; a tick means the move from the row's status to the column's status is allowed:

Bugzilla's workflow matrix: which status may follow which

Figure 23.1 The default workflow of Bugzilla 5.2. From {Start} a new bug can be UNCONFIRMED, CONFIRMED or IN_PROGRESS; RESOLVED can go on to VERIFIED, or back to UNCONFIRMED or CONFIRMED when a bug is reopened.

Installing Bugzilla

Bugzilla needs Perl, a web server and a database. Its documentation covers Linux, where it recommends installing for large sites, and Windows, with ActivePerl or Strawberry Perl; its installer, checksetup.pl, checks the Perl modules, creates the database and the first administrator. Most colleges run one Bugzilla that every student logs in to, and that is the easy way to do this practical.

The lab installed Bugzilla 5.2 on Linux with Apache and SQLite, which Bugzilla's documentation describes as easy to configure and suitable for testing and very small installations. If you install 5.2 with SQLite yourself and checksetup.pl stops with "Attempt to bless into a reference", the cause is one line of 5.2: its SQLite schema class says use base where the other databases' classes say extends. Changing that one line let the lab's installation finish.

How the lab talks to Bugzilla. You will use the web pages: File a Bug, the bug's page, Search, Reports. The lab does the same through Bugzilla's REST interface, with an API key instead of a password; any user can make one on the API Keys tab of Preferences. The commands below keep the key in K and the address in B. Each is followed by what the same step is in the browser.

Setting up the product

In the browser: Administration, Products, then add the product and a component for each part of the site. The lab creates the product and its three components:

$ K=$(cat ~/bugzilla-api-key)
$ B=http://localhost:8095/rest
$ J="Content-Type: application/json"
$ curl -s -X POST "$B/product?api_key=$K" -H "$J" -d '{"name": "Practice Portal", "description": "The college website these notes test", "version": "1.0", "is_open": true, "has_unconfirmed": true}'; echo
{"id":2}
$ for c in Registration Results Upload; do curl -s -X POST "$B/component?api_key=$K" -H "$J" -d "{\"product\": \"Practice Portal\", \"name\": \"$c\", \"description\": \"The $c pages\", \"default_assignee\": \"qa.lead@example.com\"}"; echo; done
{"id":2}
{"id":3}
{"id":4}

Each answer is the new record's number. The product is number 2 because Bugzilla's installer creates a sample product, TestProduct, as number 1, with its own component number 1; the three components are 2, 3 and 4. has_unconfirmed, in the words of Bugzilla's API documentation, makes the UNCONFIRMED status available for this product. default_assignee is who receives new bugs in the component; in the lab, the one account, the QA Lead, plays every part.

Practical 20: Bug Tracking and the Defect Life Cycle in Bugzilla

Logging the defects

The eight defects are the book's own. Bugs 1 to 5 are Practical 3's DEF-01 to DEF-05, with the severities and priorities Practical 3 gave them: Bugzilla has no "Medium", so DEF-03's becomes Normal, the middle value, and no NEW status, so each starts CONFIRMED. Bug 6 is the Marks sort that Practicals 12, 14 and 16 found. Bug 7 is a real defect found while this book was written: the upload page's styled button did nothing. Bug 8 reports DEF-02 again in other words, as a second tester might.

In the browser: New, choose the product, and fill in the form. Summary, Component, Severity and Description are on the form at once; Priority is under Show Advanced Fields:

Bugzilla's File a Bug form filled in for DEF-01

Figure 23.2 The form with the advanced fields shown: severity major and priority High chosen, the Status box offering CONFIRMED, and the summary typed. Under it, Possible Duplicates: as the summary is typed, Bugzilla lists existing bugs with similar words, here bug 1 itself, so that nobody reports the same defect twice.

As files, one per defect: the fields of a defect report, in the form Bugzilla's REST interface takes them.

{
  "product": "Practice Portal",
  "component": "Registration",
  "version": "1.0",
  "summary": "Mobile number with 9 digits is accepted",
  "severity": "major",
  "priority": "High",
  "op_sys": "All",
  "platform": "All",
  "description": "Found by test case TC08, requirement R3 (Practical 3).\nSteps: open register.php, enter the valid values, change Mobile number to 987654321, click Register.\nExpected: the registration is rejected, with the mobile number message.\nActual: \"Registration successful\", with a registration number."
}

{
  "product": "Practice Portal",
  "component": "Registration",
  "version": "1.0",
  "summary": "Mismatched passwords are accepted",
  "severity": "major",
  "priority": "High",
  "op_sys": "All",
  "platform": "All",
  "description": "Found by test case TC20, requirement R7 (Practical 3).\nSteps: enter the valid values, change Confirm password to Secret999, click Register.\nExpected: the registration is rejected, because Confirm password differs from Password.\nActual: the student is registered, and no message about the passwords appears."
}

{
  "product": "Practice Portal",
  "component": "Registration",
  "version": "1.0",
  "summary": "A student one day under 17 on 1 July 2026 is accepted",
  "severity": "major",
  "priority": "Normal",
  "op_sys": "All",
  "platform": "All",
  "description": "Found by test case TC13, requirement R4 (Practical 3).\nSteps: enter the valid values, change Date of birth to 2009-07-02, click Register.\nExpected: the registration is rejected; the student is 16 on 1 July 2026.\nActual: the student is registered. A date of birth of 2010-01-01 is rejected, so the check exists but is wrong near the boundary."
}

{
  "product": "Practice Portal",
  "component": "Registration",
  "version": "1.0",
  "summary": "Full name longer than 40 characters is accepted",
  "severity": "minor",
  "priority": "Low",
  "op_sys": "All",
  "platform": "All",
  "description": "Found by test case TC05, requirement R1 (Practical 3).\nSteps: enter the valid values, change Full name to the 41-character name Sri Venkata Lakshmi Subramaniam Chowdhary, click Register.\nExpected: the registration is rejected; the maximum is 40 characters.\nActual: the student is registered."
}

Practical 20: Bug Tracking and the Defect Life Cycle in Bugzilla

{
  "product": "Practice Portal",
  "component": "Registration",
  "version": "1.0",
  "summary": "Spelling mistake in the password message",
  "severity": "trivial",
  "priority": "Low",
  "op_sys": "All",
  "platform": "All",
  "description": "Found by test case TC17, requirement R6 (Practical 3).\nSteps: enter the valid values, change Password and Confirm password to Secret1, click Register.\nExpected: \"Password must be at least 8 characters and contain a digit.\"\nActual: \"Pasword must be at least 8 characters and contain a digit.\""
}

{
  "product": "Practice Portal",
  "component": "Results",
  "version": "1.0",
  "summary": "Sorting by Marks orders the numbers as text",
  "severity": "major",
  "priority": "High",
  "op_sys": "All",
  "platform": "All",
  "description": "Found by Practical 12's program and Practical 16's test marksSortLowestFirst.\nSteps: open results.php, wait for the table, click the Marks heading.\nExpected: the marks in ascending numeric order, 5, 9, 38 and so on up to 100.\nActual: 100, 38, 45, 5, 50 and so on, with 9 after 88."
}

{
  "product": "Practice Portal",
  "component": "Upload",
  "version": "1.0",
  "summary": "The styled upload button does nothing",
  "severity": "major",
  "priority": "High",
  "op_sys": "All",
  "platform": "All",
  "description": "Found while writing Practical 13.\nSteps: open upload.php, click \"Choose a file to upload\".\nExpected: the file dialog opens, and choosing a file uploads it.\nActual: nothing happens. A file put into the hidden file box by a test is not sent either."
}

{
  "product": "Practice Portal",
  "component": "Registration",
  "version": "1.0",
  "summary": "Confirm password is not compared with Password",
  "severity": "major",
  "priority": "Normal",
  "op_sys": "All",
  "platform": "All",
  "description": "Steps: register with Password Secret123 and Confirm password Secret999.\nExpected: an error saying the passwords differ.\nActual: the registration succeeds."
}

$ K=$(cat ~/bugzilla-api-key)
$ for n in 1 2 3 4 5 6 7 8; do curl -s -X POST "http://localhost:8095/rest/bug?api_key=$K" -H "Content-Type: application/json" -d @bugs/$n.json; echo; done
{"id":1}
{"id":2}
{"id":3}
{"id":4}
{"id":5}
{"id":6}
{"id":7}
{"id":8}

Eight bugs, numbered 1 to 8 in the order they were filed. In the browser, each Submit Bug shows the new bug's page with its number.

Moving defects through the life cycle

In the browser each move is made on the bug's page: choose the new Status, a Resolution if closing, add a comment, and Save Changes. The lab makes three kinds of move:

Bug 7 goes all the way, and every step is true. IN_PROGRESS while the cause was found; RESOLVED FIXED with the cause and the fix in the comment, the form field named document of Practical 13; VERIFIED after Practical 13's programs were run again and passed.
Bug 6 practises reopening. It is marked RESOLVED FIXED as a developer might mark a bug by mistake, and reopened to CONFIRMED because Practical 16's test still fails on it, which it really does. The Practice Portal's defects are there on purpose and are never actually fixed.
Bug 8 is closed as a DUPLICATE of bug 2.

Practical 20: Bug Tracking and the Defect Life Cycle in Bugzilla

$ K=$(cat ~/bugzilla-api-key)
$ B=http://localhost:8095/rest
$ J="Content-Type: application/json"
$ curl -s -o /dev/null -w "%{http_code}\n" -X PUT "$B/bug/7?api_key=$K" -H "$J" -d '{"status": "IN_PROGRESS", "comment": {"body": "Looking at upload.php."}}'
200
$ sleep 1
$ curl -s -o /dev/null -w "%{http_code}\n" -X PUT "$B/bug/7?api_key=$K" -H "$J" -d '{"status": "RESOLVED", "resolution": "FIXED", "comment": {"body": "The file box was named document. Inside an inline onclick or onchange attribute on a form control, document means that form field, not the page, so document.getElementById failed. Renamed the field to assignment."}}'
200
$ sleep 1
$ curl -s -o /dev/null -w "%{http_code}\n" -X PUT "$B/bug/7?api_key=$K" -H "$J" -d '{"status": "VERIFIED", "comment": {"body": "Retested: the styled button opens the file dialog, and a file chosen or set by a test is sent (Practical 13, Programs 1 and 2)."}}'
200
$ curl -s -o /dev/null -w "%{http_code}\n" -X PUT "$B/bug/6?api_key=$K" -H "$J" -d '{"status": "RESOLVED", "resolution": "FIXED", "comment": {"body": "Marked fixed."}}'
200
$ sleep 1
$ curl -s -o /dev/null -w "%{http_code}\n" -X PUT "$B/bug/6?api_key=$K" -H "$J" -d '{"status": "CONFIRMED", "comment": {"body": "Reopened: Practical 16 test marksSortLowestFirst still fails with expected [5] but found [100]."}}'
200
$ curl -s -o /dev/null -w "%{http_code}\n" -X PUT "$B/bug/8?api_key=$K" -H "$J" -d '{"status": "RESOLVED", "resolution": "DUPLICATE", "dupe_of": 2}'
200

Every answer is 200, HTTP's success code: Bugzilla accepted each move. A move the workflow forbids is refused: asked to take a CONFIRMED bug straight to VERIFIED, the lab's Bugzilla answered HTTP 400 with "You are not allowed to change the bug status from CONFIRMED to VERIFIED." sleep 1 puts a second between moves: when the lab made three moves within one second, Bugzilla recorded them as one entry in the history.

This is bug 7 at the end, as a reader of the tracker sees it:

Bug 7 in Bugzilla: VERIFIED FIXED, with its comments

Figure 23.3 Status VERIFIED FIXED, component Upload, importance High major. Below, the original report and three comments: the start of work, the cause and the fix, and the retest.

The history of one defect

Every change to a bug is kept. In the browser, the History link on the bug's page:

Bug 7's activity log: CONFIRMED to IN_PROGRESS to RESOLVED FIXED to VERIFIED

Figure 23.4 Who changed what and when: status CONFIRMED to IN_PROGRESS, then IN_PROGRESS to RESOLVED with the resolution set to FIXED, then RESOLVED to VERIFIED.

The same history from the REST interface. Bugzilla answers in JSON, whose parts come in a different order on every call, so a short Python line prints one line per change instead, in a fixed order:

Practical 20: Bug Tracking and the Defect Life Cycle in Bugzilla

$ curl -s "http://localhost:8095/rest/bug/7/history?api_key=$(cat ~/bugzilla-api-key)" | python3 -c 'import json, sys; [print(h["when"], h["who"], c["field_name"] + ":", c["removed"] or "(empty)", "->", c["added"]) for h in json.load(sys.stdin)["bugs"][0]["history"] for c in sorted(h["changes"], key=lambda c: c["field_name"])]'
2026-09-30T19:53:07Z qa.lead@example.com status: CONFIRMED -> IN_PROGRESS
2026-09-30T19:53:08Z qa.lead@example.com resolution: (empty) -> FIXED
2026-09-30T19:53:08Z qa.lead@example.com status: IN_PROGRESS -> RESOLVED
2026-09-30T19:53:10Z qa.lead@example.com status: RESOLVED -> VERIFIED

One line per field changed: when (in UTC), who, the field, and its value before and after. The second and third lines share a time, because closing a bug sets two fields at once, the status and the resolution. This record is how a team answers "when was this fixed, and who checked it?"

Summary reports

The list of bugs, in the browser: Search for the product, or Browse:

Bugzilla's bug list for the Practice Portal: 8 bugs

Figure 23.5 8 bugs found: the six open ones CONFIRMED, bug 8 RESOLVED DUPLICATE, bug 7 VERIFIED FIXED. The buttons below the list include CSV, the form the lab uses next.

Reports, then Tabular reports or Graphical reports, count the bugs of a search by one field against another. Bugzilla's guide calls a report a view of the current state of the bug database. The lab takes the list and two reports as CSV:

$ curl -s "http://localhost:8095/buglist.cgi?product=Practice%20Portal&columnlist=component,bug_severity,priority,bug_status,resolution,short_desc&ctype=csv"
bug_id,"component","bug_severity","priority","bug_status","resolution","short_desc"
1,"Registration","major","High","CONFIRMED"," ---","Mobile number with 9 digits is accepted"
2,"Registration","major","High","CONFIRMED"," ---","Mismatched passwords are accepted"
6,"Results","major","High","CONFIRMED"," ---","Sorting by Marks orders the numbers as text"
3,"Registration","major","Normal","CONFIRMED"," ---","A student one day under 17 on 1 July 2026 is accepted"
4,"Registration","minor","Low","CONFIRMED"," ---","Full name longer than 40 characters is accepted"
5,"Registration","trivial","Low","CONFIRMED"," ---","Spelling mistake in the password message"
8,"Registration","major","Normal","RESOLVED","DUPLICATE","Confirm password is not compared with Password"
7,"Upload","major","High","VERIFIED","FIXED","The styled upload button does nothing"
$ curl -s "http://localhost:8095/report.cgi?product=Practice%20Portal&x_axis_field=bug_severity&y_axis_field=bug_status&format=table&action=wrap&ctype=csv"
"Status / Severity","major","minor","trivial"
"CONFIRMED",4,1,1
"RESOLVED",1,0,0
"VERIFIED",1,0,0
$ curl -s "http://localhost:8095/report.cgi?product=Practice%20Portal&x_axis_field=priority&y_axis_field=component&format=table&action=wrap&ctype=csv"
"Component / Priority","High","Normal","Low"
"Registration",2,2,2
"Results",1,0,0
"Upload",1,0,0

Reading the three:

The list gives each bug's component, severity, priority, status and resolution; " ---" is an open bug's empty resolution. Bugzilla sorts the list by status, then importance: the CONFIRMED bugs first, High before Normal before Low, then the closed ones.
Status against severity: six bugs are major, one minor, one trivial; four of the major ones are still CONFIRMED, one is RESOLVED (the duplicate) and one VERIFIED (bug 7).
Component against priority: Registration holds six bugs, two each at High, Normal and Low; Results and Upload one High each.

The same reports as a table and as a chart in the browser:

Bugzilla's tabular report: status against severity, with totals

Figure 23.6 Status down the side, severity across: 4, 1 and 1 CONFIRMED; 1 RESOLVED; 1 VERIFIED; 8 in all. Each number is a link to the list of those bugs.

Bugzilla's bar chart of bugs by severity: major 6, minor 1, trivial 1

Figure 23.7 The graphical report of the same search by severity alone. Bar, Line, Pie, Table and CSV switch between views of the same numbers.

Practical 20: Bug Tracking and the Defect Life Cycle in Bugzilla

Observations

Bug	Defect	Component	Severity	Priority	Final status
1	Mobile number with 9 digits is accepted	Registration	major	High	CONFIRMED
2	Mismatched passwords are accepted	Registration	major	High	CONFIRMED
3	A student one day under 17 on 1 July 2026 is accepted	Registration	major	Normal	CONFIRMED
4	Full name longer than 40 characters is accepted	Registration	minor	Low	CONFIRMED
5	Spelling mistake in the password message	Registration	trivial	Low	CONFIRMED
6	Sorting by Marks orders the numbers as text	Results	major	High	CONFIRMED, after RESOLVED FIXED and a failed retest
7	The styled upload button does nothing	Upload	major	High	VERIFIED FIXED
8	Confirm password is not compared with Password	Registration	major	Normal	RESOLVED DUPLICATE of 2

Result

Eight defects were logged in Bugzilla under a Practice Portal product with three components, each with a severity and a priority. Bug 7 was tracked through CONFIRMED, IN_PROGRESS and RESOLVED FIXED to VERIFIED, bug 6 was reopened after its retest failed, and bug 8 was closed as a duplicate of bug 2; Bugzilla recorded every change in each bug's history. Defect summary reports were generated: the bug list, a tabular report of status against severity, a report of priority against component, and a bar chart by severity.

Where marks are lost

Severity and priority treated as one field. Severity is how bad; priority is how soon. Give a reason for each.

A summary in the tester's terms. "Validation regex wrong" says what you guess; "Mobile number with 9 digits is accepted" says what happens.

No steps, expected and actual result. A developer who cannot reproduce the bug closes it as WORKSFORME.

Closing your own report as VERIFIED without a retest. VERIFIED means someone checked the fix, and the comment should say how.

Reporting a defect twice. Read the Possible Duplicates list before submitting; if one matches, add to it instead.

Showing only the list. The syllabus asks for the life cycle and summary reports too: include a bug's history and a report.

For the journal

Aim; the field table; the severity, priority, status and resolution values of your Bugzilla; the workflow; your product and components; the defect reports you logged, with severity and priority and why; the moves you made and the history of one bug; the list and at least one tabular or graphical report; the observations table; the result.

Quick revision

Bugzilla bug: product and component, summary, description, severity, priority, status, resolution, assignee, comments, history.
Severity: blocker, critical, major, normal, minor, trivial, enhancement. Priority: Highest to Lowest (older installations P1 to P5).
Life cycle: UNCONFIRMED or CONFIRMED, IN_PROGRESS, RESOLVED with a resolution, VERIFIED; reopening goes back to CONFIRMED or UNCONFIRMED.
Resolutions: FIXED, INVALID, WONTFIX, DUPLICATE, WORKSFORME.
The workflow decides which moves are allowed.
Every change is in the bug's History.
Reports: tabular or graphical, one field against another; the list and reports also download as CSV.

Practical 20: Bug Tracking and the Defect Life Cycle in Bugzilla

Questions you must be able to answer

1. What is the difference between severity and priority? Give an example where they differ. Severity is how bad the problem is; priority is how soon it should be fixed. A spelling mistake on a page every visitor sees is trivial in severity but may be fixed first.

2. Name the statuses a bug passes through when it is found, fixed and checked. CONFIRMED (or UNCONFIRMED first), IN_PROGRESS, RESOLVED with the resolution FIXED, and VERIFIED.

3. What happens when a retest shows a RESOLVED FIXED bug is not fixed? The tester reopens it: its status goes back to CONFIRMED (or UNCONFIRMED), with a comment saying what the retest found.

4. When is a bug closed as DUPLICATE? When it reports the same defect as an existing bug; the duplicate is closed and points to the original's number.

5. Where does Bugzilla record who changed a bug and when? In the bug's History, the activity log: each change with who made it, when, and the old and new values.

6. What does a tabular report show? Counts of the bugs matching a search, one field against another, for example status against severity, with totals.

Contents This chapter on its own page

Chapter Twenty-Four

The Two-Hour Paper: Sitting the Examination

Syllabus topic Evaluation scheme, "External Examination – 30 Marks", "A Semester End Practical Examination of 2 hours duration for 30 marks", "Q. 1 Module 1 15, Q. 2 Module 2 15", "Certified Journal is compulsory for appearing at the time of Practical Exam", "Minimum 80% practical are required to be completed."

Aim

To sit the two-hour practical examination: to answer one practical question on each module completely within the time, and to arrive with a certified journal and a working machine.

The paper, once more

MU prints the external examination as two practical questions in two hours: Q.1 on Module 1 for 15 marks and Q.2 on Module 2 for 15 marks. A certified journal is compulsory to sit, and at least 80 per cent of the practicals must be complete, sixteen of the twenty. There is no separate viva. How the two hours are spent, and what the examiner looks at, are set out in the first chapter of this book; this chapter is the paper itself.

The questions come from the practicals. Each question is one of the twenty exercises, or a small variation on one:

Q.1, Module 1:

Practical	Tool MU names
1: a test suite in Selenium IDE	Selenium IDE (Chrome/Firefox Extension)
2: cross-website functional testing	Selenium IDE
3: manual test cases	MS Excel / Google Sheets
4: Selenium Grid	Selenium Server (Grid)
5: login automation	Selenium WebDriver (Java/PHP)
6: elements, alerts, frames, waits	Selenium WebDriver (Java)
7: data-driven testing	Selenium WebDriver + Apache POI
8: marks from a dynamic table	Selenium WebDriver
9: counting the objects on a page	Selenium WebDriver
10: list and combo box	Selenium WebDriver

Q.2, Module 2:

Practical	Tool MU names
11: checkboxes	Selenium WebDriver
12: dynamic table, sorting, filtering	Selenium WebDriver
13: file upload and download	Selenium WebDriver + Robot Class
14: screenshot and logging	Selenium WebDriver + Log4j
15: Page Object Model	Selenium WebDriver + TestNG
16: TestNG annotations and reports	Selenium WebDriver + TestNG
17: Jenkins	Jenkins + Selenium WebDriver
18: cross-browser testing	Selenium Grid
19: load testing	Apache JMeter
20: bug tracking	Bugzilla

A paper of the printed shape

Semester End Practical Examination                     Total Marks: 30
Software Testing and Quality Assurance Practical        Duration: 2 Hours

Q.1  Write and execute a Selenium WebDriver program to test the login
     page of the given web application with one valid and two invalid
     sets of credentials. Report the expected and actual result and a
     pass or fail for each test case.                            (15)

Q.2  Using TestNG with Selenium WebDriver, automate the checkbox page of
     the given web application: in one test count the checkboxes and the
     ticked ones; in a second test, run after the first, tick two named
     facilities and validate the page's own count. Generate the TestNG
     report.                                                     (15)

Q.1: the worked answer

On the answer sheet, before touching the machine (five minutes):

Aim: to test the login page with one valid and two invalid sets of credentials, and to report each result.
Tool: Selenium WebDriver 4, Java, Chrome.
Test cases:

The Two-Hour Paper: Sitting the Examination

No.	Username	Password	Expected result
TC1	asha	Asha@2026	the dashboard: "Welcome, Asha Patil"
TC2	asha	wrong	"Invalid username or password."
TC3	(empty)	(empty)	"Please enter both username and password."

Then the program. The test cases are data, so one loop runs all three. It waits for the old page to go stale before reading, the lesson of Practicals 5 and 15, and logs out after the valid login so the next case starts logged out:

package practicals;

import java.time.Duration;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public class ExamLogin {
    public static void main(String[] args) {
        // Each test case: username, password, the element that should appear, the text it should hold.
        String[][] cases = {
            {"asha", "Asha@2026", "welcome", "Welcome, Asha Patil"},
            {"asha", "wrong", "error", "Invalid username or password."},
            {"", "", "error", "Please enter both username and password."},
        };
        WebDriver driver = new ChromeDriver();
        WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(5));
        int passed = 0;
        try {
            for (String[] c : cases) {
                driver.get("http://localhost:8080/login.php");
                driver.findElement(By.id("username")).sendKeys(c[0]);
                driver.findElement(By.id("password")).sendKeys(c[1]);
                WebElement button = driver.findElement(By.id("loginBtn"));
                button.click();
                new WebDriverWait(driver, Duration.ofSeconds(5)).ignoring(WebDriverException.class)
                        .until(ExpectedConditions.stalenessOf(button));   // the answer is a new page
                String actual = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id(c[2]))).getText();
                boolean ok = actual.equals(c[3]);
                if (ok) {
                    passed++;
                }
                System.out.printf("%-7s | %-9s | expected: %s | actual: %s | %s%n",
                        c[0].isEmpty() ? "(empty)" : c[0], c[1].isEmpty() ? "(empty)" : c[1],
                        c[3], actual, ok ? "PASS" : "FAIL");
                if (c[2].equals("welcome")) {
                    driver.findElement(By.id("logout")).click();       // log out before the next case
                }
            }
        } finally {
            driver.quit();
        }
        System.out.println(passed + " of " + cases.length + " test cases passed");
    }
}

asha    | Asha@2026 | expected: Welcome, Asha Patil | actual: Welcome, Asha Patil | PASS
asha    | wrong     | expected: Invalid username or password. | actual: Invalid username or password. | PASS
(empty) | (empty)   | expected: Please enter both username and password. | actual: Please enter both username and password. | PASS
3 of 3 test cases passed

On the answer sheet, after the run:

Output: copied exactly, as above.
Observations: 3 test cases run; 3 passed, 0 failed. The valid login reached the dashboard; both invalid logins were refused with the page's own messages.
Result: the login page accepts the valid credentials and rejects a wrong password and empty fields with the correct messages; all three test cases passed.

Q.2: the worked answer

On the answer sheet first: Aim, to count and validate the checkboxes and to tick two facilities, with TestNG; Tool, Selenium WebDriver 4 with TestNG 7, run through Maven; and the two test cases with their expected results: 10 checkboxes with 3 ticked, and after ticking Canteen and Hostel, the page saying "5 facilities selected".

Then the test class. priority makes the count run before the ticking, because ticking changes what the count would see; @BeforeClass opens the page once for both tests:

The Two-Hour Paper: Sitting the Examination

package exam;

import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertTrue;

import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class ExamCheckboxTest {
    private WebDriver driver;

    @BeforeClass
    public void openPage() {
        driver = new ChromeDriver();
        driver.get("http://localhost:8080/checkboxes.php");
    }

    @Test(priority = 1, description = "Ten checkboxes, three of them ticked")
    public void countCheckboxes() {
        List<WebElement> boxes = driver.findElements(By.cssSelector("input[type=checkbox]"));
        long ticked = boxes.stream().filter(WebElement::isSelected).count();
        System.out.println("Checkboxes: " + boxes.size() + ", ticked: " + ticked);
        assertEquals(boxes.size(), 10);
        assertEquals(ticked, 3L);
    }

    @Test(priority = 2, description = "Tick Canteen and Hostel by their labels")
    public void tickTwoFacilities() {
        for (String name : new String[] {"Canteen", "Hostel"}) {
            WebElement box = driver.findElement(By.xpath("//label[normalize-space()='" + name + "']/input"));
            if (!box.isSelected()) {
                box.click();
            }
            assertTrue(box.isSelected(), name + " should be ticked");
        }
        String summary = driver.findElement(By.id("summary")).getText();
        System.out.println("The page says: " + summary);
        assertEquals(summary, "5 facilities selected");
    }

    @AfterClass
    public void closePage() {
        driver.quit();
    }
}

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Examination Q2">
  <test name="Checkboxes">
    <classes>
      <class name="exam.ExamCheckboxTest"/>
    </classes>
  </test>
</suite>

$ mvn test -Dsurefire.suiteXmlFiles=exam.xml
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------< stqa:stqa-practicals >------------------------
[INFO] Building stqa-practicals 1.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/main/resources
[INFO]
[INFO] --- compiler:3.16.0:compile (default-compile) @ stqa-practicals ---
[INFO] Recompiling the module because of added or removed source files.
[INFO] Compiling 1 source file with javac [debug release 17] to target/classes
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ stqa-practicals ---
[INFO] skip non existing resourceDirectory /home/student/stqa-practicals/src/test/resources
[INFO]
[INFO] --- compiler:3.16.0:testCompile (default-testCompile) @ stqa-practicals ---
[INFO] Recompiling the module because of changed dependency.
[INFO] Compiling 1 source file with javac [debug release 17] to target/test-classes
[INFO]
[INFO] --- surefire:3.5.6:test (default-test) @ stqa-practicals ---
[INFO] Using configured provider org.apache.maven.surefire.junit4.JUnit4Provider
[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
Oct 01, 2026 12:59:22 AM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 154, returning the closest version; found: 153; Please update to a Selenium version that supports CDP version 154
Checkboxes: 10, ticked: 3
The page says: 5 facilities selected
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.094 s -- in TestSuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.915 s
[INFO] Finished at: 2026-10-01T00:59:23+05:30
[INFO] ------------------------------------------------------------------------
$ ls target/surefire-reports/*.html
target/surefire-reports/emailable-report.html
target/surefire-reports/index.html

The Two-Hour Paper: Sitting the Examination

On the answer sheet, after the run:

Output: the two lines the tests printed, "Checkboxes: 10, ticked: 3" and "The page says: 5 facilities selected", and Surefire's "Tests run: 2, Failures: 0, Errors: 0, Skipped: 0". The JUnit runner's "Tests run: 0" before it is the second provider of chapter six, finding nothing to run.
The report: emailable-report.html and index.html in target/surefire-reports, as Practical 16 showed. Open one and show it to the examiner, or copy its summary into the answer.
Observations: 2 tests run in priority order; 2 passed. 10 checkboxes, 3 ticked; after the two ticks the page's own count read 5.
Result: the checkbox counts and the dynamic selection were validated with TestNG, and TestNG's HTML report was generated.

When something goes wrong in the hall

Every row is an error this book met while it was being written, with the chapter where it happens and is explained:

What you see	What it means	What to do	Chapter
`NoSuchElementException`	the locator finds nothing, yet	check the locator against the page source; if the element comes later, wait for it	six, Practical 6
`TimeoutException: Expected condition failed`	a wait's condition never came true	read which condition; check its locator and value	Practical 6
`StaleElementReferenceException`	the page changed after the element was found	find the element again	six
A test reads the old page after a click that sends a form	the new page had not arrived	wait for the clicked element to go stale	Practicals 7, 15
`InvalidArgumentException` on `click()` of a file box	WebDriver refuses to click file boxes	`sendKeys` the file's full path	Practical 13
`Tests run: 0` and BUILD SUCCESS	the wrong test runner ran your tests	both Surefire providers in the pom	six
`NullPointerException` about `driver`, running by groups	the set-up method belongs to no group	`alwaysRun = true` on it	Practical 16
Only ERROR lines appear, and no log file	`log4j2.xml` is not on the classpath	put it in `src/main/resources`	Practical 14
A red warning about CDP version	Selenium's CDP files are older than Chrome	nothing: the program ran; it is a warning	six
The Jenkins build is red though only tests failed	Maven stopped at the first failing test	`-Dmaven.test.failure.ignore=true`, and the build is UNSTABLE	Practical 17
A Grid request hangs for minutes	no node offers that browser	start the node, or `--reject-unsupported-caps true` on the hub	Practical 18
JMeter shows a high error %	responses slower than the Duration Assertion, or wrong pages	read the dashboard's Errors table	Practical 19
Bugzilla refuses a status	the workflow forbids that move	make an allowed move first	Practical 20

If a fix does not come in time, write the error down exactly, with one sentence on what it means and what you would try next. The first chapter explains why that is worth marks.

The Two-Hour Paper: Sitting the Examination

The machine, the day before

Every tool starts: java -version, mvn -v, Eclipse, Chrome and Firefox, and, if your question could need them, Jenkins, JMeter and Bugzilla.
The drivers are already there. Selenium Manager downloads a browser's driver the first time it is needed and keeps it in ~/.cache/selenium, as its documentation says. Run one program on each browser on the exam machine beforehand, in case the hall has no internet.
The project builds: mvn test-compile in your project folder succeeds, so Maven has every library.
The Practice Portal runs, or whatever application the examiner gives you opens in the browser.
You know where your practicals are: one project, one package per practical, as this book builds it, so any earlier program can be opened and adapted.

The journal, before it is certified

The certified journal is compulsory to sit, and the journal is also 5 of the internal marks. Before you take it to be certified:

At least sixteen practicals complete, each with everything the first chapter lists: aim, tool, theory, the page under test, test cases or procedure, the program or configuration, the output, the observations, the result, the date and the signature.
Outputs that are real: printed from your own runs, not copied from a friend or from this book. You may be asked to explain any of them.
The figures MU's exercises ask for: the HTML reports of Practicals 16 and 17, the JMeter dashboard of Practical 19, the Bugzilla report of Practical 20.

Result

A paper of MU's printed shape, one question from each module for 15 marks, was answered completely: Q.1 by a Selenium WebDriver program whose three test cases all passed, Q.2 by a TestNG class whose two prioritised tests passed and produced TestNG's HTML report. Each answer was written in the order the examiner reads it: aim, tool, test cases, program, output, observations, result.

Where marks are lost

No test cases written before the program. The examiner cannot tell what "passed" means.

Output left blank. Write the real output, or the real error with a line on what it means.

Observations and result missing. The run is only half the answer.

Starting the harder question first and finishing neither.

A machine never tried before the day. A driver that must be downloaded in a hall with no internet is a lost question.

A journal short of sixteen practicals, or uncertified. You cannot sit the paper at all.

For the journal

The journal is certified before this paper; there is no journal entry for the paper itself. Use the checklist above on every practical before you hand the journal in.

The Two-Hour Paper: Sitting the Examination

Quick revision

External paper: 2 hours, 30 marks: Q.1 on Module 1 for 15, Q.2 on Module 2 for 15. No separate viva.
Certified journal compulsory; at least 16 of the 20 practicals complete.
Write before you build: aim, tool, test cases with expected results.
After the run: output exactly, observations with counts, a one-line result.
Test cases as data, waits for the new page, and a PASS or FAIL for each.
Try the whole machine the day before, drivers included.

Questions you must be able to answer

1. How are the 30 external marks divided, and how long is the paper? Two practical questions of 15 marks, one on each module, in two hours.

2. What must you have before you are allowed to sit? A certified journal, with at least 80 per cent of the practicals, sixteen of twenty, completed.

3. In Q.1, why does the program wait for the Login button to go stale? Because the answer to the login is a new page; reading before it arrives would read the old one.

4. In Q.2, why does countCheckboxes have a lower priority number than tickTwoFacilities? So that it runs first: ticking two boxes changes the counts it checks.

5. Your program will not run and time is short. What do you write? The exact error the tool printed, what it means, and what you would try next, together with the test cases and the program.

Contents This chapter on its own page

The rest of this subject

These notes are cut from the University's printed syllabus. Open the syllabus itself for the same subject.

Syllabus for Software Testing and Quality Assurance Practical

Issue