Testing a site: what an end-to-end test is

An end-to-end (e2e) test is a program that walks a person’s path through a site in a real browser and compares the result with what is expected.

Why have a machine check the site

The first test of a site is you: you opened it, clicked, looked. On a small site, and the first time, that is enough. But every change can break something far away: someone renamed a class in the footer and the form at the top stopped submitting; a plugin was updated and a field lost its label. You check the place you edited and don’t check the rest. Other people find out before you do.

A test is a small program that does what a person would do and compares the result with what it should be. Its value isn’t that it is smarter but that it never tires and never forgets: you can run it on every change in a minute.

Tests come in different sizes.

• A unit test checks one small function apart from everything else. “The function that computes an order’s cost, given a 10 percent discount, returns 900 out of 1,000.” Fast, cheap, precise.
• An integration test checks how several parts work together. “The inquiry receiver puts an inquiry in the database, and it shows in the list.”
• An end-to-end test (e2e) checks a person’s whole path in a real browser. Open the home page, press “Book,” type a phone number, press “Send,” see “Thank you.” The closest to real life, and the slowest.

The idea comes from Mike Cohn, and it is well described in an article by Ham Vocke on Martin Fowler’s site. It is a pyramid: at the bottom many fast unit tests, higher up the integration tests, and at the very top a few end-to-end tests. The higher the floor, the more it costs to write and maintain, but the smaller the chance that everything is green while it is broken for a person. A site with no server logic of its own (a static one) may have no unit tests at all. Still, two or three end-to-end tests on the main action remain the most useful check of all.

Check yourself: how does an end-to-end test differ from a unit test?

A unit test checks one small part on its own, and an end-to-end test walks a person’s whole path in a real browser. So an end-to-end test is closer to reality, but slower and costlier to maintain.

What an end-to-end test looks like

One of the most widespread tools is Playwright (made by Microsoft). It starts a real browser (Chromium, Firefox or WebKit) and drives it from code. Here is a test of the main action:

import { test, expect } from '@playwright/test';

test('book a consultation', async ({ page }) => {
  await page.goto('/');
  await page.getByRole('link', { name: 'Book' }).click();
  await page.getByLabel('Phone').fill('+1 000 000 0000');
  await page.getByRole('button', { name: 'Send' }).click();
  await expect(page.getByRole('status')).toContainText('Thank you');
});

Read it as a story. Open the page. Find the link named “Book” and click it. Find the field labeled “Phone” and type a number. Press the “Send” button. Make sure the status region contains the words “Thank you.” Notice that the test finds elements the way a person or a screen reader does, by role and label. That isn’t an accident but the advice of the tool’s own authors. Such a test is steady: a class can be renamed, while the label and the role stay. It also checks accessibility along the way: if a field lost its label, the test won’t find it, just as a screen reader wouldn’t.

Two properties make Playwright tests less brittle. First, auto-waiting: before each action it waits until the element appears, becomes visible and isn’t covered, so you needn’t insert a “wait a second” pause. Second, it opens a fresh clean browser every time, so tests don’t affect each other.

This textbook is checked the same way: it has end-to-end tests, as well as the accessibility and speed checks from the two previous lessons. All of it runs automatically on every change.

Try it in the diagram: break the site, choose how the test finds elements, and see what it notices.

The test walks the site’s main action the way a person does. Break the site in any way, choose how the test finds elements, and run it.

Break the site

The phone field lost its label The cookie banner covers the submit button The server answers with a 500 error on submit A developer renamed the button’s class, and visually it is the same

By role and label By CSS class Run the test

1. · Open the home page
2. · Find the link “Book” by role and name and click it
3. · Find the “Phone” field by its label and type a number
4. · Click the “Book” button (found by role and name)
5. · Wait for the message “Thank you” (role status)

Press “Run the test.”

Text alternative: the diagram runs an imaginary test of five steps. A test that finds elements by role and label notices a vanished field label, a covered button and a server error, and doesn’t react to a renamed class. A test by CSS class misses the vanished label and fails on a harmless class rename.

Check yourself: why is it better to find an element by role and label than by CSS class?

A class is a detail of styling. It can be renamed without changing anything visible, and the test fails for nothing. A role and a label are seen by both a person and a screen reader, so a test that finds elements by them also checks accessibility.

What to test on a site

You don’t need to test everything: every test has to be maintained. Start with the question “what can we absolutely not afford to break?” For a site that is almost always:

1. The main action from the brief: book, buy, call, subscribe. The path from the home page to the result. This is the first and main test.
2. An error and an empty form: what a person sees if they enter something wrong. A message next to the field, focus on the error.
3. The main pages open (not a 404), and links between sections don’t lead nowhere.
4. The keyboard: the form can be filled in and submitted without a mouse.
5. Languages: the switcher leads to the right version, and the page’s lang matches the text.
6. Consent and the policy are present beside the form (more on that later in this part).
7. The phone: the main action on a narrow screen (375 pixels) works the same.

Tests don’t need to check other people’s services (does mail work behind the scenes), pixel-perfect matches, or small texts that you change often.

It helps to draw up the plan in words, before any code. Scenarios are customarily written as “Given, When, Then”: where we start, what the person does, what must happen. They are understandable to more than a programmer, and you can show them to a client: “here is what we check on every change.”

Tests and data

A test that submits a form creates a real inquiry. So:

• The data is made up. “Test,” a phone made of zeros, an email at example.com (a domain reserved for examples). Real names and phone numbers of people aren’t used in tests. They are personal data, and you have no right to spend them on a check.
• Test inquiries are marked, so they don’t reach real customers: a separate field, a service address or a separate test environment.
• A test doesn’t hit someone else’s live site. Check your own site on your own test environment or your own copy.
• The checks of forms and hidden bot traps (the next lesson) must not break because of tests: a test is a visitor too, and it needs a way through.

Flaky tests

The worst test is one that sometimes passes and sometimes doesn’t for no reason. It is called flaky. A test that can’t be trusted is worse than none: people stop noticing red, and a real breakage drowns. The causes are usually these: a pause instead of waiting, data shared between tests, the network, the clock. You cure the cause and don’t rerun until it is green.

A useful rule: a red test either finds a real breakage or shows that the test needs rewriting. Ignoring red is wrong in either case.

Where tests run

A test is useful when it runs without your memory. In the lesson on CI/CD you saw that a deploy system can run checks on every commit and refuse to deploy if anything is red. Site tests plug in exactly there. For a small project it is enough that they run on every push of changes. Manual checks can’t be replaced: a real phone, a screen reader and a human eye find what a test didn’t expect.

Practice

You are in the era of 2022: the site deploys automatically, and it is time to add a check to that pipeline. Draw up a plan of end-to-end tests for your project’s main action and finish a Playwright test.

The page keeps a table of scenarios and a block of code in the note. The check reads text: it makes sure the plan has the main action and the sections from the brief, and the code has a role, a label and a check of the result without pauses. It can’t run the test. You’ll run it for real when you connect Playwright to your own project: the instructions are in the sources.

What used to be different

In the 2000s a site was checked by the person who made it: opened in three browsers and “clicked around.” Later came tools like Selenium that could drive a browser but were slow and temperamental: tests failed on their own and people stopped trusting them. Modern tools (Playwright, Cypress) removed the main inconveniences: waiting is built in, browsers install with one command, and a test reads like a story. But the main rule is the same: few tests, on what must not break.