Testing WCAG 2.2

The goal for this site is WCAG 2.2 AA. Like the size budgets, that is checked by tests, not by hand. One is a script that opens the built site in a real browser, the other a unit test for the colours.

Every page, in every mode

A page can pass in light mode and fail in dark mode, or work on a desktop and break on a phone. That's why the script builds the site into a temp folder and runs axe (a library that checks a page against the WCAG rules) in headless Chromium on every page, in light and dark, at 1440 and 390 pixels wide. open() starts a fresh tab with that theme and width and waits until every island is hydrated:

export const AXE_TAGS = ["wcag2a", "wcag2aa", "wcag21a", "wcag21aa", "wcag22aa"];

for (const p of pages)
  for (const theme of ["light", "dark"] as const)
    for (const viewport of ["desktop", "mobile"] as const) {
      const { ctx, page } = await open(browser, url(p), { viewport, theme });
      const label = `${p} ${theme} ${viewport}`;
      await axe(page, label);

The tags pick the rule sets, WCAG 2.0 to 2.2 up to level AA. Some states only exist after a click, so axe runs again on the terminal after a few commands, the notes filter set to TIL, the contact form with errors and the open mobile menu. Then every page loads once more without its scripts, because it has to work without JavaScript, too.

Every result goes through check(name, ok, detail), which counts one check and records a failure when ok is false. An axe run without violations is one check, one focus ring at one Tab stop is another. A local run makes 645, most of them from the keyboard tests below, because every Tab stop counts twice. With the analytics banner switched on, the banner and the footer's Privacy settings button add Tab stops, and it makes 757. It runs on its own with bun run test:a11y, not in bun test.

Tab order

axe doesn't press any keys, so the script also tabs through every page at both widths and builds two lists. The first comes from the DOM: every visible link, button, input, summary and tabindex element in source order, each as a short label like a[Notes]. For the second, the script presses Tab and writes down the label of each focused element until focus is back at the first one. The test passes when both lists are the same:

const expected = await expectedFocusables(page);
const stops = await tabThrough(page);
const ids = stops.map((s) => s.id);
check(`tab order ${label}`, JSON.stringify(ids) === JSON.stringify(expected));
for (const s of stops) {
  check(`2.4.7 focus visible ${label}`, s.indicator, s.id);
  check(`2.4.11 focus not obscured ${label}`, !s.obscured, s.id);
}

At every stop, the focus ring counts as drawn when the element or its wrapper has an outline or a box shadow. It counts as not obscured when it is in the window and the browser finds it under its centre or one of two corners.

The terminal

In the terminal, Tab also completes commands, and an input that keeps Tab forever would trap keyboard users. So the key handler only takes over Tab when there is a completion:

case "Tab": {
  if (e.shiftKey) return PASS;
  const completion = tabCompletion(fs, s);
  return completion ? { action: { type: "complete", completion }, preventDefault: true } : PASS;
}

PASS lets the browser move focus as usual. tabCompletion() returns nothing when the input is empty, or when the word is complete and the suggestions would stay the same. The test checks both cases, first the empty input, where one Tab has to leave the field (active() returns the class name of the focused element):

await input.focus();
await page.keyboard.press("Tab");
check("terminal: Tab on an empty input leaves it (no keyboard trap)", !(await active()).includes("terminal__input"), await active());

Then it types he, Tab has to turn it into help and keep focus, and a few more presses have to leave the field. The theme switch and the mobile menu are tested with Enter and Space.

Contrast in both themes

Contrast needs no browser. All colours are tokens in tokens.css, so a unit test reads that file and computes the contrast of every token pair the CSS uses, in both themes. Text needs 4.5:1, borders, states and focus rings 3:1. Each line names foreground, background, minimum and where the pair appears:

const PAIRS = [
  ["--text", "--bg", TEXT, "body text"],
  ["--muted", "--bg", TEXT, "dates, labels, footer"],
  ["--accent", "--bg", UI, "focus ring"],
  ["--field-line", "--field", UI, "input and checkbox border"],
];

A new colour combination gets a new line here. The commit with these tests also added a --field-line token for input and checkbox borders, a lighter terminal placeholder, a 24 px captcha checkbox and 44 px targets in the footer and on the 404 page.

What WCAG 2.2 added

WCAG 2.2 added new criteria at level A and AA. Each one needs an answer, and some of the answers are the checks above:

The banner is the one fixed element on the site. While it shows, the page gets extra space at the bottom and focus scrolls above it, and with the banner switched on, the Tab stop check tests exactly that.