Automated accessibility checks confirm that an error message is associated with its field; they cannot tell you that NVDA reads it twice, that VoiceOver on iOS never announces the live region, or that JAWS reads the error summary’s heading but not its links. Those are the differences users hit, and they only show up with real screen readers.

Testing every combination is impossible; testing none is how forms ship with silent errors. This page, part of ARIA live regions for form errors, defines a small matrix of screen reader and browser pairings that covers most users, a script of form-error scenarios to run on each, and what “passing” sounds like — so QA can run it before every release.

Context and prerequisites

The pairings that matter, based on how screen readers are commonly used (surveys of screen-reader users, such as WebAIM’s, consistently show a few dominant combinations):

  • NVDA + Chrome (Windows) — free, widely used; a primary desktop target.
  • JAWS + Chrome (Windows) — common in workplaces and education; behaves differently from NVDA in places.
  • VoiceOver + Safari (macOS) — the default on Macs; Safari is the browser VoiceOver is best tested with.
  • VoiceOver + Safari (iOS) — the dominant mobile screen reader; touch navigation changes how forms are explored.
  • TalkBack + Chrome (Android) — the main Android pairing.
  • NVDA + Firefox — useful as a secondary check where Firefox usage is significant.

Screen readers have two broad interaction modes on desktop: browse/virtual cursor (reading the page) and focus/forms mode (typing into fields). Error behaviour must be checked in both, because descriptions and live regions are handled differently in each.

The testing matrix NVDA with Chrome on Windows is a primary target; check live region timing and forms mode transitions. JAWS with Chrome on Windows is primary for enterprise audiences; check description reading and summary links. VoiceOver with Safari on macOS is primary; check aria-describedby reading and rotor navigation to errors. VoiceOver with Safari on iOS is primary for mobile; check swipe order, live regions and focus after submit. TalkBack with Chrome on Android is primary for mobile; check live regions and focus. NVDA with Firefox is secondary. Pairing Platform Priority Listen especially for NVDA + Chrome Windows primary live region timing; forms mode JAWS + Chrome Windows primary (enterprise) descriptions; summary links VoiceOver + Safari macOS primary describedby; rotor to errors VoiceOver + Safari iOS primary (mobile) swipe order; focus after submit TalkBack + Chrome Android primary (mobile) live regions; focus NVDA + Firefox Windows secondary regressions vs Chrome

The core pattern: a scripted scenario set

The “code” here is a test script — a checklist QA can follow identically on each pairing. Keep it in the repository next to the form so it changes when the form does.

FORM ERROR SCRIPT — run on each pairing in the matrix
Setup: fresh page load; screen reader on; default verbosity.

S1  Field error on blur
    Tab to "Email address". Type "ada@". Tab away.
    EXPECT: on leaving, nothing announced OR one polite announcement of the error.
    Shift+Tab back to the field.
    EXPECT: label, "invalid entry"/"invalid data", then the error text, then the hint.

S2  Error clears when fixed
    Type "example.com". Tab away and back.
    EXPECT: no "invalid"; error text not read.

S3  Submit with several errors
    Clear two required fields. Press Enter in a text field.
    EXPECT: focus moves to the error summary; its heading ("There are 2 problems") is read,
            then the list is discoverable by arrowing/swiping. No duplicate announcement.

S4  Summary link to field
    Activate the first summary link.
    EXPECT: focus lands IN the field (forms mode on desktop), label + invalid + error read.

S5  Group error (radio/checkbox)
    Submit without choosing a contact method.
    EXPECT: entering the group reads legend + error once; each option is not marked invalid
            unless designed so.

S6  Async status
    Type an existing username, pause.
    EXPECT: at most one announcement of the result; "checking" is not announced.

S7  Server error after submit
    Trigger a server-side error (test account).
    EXPECT: message reaches the right field or a form-level alert; announced once.

Record: PASS / FAIL / NOTE per step, with the exact words spoken for any FAIL.

Step-by-step walkthrough

  1. Fix the matrix to your audience. Use your analytics and user research to confirm the pairings; the six above are a sound default for public-facing forms.
  2. Script the scenarios. Each scenario is a sequence of real keystrokes or gestures and an expected spoken outcome. Vague instructions (“check errors are announced”) produce inconsistent results.
  3. Record exact speech for failures. “NVDA read the error twice” is actionable; “announcements weird” is not.
  4. Run desktop scenarios in both modes. Arrow through the form in browse mode and Tab through it in focus mode; descriptions and live regions can behave differently.
  5. Run mobile scenarios with gestures. Swipe right to move, double-tap to activate, and use the rotor (iOS) or reading controls (Android) to jump between form controls.
  6. Re-run on every change to error behaviour. Timing changes, new summary markup, a new live-region policy such as throttling live region announcements — each can change what is spoken.

Why automated checks are not enough on their own

Automated tools verify structure: that aria-describedby points to an existing element, that the field has a name, that aria-invalid is set. Screen readers then decide how and when to speak that structure, and they differ. One reads descriptions immediately on focus, another after a pause, a third only on request; one speaks both a live region and the focused element’s description, another only one of them. Forms built only to pass automated checks often sound fine in one screen reader and broken in another. A short manual script across a fixed matrix, run at predictable points in the release cycle, is how teams catch those differences before users do.

What passing sounds like When focusing a field with an error, the screen reader should say the label, that the entry is invalid, and then the error text, once. After an invalid submit, it should read the summary heading and let the user move through the links, with no separate announcement of the same errors. On entering a group with an error, it should read the legend and the group error once. After an async check, it should announce the result once, without announcing that checking started. Field on focus Label → invalid → error. Once. Submit Summary heading read. Links navigable. No duplicate stream. Group Legend + error, once. Async result Result once. No "checking".

Failure modes and edge cases

1. Errors read twice

A message both referenced by aria-describedby and announced through a live region is often spoken twice on blur. Decide: announce on blur or rely on the description when the user returns — not both for the same event.

2. Errors never read on mobile

On iOS, VoiceOver may not announce live-region changes that happen while focus is on a text field with the keyboard open. Moving focus to the error summary on submit is more reliable than live announcements on mobile, per moving focus to the first invalid field.

3. Verbosity settings

Users customise verbosity; some turn off description reading. Error messages that live only in descriptions may then be missed. The error summary on submit provides a second route.

4. Browse mode skipping error text

In browse mode, users may arrow past error paragraphs that are visually adjacent but outside the field’s label. That is acceptable if the description is announced on focus; test that it is.

5. Test environment drift

Screen reader and browser versions change behaviour. Record the versions used in each test run, and keep a note of known quirks per version so regressions can be told apart from upstream changes.

Fitting the matrix into a release cycle Automated accessibility checks run on every pull request and catch structural problems. When a change touches error behaviour, the short script is run on two primary pairings, such as NVDA with Chrome and VoiceOver with Safari on iOS. Before a major release, the full script runs on every pairing in the matrix, with results recorded alongside the screen reader and browser versions. Every pull request axe + component tests. Structure: names, descriptions, aria-invalid. Changes to error behaviour Short script on 2 pairings. e.g. NVDA + Chrome, VoiceOver + iOS Safari. Before major releases Full script on the whole matrix. Record versions and exact speech for failures.

Verification checklist

Frequently Asked Questions

Can developers run these tests without being screen-reader users?

Yes, with some practice. Learn a handful of commands per screen reader (move, activate, list form controls, read current element) and follow the script. For complex flows, sessions with experienced screen-reader users reveal issues a scripted test will not.

Is there automation for screen-reader output?

Tools exist that drive NVDA and VoiceOver programmatically and capture speech, useful for regression checks on stable flows. They are complex to maintain; most teams combine automated structure checks with a manual scripted pass.

Which one pairing should a small team start with?

NVDA with Chrome on desktop and VoiceOver with Safari on iOS cover a large share of users and exercise both desktop and mobile patterns. Add the others as time allows.

Related

← ARIA Live Regions for Form Errors