Property-Based Testing for Validators

Hand-written validator tests check the inputs someone thought of — "", "abc", "[email protected]" — and miss the ones users actually produce: a trailing no-break space, a number with two decimal separators, an emoji in a name, a date on 29 February, a card number whose formatting round-trip drops a digit.

Property-based testing flips the approach: instead of listing inputs, you state properties that must hold for all inputs, and a library such as fast-check generates hundreds of cases, including awkward ones, and shrinks any failure to a minimal example. Validators, parsers and formatters — pure functions with clear invariants — are ideal subjects. This page is part of testing form validation.

Context and prerequisites

Properties that suit form code:

• Round trip. parse(format(x)) === x for every valid raw value — the formatter and parser agree.
• Idempotence. normalise(normalise(s)) === normalise(s) — trimming or formatting twice changes nothing more.
• Validity is preserved. If validate(x) passes, then validate(normalise(x)) passes — normalisation never makes a valid value invalid.
• Never throws. validate(anyString) returns an error or null, never an exception — validators handle garbage.
• Agreement. Two implementations (client and server, old and new schema) return the same verdict — the migration check from migrating from Yup to Zod, automated.
• Monotonicity. Adding characters to a too-long value never makes it valid.

When a property fails, fast-check shrinks the input to a minimal counterexample (for example, from a 40-character random string to " "), which usually points straight at the bug.

The core pattern: properties with fast-check

import fc from "fast-check";
import { describe, it, expect } from "vitest";
import { localeNumber } from "../src/locale-number";          // parse/format by locale
import { groupDigits } from "../src/card";                     // "4111111111111111" → "4111 1111 1111 1111"
import { checkEmail } from "../src/email";
import { normaliseName } from "../src/names";

describe("form functions: properties", () => {
  it("number formatting round-trips in every supported locale", () => {
    const locales = ["en-US", "en-GB", "de-DE", "fr-FR", "de-CH", "en-IN"];
    fc.assert(
      fc.property(
        fc.constantFrom(...locales),
        // Money-like values: up to 10^9 with 2 decimals, as integers of cents.
        fc.integer({ min: -100_000_000_000, max: 100_000_000_000 }),
        (locale, cents) => {
          const n = cents / 100;
          const ln = localeNumber(locale, { minimumFractionDigits: 2, maximumFractionDigits: 2 });
          expect(ln.parse(ln.format(n))).toBeCloseTo(n, 2);
        },
      ),
      { numRuns: 500 },
    );
  });

  it("card grouping never loses or adds digits", () => {
    fc.assert(fc.property(
      fc.stringMatching(/^\d{12,19}$/),
      (digits) => { expect(groupDigits(digits).replace(/ /g, "")).toBe(digits); },
    ));
  });

  it("email check never throws, whatever the input", () => {
    fc.assert(fc.property(
      fc.string({ unit: "grapheme", maxLength: 200 }),   // includes emoji and combining marks
      (s) => { expect(() => checkEmail(s)).not.toThrow(); },
    ));
  });

  it("name normalisation is idempotent", () => {
    fc.assert(fc.property(fc.string({ unit: "grapheme" }), (s) => {
      const once = normaliseName(s);
      expect(normaliseName(once)).toBe(once);
    }));
  });

  // Regression seeds: inputs that once failed are always re-run.
  it("known tricky inputs", () => {
    for (const s of [" [email protected]", "[email protected].", "1.234,56", "ÉLODIE"]) {
      expect(() => checkEmail(s)).not.toThrow();
    }
  });
});

Step-by-step walkthrough

1. Pick pure functions with clear invariants. Parsers, formatters, normalisers, validators and schema adapters. Components and effects are poor subjects; test them with example-based tests.
2. State properties, not examples. Round trip, idempotence, never-throws, agreement between implementations. Each property replaces dozens of hand-picked cases.
3. Choose generators that match reality. fc.string({ unit: "grapheme" }) includes emoji and combining characters; fc.stringMatching(regex) produces structured strings; fc.constantFrom(...) enumerates locales or countries.
4. Keep domains honest. Generate only values the function is meant to accept for round-trip properties (valid raw values), and anything at all for never-throws properties.
5. Let shrinking do the diagnosis. A failing case is reduced to something like " " or "0,"; add it to a regression list so it is always re-checked.
6. Pin the seed in CI when debugging. fast-check prints the seed and path of a failure; re-running with them reproduces it exactly.

Why generated inputs find different bugs

Hand-written tests are biased toward the cases the author already had in mind while writing the code — which are the cases the code already handles. Generated inputs have no such bias. They routinely produce strings that humans rarely think to type but real users and systems do produce: no-break spaces from copy-paste, full-width digits from Asian input methods, combining accents, very long values, values at exact boundaries. For form code, which sits exactly at the boundary between unpredictable human input and strict data, that difference is the whole point. A few properties per parser or validator often find more real bugs in an afternoon than a year of example tests.

Where to start in an existing codebase

Adding property tests to a mature form codebase works best in a particular order. Start with never-throws properties on every exported validator and parser: they need no thought about expected outputs, they run quickly, and crashes on odd input are the most damaging bugs because they often take the whole form down. Next add round-trip properties for each formatter and parser pair — number, currency, date, phone, card — since those pairs are where locale and edge-case bugs concentrate. Then add agreement properties wherever two implementations must match: the client and server copies of a rule, or an old and a new schema during a migration. Only then consider more specific invariants. Each step tends to surface a few real bugs; fixing them, and adding the shrunk inputs to the example suite, leaves the codebase measurably more robust within a sprint.

Failure modes and edge cases

1. Properties that restate the implementation

expect(validate(x)).toBe(x.includes("@")) just duplicates the code. Good properties relate functions to each other (round trip, agreement) or state universal truths (never throws, idempotent).

2. Generators that are too narrow

fc.string() without a unit option generates characters from a limited range; use unit: "grapheme" or "binary" to include the characters that cause real bugs.

3. Floating-point round trips

parse(format(n)) === n fails for values the format rounds (more decimals than the format shows). Generate values at the format’s precision — integers of minor units, as above — or compare with a tolerance.

4. Slow properties

Properties that render components or call networks are slow and flaky. Keep them on pure functions; hundreds of runs of a pure parser take milliseconds.

5. Locale data differences

Intl output can differ between Node versions and browsers (for example, which space character a locale uses). Run property tests in the same runtime as production code paths, and treat locale-data differences found this way as real bugs to handle, as the French example shows.

Verification checklist

• Every parser and formatter has a round-trip property.
• Every validator has a never-throws property over arbitrary strings.
• Normalisers have an idempotence property.
• Client and server validators (or old and new schemas) have an agreement property.
• Generators include graphemes such as emoji, no-break spaces and combining marks.
• Shrunk failures are added to a regression list.
• Property tests run on pure functions and finish in seconds.

Frequently Asked Questions

Related

• Testing Form Validation
• Composing Pure Validator Functions
• Locale-Aware Number and Currency Inputs

← Testing Form Validation