type: decision
status: active
timestamp: 2026-06-20
tags: [decisions, architecture, a11y, accessibility, axe, pa11y, lighthouse]

Accessibility — three-tool stack (axe + Pa11y + Lighthouse CI)

axe-core + Pa11y + Lighthouse CI per PR on any new a11y violation in any tool. Each tool catches a different category.

Accessibility — three-tool stack (axe + Pa11y + Lighthouse CI)

Decision

Every site’s per-PR CI runs all three a11y tools in parallel:

axe-core — Deque’s static rule engine, run via @axe-core/playwright.
Pa11y — dynamic runner with the HTMLCS ruleset (and axe runner alongside).
Lighthouse CI — score
- perf budget, posted as a PR comment.

PR fails on any new violation in any of the three tools. The a11y category in Lighthouse CI is required to score 100. All three are free, OSS, no card.

Why

No single tool catches everything. axe is the industry- standard rule engine but Deque’s catalog is finite. Pa11y’s HTMLCS catches a slightly different set of structural / visual rules. Lighthouse CI catches presence-of-best-practice rules Lighthouse alone scores on (e.g. landmarks, lang attribute).
Cost is zero. All three are OSS, run inside the existing GitHub Actions runner against the existing Playwright install. No new vendor, no card.
Different views serve different reviewers. axe + Pa11y print violations (good for a fix-list); Lighthouse posts a PR comment with a score (good for reviewer at-a-glance). Both views matter.
Aligns with the code-quality stack — that decision locks the family on layered tooling (Dependabot + biome + CodeRabbit + Sonarcloud); a11y deserves the same defensive layering.

Implications

Architecture

Three GitHub Actions jobs in each site’s ci.yml:
- a11y-axe — runs Playwright + @axe-core/playwright over the site’s key routes.
- a11y-pa11y — runs pa11y-ci against the Cloudflare Pages preview URL.
- lighthouse-ci — runs treosh/lighthouse-ci-action@v11 and posts the score as a PR comment.
All three jobs depend on the Cloudflare Pages preview deploy job; they can run in parallel after the preview is up.
Tests live under tests/a11y/ per site; the kit ships a baseline spec at @chirag127/oriz-kit/testing/a11y.spec.ts that each site imports + extends with its own routes.
lighthouserc.json per site sets the assertions — a11y minScore 1.0 (required), perf 0.9, best-practices 0.95, SEO 1.0.

Per-PR gating

Any violation fails the PR — no warnings, no exceptions. If a finding is a known false positive, it’s silenced via the tool’s config (axe: disableRules; Pa11y: ignore; Lighthouse: per-rule assertions: off) and a comment in the config explains why.
The Sonarcloud quality gate (from the code-quality stack) treats these as separate jobs — they don’t collapse into one score.

Why fail-on-any vs warn-only

The family’s site count (11+) makes “warn-only” indistinguishable from “off” — no one chases a long warning list across 11 PRs.
All three tools are tunable via config — false positives become documented ignore entries, not silent passes.
Aligns with the never-hit-quotas philosophy applied to quality: fail loudly, never drift silently.

What we don’t do

No paid a11y tools — accessibility-checker (IBM) is OSS but smaller community; WAVE API is paid past its free tier; Axe DevTools Pro is paid. Three free tools cover us.
No manual a11y review as the only check — manual review still happens for keyboard-nav and screen-reader UX (per if it exists in the family rules), but the automated trio is the floor.
No suppressing failures across the board. Tunable per-rule via config files; suppressing globally is not.

Cross-refs

Edit on GitHub · Back to index