Testing and production readiness

Define evidence before implementing a release. Unit tests, seeded screens, browser checks, and government operational approval answer different questions.

Current evidence and limits

The repository contains Bun unit tests, opt-in database integration tests, Playwright journeys, and performance instrumentation. CI uses temporary PostgreSQL 18 and runs typecheck, lint, format, migrations, seed, and Bun tests with database integration enabled. Build and browser tests are not currently part of that CI workflow because they need configured identity keys.

The implementation-status document reports 537 tests and 93 runnable browser cases for a September 14 review, with two Clerk-provisioning checks skipped and some latency work open. Treat these as historical results. They do not prove the October 5 source or the new production build passes. This pack does not claim a fresh full functional/security test run.

Sources: CI workflow notes, dated implementation status, performance method, browser configuration. Run only the appropriate checks on approved isolated test targets.

Requirements-to-evidence matrix

Requirement / risk	Method and concrete cases	Acceptance evidence / owner
Correct policy and calculations	Independent approved cases for each supported algorithm/policy, household boundary, rounding, thresholds, education, proration days 1/30/31, self-employment zero/profit/loss and tied scenarios.	Policy owner signs expected results. Engineer matches every supported result. Workbook anomaly is explicitly accepted or corrected.
Current checks/results	Edit after validation, edit after Final Results, rebind policy, reset/undo stale final, unsupported version.	Old results cannot satisfy current transition guards. QA records rejection and correct recovery.
Authorized work only	Six-role matrix, cross-team/program direct URLs and APIs, forged request actor/role, export/download keys, disabled account and expired delegate.	Every forbidden path denies access without revealing protected data. Security/QA sign the matrix.
Reliable saves	Two sessions on one version, repeated request ID, retry after uncertain response, rejected command, server restart and interrupted network.	One accepted effect per request, no silent overwrite, clear conflict recovery and preserved approved evidence.
Complete workflow	New source batch → activation → examiner → supervisor return/correction → approved non-drop → PEC → memo/response → release. Separate manager-approved drop.	A new review completes without manipulating DB or seeded state. QC owner signs each role action.
Correct intake and refresh	Exact layout/header, quoted pipes, empty columns, individual rows, invalid IDs, recycled number/FFY, unknown staff, refresh protected fields and dropped records.	Rejected data is quarantined, source lineage retained, identities isolated and accepted totals reconciled.
Documents and correspondence	Malformed/oversized PDFs, wrong owner/review, malware quarantine, expired URL, concurrent attachment, issued template/facts, duplicate delivery and amendment.	Unsafe files are blocked. Issuance and receipt prove the actual approved delivery event.
Reports and exports	Released/non-released/disputed/drop states, FFY rollover, rebuilt projection, frozen batch, padded codes, format rejection and receipt lookup.	Zero unexplained counts/field mismatches. Statistics validates receiving-system acceptance before replacing that route.
DCA when separately built	Repeated handoff, memo revision change, returns/resubmissions, deadline extension, dispute, closure/reopen, unauthorized decision and Access conflict.	QC and DCA owners reconcile actions, source evidence, queues and closure.
Recovery and records	Restore DB plus objects, missing object, retained/legal-hold artifacts, failed migration, audit admin rights and safe disposal.	IT/records prove agreed recovery and retention behavior without losing required decisions.
Accessibility and task fit	Keyboard-only dense forms, focus/validation, screen reader labels, dialog escape/return focus, zoom, contrast and reduced motion.	Independent manual checks plus user task completion. Accessibility owner approves selected criteria.
Internal performance and load	Representative retained dataset, 12 editors assumed, burst of 41 staff, month-end report/export/import and worker failures.	Record throughput/errors and p50/p95/p99 from enough samples. No accepted-write loss. Operations signs targets.

Independent expected cases

A golden case has an approved input and expected output calculated outside the code being tested. A test that calls the same calculation to produce its expected answer cannot validate that calculation. Policy SMEs provide the source table, rule explanation, expected benefit/flags, effective date, and known exception. QA tracks rule coverage and unexpected differences.

Keep examples synthetic until the real-data test environment is approved. If historic cases are used later, define access and anonymization controls. Tests and traces can contain SPII. They require the same handling decisions as records. Do not put session state or trace files in the repository.

Performance targets and measurements

Current demo targets are below 1,500 ms local cold component readiness and 500 ms navigation. Simulated internet targets are 2,500 ms and 1,000 ms. They are code budgets, not performance claims. Simulation does not establish latency on the agency network. Current payload budgets are in budgets.ts.

For the internal production proposal, start with p95 below 2 seconds for routine queue/search/workspace readiness and below 1 second for accepted ordinary saves, under the agreed representative load. Exclude large batch jobs from those action targets and give them separate progress/completion targets. These thresholds need operations approval and baseline measurements.

Use warm and cold entry, different roles, realistic document/Activity history, peak month-end work, and all 41 accounts making a burst. Sample enough requests to support percentile claims. Record hardware/service size, dataset, concurrency, network, revision and failed samples. A 4,800-review seed is a reproducible fixture, not a capacity ceiling or proof of live volume.

Security and accessibility assessment

Security chooses the agency control baseline and threat model. Use OWASP ASVS to define verifiable controls and an independent assessment scope. Include internal threats, privileged users, object retrieval, exports, raw intake, backups and support tooling. Being reachable only internally does not establish access control.

Use WCAG 2.2 as a proposed accessibility test reference with the agency accessibility owner. Confirm the binding agency criteria before stating compliance. Automated scans are useful evidence but do not replace manual keyboard, assistive-technology, or staff task tests.

Release gates

Gate	Required proof	Approver
G1. Requirements ready	Signed defect map/workflow, sources, field/rule authority, scope and staff commitments.	QC product/policy owner and sponsor.
G2. Platform ready	Agency identity/access matrix, internal deployment, secret/environment separation, scan flow and initial restore.	IT/security and engineering.
G3. Functional slice ready	Golden calculations plus a newly imported non-drop/drop journey, corrections, memo/response and reconciled reports.	QA and QC operations.
G4. Real-data pilot ready	Required security/privacy/records/accessibility dispositions, backup/recovery, monitoring, support and controlled release.	Named agency approving officials and service owner.
G5. Pilot accepted	No unexplained reconciliation differences, resolved critical defects, acceptable task outcomes and measured benefit/support findings.	QC sponsor and service owner.
G6. Cutover accepted	Final delta/authority reconciliation, rollback plan, legacy evidence retention, two-maintainer handover.	Operations/IT and sponsor.

A federal export or DCA integration has its own receiving-owner gate. If that gate is open, keep the legacy route for that function and state the limit. No waiver should be hidden in a generic all tests passed statement.

Test execution and defect handling

Link every approved requirement to cases, owners, and evidence location. Keep expected results independent.
Run fast pure checks on each change. Run database/contract checks against disposable environments.
Run build, browser, access and relevant rule tests before release. Add a focused regression test when a material defect demonstrates an untested risk.
Have staff run UAT scenarios using their real role and approved test identity. Test source-to-output paths and error recovery.
Record defect severity, affected requirement, safe workaround, owner and retest evidence. Security and policy defects need their respective owners.
Publish a release evidence summary with revision, environment, dataset, checks passed, failures/skips, known limits and signed gate decisions.

Suggested release evidence record

Field	Required content
Release and environment	Commit/artifact ID, schema/policy/template versions, internal environment and build date.
Scope and cases	Included programs/functions, requirement IDs, expected cases and tested roles.
Results	Passed/failed/skipped cases and exact evidence references. No invented aggregate.
Open defects	Severity, owner, mitigation, retest and approved disposition if applicable.
Operations	Migration/restore drill, monitoring, support contacts, rollback triggers and data authority.
Approvals	Named QA, policy, security/records, service owner and sponsor decisions as required.

A documentation check validates this wiki. It does not satisfy the production gates above.