The main test suite talks to stripe-mock and to stubbed Square, SumUp and email responses. That makes it fast and deterministic, and it runs on every pull request. It also means the suite can only ever agree with what we think the providers do.
The one class of bug mocks cannot find is drift between our code and the real provider: an API field that changed, a redirect URL the provider rejects, a webhook that arrives in a different shape, a sandbox card page that moved its button. For anything that takes money or sends mail, that class of bug matters most.
e2e-payments/
is a separate harness. For each provider it:
src/index.ts) on a fresh
file-backed database;It is orchestrated by the same Cucumber runner as the main specs.
live-payment-providers.feature
is the human contract. Its scenarios include a refund where the
provider succeeds while the local database write fails, a replayed
SumUp callback with forged ids that must be refused without a read, a
price change while a visitor is mid-checkout, and a complex order of a
package, a member and a plain listing paid in one go.
Every scenario gets its own identity (booker email, listing names, owner credentials, never the defaults), so a leftover from one run can never satisfy an assertion in the next.
The schedule
The workflow runs at 03:17 UTC every night, and can be started by hand. Off the hour, so it does not queue behind every other scheduled job on GitHub.
Each provider is its own leg, with fail-fast: false, so one broken
sandbox does not hide the others.
# It is deliberately NOT a PR gate: it needs
# network to third-party sandboxes and is
# inherently slower/flakier than mocked tests.
# It runs nightly and on demand. Every leg must
# execute: a missing provider secret fails its
# leg.
on:
workflow_dispatch:
schedule:
# 03:17 UTC nightly (off-peak, avoids the
# top-of-hour rush).
- cron: "17 3 * * *"
The harness fills fields by their accessible name and clicks buttons
by their visible text, the way a person would. It never forces a
click and never calls form.submit(). Some headless Chromium builds
stall after a form POST. The harness detects that and falls back to
requestSubmit, which still runs browser validation and the app's own
handlers, and it only does so when a page-side witness proves the
first click never fired. A refund form can never be submitted twice
by the fallback.
A second workflow, email-sandbox-e2e.yml, runs at 03:47 UTC. It
sends real requests through the production email code to Mailgun (EU
and US), Postmark, Resend and SendGrid: one single send with an
attachment and reply-to, and one bulk send with unsubscribe
substitution. Every message goes to a safe address, such as a
provider's discard sink.
The two runs treat a missing secret differently, on purpose:
deno task suite:production runs at 04:23 UTC. It gathers the checks
that ask production-shaped questions and are too slow or too noisy to
gate every pull request:
deno audit --level=critical security report.A schedule-only run has a weakness: a change that breaks it merges
green and fails the next night. The cheapest of those to catch is a
renamed button. So deno task check:e2e-labels runs in every
precommit and checks that every label the payment harness clicks or
asserts is real copy from the message catalog. A copy rename now
fails the pull request that made it, not the next night's run.