e2e coverage pattern β UI change β committed e2e spec β
Why this exists β
Features are built by one agent per PR, but consolidated deploys run later via a separate agent. The author's knowledge of "how do I know this works" is lost at the handoff. #1371 is the canonical failure: PR #1353 replaced the native Fiori value help on the Missions path-item tutorial field with a custom dialog; PR #1366 (a day later) widened the native value help that #1353 had stopped using β so #1366 was dead code on that field. Every gate passed (#1353's unit test was a static grep, #1366's CDS compiled, the bundle gate confirmed bytes shipped, smoke didn't open the picker). Nobody caught it until a human tried to use it in PROD.
Root pattern: every gate checked "did the artifact deploy," never "does the feature work for a user." A per-PR unit test cannot catch a cross-PR seam; only a committed e2e spec that drives the deployed UI can.
The pattern β
- When you change user-facing UI (
app/admin/**,app/**/webapp/**,hugo/layouts/**,hugo-apps/**), add or update an e2e spec undertest/e2e/that exercises the changed surface against the deployed env. - A PR advisory check (
.github/workflows/e2e-coverage-nudge.yml) reminds you if you changed UI without touchingtest/e2e/. It is advisory β it never blocks merge (a hard gate would just breed no-op specs). - The existing post-DEV-deploy e2e CI job (
deploy.ymljobe2e,needs: [deploy, smoke-test]) runs thetest/e2e/specs against the deployed DEV env. That job β not the nudge β is where coverage is actually proven, because the specs drive the real UI.
Writing an e2e spec β
- Templates: the existing specs in
test/e2e/(admin-shell.test.js,tutorial-serve.test.js, β¦) andtest/e2e/README.md. - Specs self-skip when
SMOKE_BASE_URL/ credentials are absent (describe.skipIf(!hasBaseUrl() || !hasCredentials())), sonpm testis never affected β they only run post-deploy where those env vars are set. - Auth uses
SMOKE_TECH_USER/SMOKE_TECH_PASSWORDvia the approuter. - Assert on observable behavior, not code shape: e.g. open a control, assert the rendered columns/rows a user sees. (A static string check of the source is what let #1366 through.)
PROD β
PROD deploy is unchanged: blue-green + smoke. The operator confirms DEV e2e is green before promoting to PROD. There is no automated DEVβPROD e2e coupling by design (avoids brittle cross-env status-plumbing).
Not covered yet (follow-ups) β
- A concrete e2e spec for the Missions path-item picker (#1371 flow: open the Task value help β assert Title/Slug/Tag/Legacy ID columns β slug search finds a known row like
cp-aibus-dox-*). - Backfill for other critical admin flows.