feat/l-1-flow-scaffolds #2
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/l-1-flow-scaffolds"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Show previous round
23 scaffold TOML flows added. Four actionable findings: (1) a systematic
storage_state.pathbug that will silently break all authenticated flows because../../escapes the repo root; (2) fiveexpect_textsteps using literalTODO_*values that will produce permanent CI failures, contrary to theexpect_no_errorspass-through convention used everywhere else in this PR; (3) two OAuth callback URLs that trigger real or crash-prone backend calls rather than clean test failures; and (4) a data-leak verification step whose action does not match its stated acceptance criterion.Findings (5)
[MAJOR]
storage_state.pathuses../../— likely escapes repo root across all authenticated flowsflows/04-parent-child-handoff.toml:22
Every flow that declares
[storage_state]usespath = "../../mock-data/pwa/playwright-storage/<file>.json"(same pattern: flows/05 line 20, 06 line 20, 07 line 21, 08–09 line 21, 10–16 line 21, and likely 17–20). If the FlowSpec runner resolvespathrelative to the TOML file location (flows/), then../../ascends two directory levels — one above the repo root — and the file will never be found. If resolved relative to CWD (repo root),../../still escapes the repo. The correct relative path fromflows/to a siblingmock-data/directory is../mock-data/…; from repo root it is simplymock-data/…. Either (a) correct every occurrence topath = "../mock-data/pwa/playwright-storage/<file>.json", or (b) add a note to CONTRIBUTING.md documenting the runner's path-resolution rule and adjust the prefix accordingly. Without this fix, every authenticated flow silently starts without Playwright storage state and runs as an unauthenticated user, making all auth-dependent assertions misleading rather than correctly failing.[MAJOR]
expect_textsteps withTODO_*literal text values will always fail in CIflows/02-oauth-google-signin.toml:43
Five flows use
action = "expect_text"withtextvalues the application will never render: flows/02 line 43 ("TODO_HOME_GREETING"), flows/03 line 38 ("TODO_HOME_GREETING"), flows/05 line 33 ("TODO_TASK_LIST_HEADER"), flows/07 line 41 ("TODO_LOCKED_STATE_MESSAGE"), flows/09 line 39 ("TODO_TEST_MESSAGE"). The established scaffold convention throughout this same PR isaction = "expect_no_errors"as a no-op pass-through (e.g., flows/06 lines 30–33, flows/08 lines 32–41, flows/10 lines 30–35). Change each of these five steps toaction = "expect_no_errors"and remove theselectorandtextkeys until the real assertion target is known, or addskip = trueat the step level if the FlowSpec schema supports it.[MINOR]
code=TEST_PLACEHOLDERin Google OAuth callback URL triggers a real outbound token-exchange callflows/02-oauth-google-signin.toml:35
url = "/oauth/callback/google?code=TEST_PLACEHOLDER"will cause the identity-service backend to attemptPOST https://oauth2.googleapis.com/tokenwith a syntactically plausible but rejected code. This is not a test-assertion failure — it is an outbound network request that (a) may hang for tens of seconds in CI environments with restricted egress to Google, blocking the entire run, and (b) increments Google API error-rate quota on every execution. Add a prerequisite step that registers an HTTP mock for the identity-service's Google token endpoint (e.g.,action = "register_mock"if FlowSpec supports it), or addskip = true/enabled = falseto this step until the mock layer is wired. Document which approach the FlowSpec supports so flows/03 can follow the same pattern for Apple.[MINOR]
id_token=TODO_MOCK_JWSwill cause a synchronous JWS parse failure in the identity-service handlerflows/03-oauth-apple-signin.toml:31
url = "/oauth/callback/apple?id_token=TODO_MOCK_JWS"passes a value that JWT/JWS libraries reject before the HTTP handler can write a response:TODO_MOCK_JWSis not a valid base64url-encoded compact serialization. Unlike the Googlecodeexchange (which fails at the token-endpoint call), Appleid_tokenvalidation is synchronous in the request path. If the identity-service does not wrap Apple JWS parsing in a recovery middleware, the handler may panic or close the connection without writing an HTTP status, causing the FlowSpec runner to see a connection-reset rather than a clean 4xx. Apply the same mock-registration orskip = trueguard described for flows/02 line 35.[MINOR]
expect_no_errorsdoes not verify DOM-level parent-data isolation as the acceptance criterion requiresflows/04-parent-child-handoff.toml:52
The acceptance criterion on line 11 states "zero parent-data leakage post-handoff". The step at lines 51–52 (
action = "expect_no_errors") only detects JS console errors; parent data rendered in the child session's DOM — for example, the adult-1 greeting"Cześć, Anna"asserted on line 34 — would pass this step silently. If FlowSpec supportsexpect_not_text, add a step immediately after line 52 withaction = "expect_not_text",selector = "body",text = "Cześć, Anna". Ifexpect_not_textis not yet implemented, replace the stepnamewith a comment that records the exact selector and negated string (e.g.,# TODO: expect_not_text selector=body text="Cześć, Anna") so this placeholder is distinguishable from the genericexpect_no_errorsscaffolds elsewhere.round 1 • 2026-05-14T19:49:12.765Z → 2026-05-14T19:54:50.963Z
im2be-pr-reviewer — round 2
Reviewing PR #2 in
affinity-intelligence-rework/im2be-qa-rig(head
ee3f5c5, basemain).No new findings this round.
_Model output was not valid JSON. First 200 chars: Based on the PR diff provided (the truncated portion beyond flow 13 is not accessible for direct file reads — findings are scoped to the visible diff only), here is the review output:
{ "summary": "R_
round 2 • 2026-05-14T20:46:09.941Z → 2026-05-14T20:49:53.136Z