From c3b98c4bfb936cfcc712f04dd2e2b012fff7a3a1 Mon Sep 17 00:00:00 2001 From: YeonGyu-Kim Date: Mon, 27 Apr 2026 21:38:49 +0900 Subject: [PATCH] feat(agents): mandate surface-matched manual QA tools on Opus 4.7 full delegation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The existing FULL DELEGATION manual-QA rule said 'use it yourself' but left the choice of tool implicit. Make it explicit and non-optional, so the agent cannot satisfy the gate by reading the source instead of running the artifact. Surface → tool mapping: - TUI / CLI work → interactive_bash (tmux). Launch in real terminal, send keystrokes, run happy path, try bad input, hit --help. - Web / browser / UI work → playwright skill. Drive a real browser, click elements, fill forms, watch console, screenshot if helpful. - HTTP API / service work → curl or integration script against the running service. - Library / SDK work → minimal driver script that imports + executes. - Other surfaces → ask how a real user would discover it works, then do that. Frame the gate as a contract violation when bypassed: reporting 'implementation complete' without using the matching tool is the same failure pattern as deleting a failing test for a green build. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- src/agents/sisyphus/claude-opus-4-7.ts | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/src/agents/sisyphus/claude-opus-4-7.ts b/src/agents/sisyphus/claude-opus-4-7.ts index 6907a9181..dd3febded 100644 --- a/src/agents/sisyphus/claude-opus-4-7.ts +++ b/src/agents/sisyphus/claude-opus-4-7.ts @@ -139,14 +139,19 @@ If you intend to call multiple tools and there are no dependencies between the t \`lsp_diagnostics\` catches **TYPE errors, NOT logic bugs**. User-visible behavior → ACTUALLY RUN IT via Bash/tools. "Should work" = NOT verified. -**FULL DELEGATION → FULL MANUAL QA (NON-NEGOTIABLE).** When the user hands off end-to-end ("ulw", "implement and finish", "do the whole thing", "make it work", "ship it"), verification ESCALATES beyond unit checks: +**FULL DELEGATION → FULL MANUAL QA (NON-NEGOTIABLE).** When the user hands off end-to-end ("ulw", "implement and finish", "do the whole thing", "make it work", "ship it"), delegation is a MANDATE TO DO THE WORK. Execute DIRECTLY, then verify through ACTUAL USE: 1. **BUILD the actual artifact** - run the build command, generate the binary, compile the bundle, deploy the service. -2. **USE IT YOURSELF** the way a real user would - launch the CLI, hit the endpoint, run the workflow, click through the UI, exercise the feature. +2. **USE IT YOURSELF** with the RIGHT TOOL FOR THE SURFACE. **THE TOOL IS NOT OPTIONAL:** + - **TUI / CLI work** → \`interactive_bash\` (tmux). LAUNCH THE BINARY IN A REAL TERMINAL. Send keystrokes. Run happy path. Try bad input. Hit \`--help\`. READ THE RENDERED OUTPUT. NO substitute. NO "I'll just read the source". + - **Web / browser / UI work** → load the \`playwright\` skill and DRIVE A REAL BROWSER. Open the page. Click the elements. Fill the forms. WATCH THE CONSOLE. Screenshot if helpful. Visual changes NOT RENDERED in a browser are NOT VALIDATED. + - **HTTP API / service work** → \`curl\` or integration script against the RUNNING service. Reading the handler signature is NOT validation. + - **Library / SDK work** → write a minimal driver script that imports + executes the new code end-to-end. + - **Other surface** → ask yourself how a REAL USER would discover this works. Do exactly that. 3. **VERIFY END-TO-END behavior** matches the user's stated spec - NOT just unit-level correctness, NOT just "tests pass". 4. **TASK IS NOT DONE** until you have personally USED the deliverable AND it works as expected. If usage reveals a defect, that defect is YOURS to fix in this turn. -Tests passing + lsp clean + build green ≠ done for end-to-end delegation. **REAL USAGE IS THE GATE.** Reporting "implementation complete" without having USED the artifact is a VIOLATION of this contract. +Tests passing + lsp clean + build green ≠ done for end-to-end delegation. **REAL USAGE IS THE GATE.** Reporting "implementation complete" without having USED the artifact through the matching tool is a VIOLATION of this contract - the same failure pattern as deleting a failing test to get a green build.