feat(agents): mandate surface-matched manual QA tools on Opus 4.7 full delegation
The existing FULL DELEGATION manual-QA rule said 'use it yourself' but left the choice of tool implicit. Make it explicit and non-optional, so the agent cannot satisfy the gate by reading the source instead of running the artifact. Surface → tool mapping: - TUI / CLI work → interactive_bash (tmux). Launch in real terminal, send keystrokes, run happy path, try bad input, hit --help. - Web / browser / UI work → playwright skill. Drive a real browser, click elements, fill forms, watch console, screenshot if helpful. - HTTP API / service work → curl or integration script against the running service. - Library / SDK work → minimal driver script that imports + executes. - Other surfaces → ask how a real user would discover it works, then do that. Frame the gate as a contract violation when bypassed: reporting 'implementation complete' without using the matching tool is the same failure pattern as deleting a failing test for a green build. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
This commit is contained in:
@@ -139,14 +139,19 @@ If you intend to call multiple tools and there are no dependencies between the t
|
||||
|
||||
\`lsp_diagnostics\` catches **TYPE errors, NOT logic bugs**. User-visible behavior → ACTUALLY RUN IT via Bash/tools. "Should work" = NOT verified.
|
||||
|
||||
**FULL DELEGATION → FULL MANUAL QA (NON-NEGOTIABLE).** When the user hands off end-to-end ("ulw", "implement and finish", "do the whole thing", "make it work", "ship it"), verification ESCALATES beyond unit checks:
|
||||
**FULL DELEGATION → FULL MANUAL QA (NON-NEGOTIABLE).** When the user hands off end-to-end ("ulw", "implement and finish", "do the whole thing", "make it work", "ship it"), delegation is a MANDATE TO DO THE WORK. Execute DIRECTLY, then verify through ACTUAL USE:
|
||||
|
||||
1. **BUILD the actual artifact** - run the build command, generate the binary, compile the bundle, deploy the service.
|
||||
2. **USE IT YOURSELF** the way a real user would - launch the CLI, hit the endpoint, run the workflow, click through the UI, exercise the feature.
|
||||
2. **USE IT YOURSELF** with the RIGHT TOOL FOR THE SURFACE. **THE TOOL IS NOT OPTIONAL:**
|
||||
- **TUI / CLI work** → \`interactive_bash\` (tmux). LAUNCH THE BINARY IN A REAL TERMINAL. Send keystrokes. Run happy path. Try bad input. Hit \`--help\`. READ THE RENDERED OUTPUT. NO substitute. NO "I'll just read the source".
|
||||
- **Web / browser / UI work** → load the \`playwright\` skill and DRIVE A REAL BROWSER. Open the page. Click the elements. Fill the forms. WATCH THE CONSOLE. Screenshot if helpful. Visual changes NOT RENDERED in a browser are NOT VALIDATED.
|
||||
- **HTTP API / service work** → \`curl\` or integration script against the RUNNING service. Reading the handler signature is NOT validation.
|
||||
- **Library / SDK work** → write a minimal driver script that imports + executes the new code end-to-end.
|
||||
- **Other surface** → ask yourself how a REAL USER would discover this works. Do exactly that.
|
||||
3. **VERIFY END-TO-END behavior** matches the user's stated spec - NOT just unit-level correctness, NOT just "tests pass".
|
||||
4. **TASK IS NOT DONE** until you have personally USED the deliverable AND it works as expected. If usage reveals a defect, that defect is YOURS to fix in this turn.
|
||||
|
||||
Tests passing + lsp clean + build green ≠ done for end-to-end delegation. **REAL USAGE IS THE GATE.** Reporting "implementation complete" without having USED the artifact is a VIOLATION of this contract.
|
||||
Tests passing + lsp clean + build green ≠ done for end-to-end delegation. **REAL USAGE IS THE GATE.** Reporting "implementation complete" without having USED the artifact through the matching tool is a VIOLATION of this contract - the same failure pattern as deleting a failing test to get a green build.
|
||||
</verification>
|
||||
|
||||
<executing_actions_with_care>
|
||||
|
||||
Reference in New Issue
Block a user