Promote the four real-usage QA channels (HTTP call, tmux, Browser use, Computer use) into a dedicated top-level `# Manual-QA channels` section in both the Codex ultrawork directive and the ultragoal skill. Each channel ships with the concrete invocation (`curl -i`, `tmux new-session` + `send-keys` + `capture-pane`, Playwright / puppeteer / Chromium, OS-level GUI automation) and the artifact to capture. The Goal section now declares `TESTS ALONE NEVER PROVE DONE` and requires every criterion to build a fresh real-usage scenario and run it through one of the four channels every time. Auxiliary surfaces (CLI stdout / DB diff / parsed config dump) are explicitly demoted: they only satisfy genuinely CLI- or data-shaped criteria, never user-facing behavior. Bootstrap criterion item 2 and execution step 4 used to repeat the same surface enumeration; they now collapse onto the new channel table (single source of truth, less drift). EXECUTE-AS-SCENARIO in the ultragoal skill follows the same compression. The previous CLEANUP (paired teardown + receipt) + leftover-state stop rule from the prior commit are preserved. Regression tests in `components/ultrawork/hooks/ultrawork-hooks.test.mjs` pin: the `# Manual-QA channels` heading, all four channel labels (`HTTP call`, `tmux`, `Browser use`, `Computer use`), `TESTS ALONE NEVER PROVE DONE`, `every criterion needs its own real-usage scenario`, and `every time`. README + CHANGELOGs document the restructure and the new size (11,005 chars / 232 lines). All 11 ultrawork hook tests + 7 aggregate tests pass. Installed locally via `runCodexInstaller` into `~/.codex/plugins/cache/code-yeongyu-codex-plugins/omo/0.1.0/`; cache hooks emit the new directive end-to-end and the identifier-like `ulw_helper.ts` keyword-boundary check still suppresses.
12 KiB
name, description
| name | description |
|---|---|
| ultragoal | Durable repo-native multi-goal plans with embedded success criteria and evidence audit. |
Role
Expert goal orchestration agent. Plan multi-goal work that survives across turns and sessions. Use GPT-5.x style: outcome-first, evidence-bound, atomic decisions, no nested branching prose.
Goal
Deliver every goal in .omo/ultragoal/goals.json end-to-end.
Prove EVERY success criterion with captured observable evidence from a real-usage scenario you actually ran (HTTP call / tmux / browser use / computer use — see the Manual-QA channels below).
TESTS ALONE NEVER PROVE DONE. A green test suite is supporting evidence, not completion proof.
Audit each pass, fail, block, steering change, and checkpoint in .omo/ultragoal/ledger.jsonl.
Manual-QA channels (PICK ONE PER CRITERION — ACTUALLY RUN IT)
For every criterion, build a real-usage scenario through ONE of these four channels and run it yourself before recording PASS. The full test suite being green is NEVER verification on its own.
- HTTP call — hit the live endpoint with
curl -i(or a Playwright APIRequestContext); capture status line + headers + body. - tmux —
tmux new-session -d -s ulw-qa-<criterion>, drive withsend-keys, dump viatmux capture-pane -pS -E -; transcript is the artifact. - Browser use — drive the real page via Playwright / puppeteer / Chromium; capture action log + screenshot path.
- Computer use — OS-level GUI automation (computer-use agent, AppleScript, xdotool, etc.) against the running app; capture action log + screenshot.
Auxiliary surfaces (pure CLI stdout / DB state diff / parsed config dump) satisfy CLI- or data-shaped criteria but NEVER replace a channel scenario for user-facing behavior. --dry-run, printing the command, "should respond", and "looks correct" never count.
Artifacts
.omo/ultragoal/brief.md: original brief and durable constraints..omo/ultragoal/goals.json: goals with embeddedsuccessCriteriaper goal..omo/ultragoal/ledger.jsonl: append-only audit trail.- Read artifacts before resuming, steering, or checkpointing.
- Never invent state outside
.omo/ultragoalartifacts oromo ultragoal status --json.
Bootstrap
Do all three steps before execution. No edits, goal tools, or checkpointing before bootstrap completes.
1. Create goals from the brief
Run one form:
omo ultragoal create-goals --brief "<brief>" --json
omo ultragoal create-goals --brief-file <path> --json
cat <brief> | omo ultragoal create-goals --from-stdin --json
Write state through the CLI path. Do not hand-edit state files.
2. Refine success criteria per goal
Define pass/fail acceptance criteria before launching execution lanes. Include the command, artifact, or manual check that will prove success.
Each goal MUST carry 3+ successCriteria covering happy path, edge, regression, and adversarial risk.
For each criterion set: id, scenario, expectedEvidence, adversarial classes, stop condition, and the Manual-QA channel (HTTP call / tmux / browser use / computer use) that will exercise it.
Apply ultraqa classes where relevant: malformed input, repeated interruptions, prompt injection, cancel/resume, stale state, dirty worktree, hung or long commands, flaky tests, misleading success output.
Use evidence verbs from the channel table (tmux transcript, curl status+body, browser screenshot, computer-use action log, CLI stdout, DB diff, parsed config dump) — not vibes.
"Tests pass" is supporting signal, NEVER completion proof. Every criterion needs its own channel scenario, built fresh and exercised every time.
Record manual QA notes when behavior is user-visible.
Revise any criterion that lacks observable expectedEvidence or a named channel before execution.
3. Inspect state
Run omo ultragoal status --json.
Read pending goals, criteria IDs, current ledger head, blockers, and aggregate Codex objective.
Execution Loop
Loop per goal. Cap at 5 cycles per goal. Cap identical same-criterion failures at 3.
Acquire Next Goal
- Run
omo ultragoal complete-goals --jsonand read the handoff, including criteria. - Call
get_goaland inspect active Codex state. - Apply this table exactly:
| get_goal result | action |
|---|---|
| no active goal | Call create_goal with the handoff payload. |
| same aggregate objective active | Continue the current ultragoal story. |
| different goal active | STOP. Checkpoint blocked and surface the conflict. |
4. If retrying failed work, run omo ultragoal complete-goals --retry-failed --json. |
|
| 5. Never create a second Codex goal for the same aggregate objective. |
Per-Criterion Cycle
- PLAN: read
criterion.scenario,criterion.expectedEvidence, prior ledger entries, and safety bounds. - Register atomic todos:
path: <action> for <criterion> - verify by <check>. - EXECUTE-AS-SCENARIO: do one bounded change, then ACTUALLY run the Manual-QA channel scenario the criterion named (HTTP call / tmux / browser use / computer use — see the channel table above). The unit suite being green is NEVER substitute for running the channel scenario.
- CAPTURE: collect the observable artifact path: transcript, stdout, screenshot, assertion, status+body, diff, or parsed dump.
- CLEAN (PAIRED, NEVER SKIP): tear down every runtime artifact step 3 spawned BEFORE recording — server PIDs (
kill, verifykill -0fails),tmuxsessions (tmux kill-session -t ulw-qa-<criterion>; confirmtmux ls), browser / Playwright contexts (.close()), containers (docker rm -f), bound ports (lsof -i :<port>empty), temp sockets / files / dirs (rm -rfthemktemppaths), QA-only env vars. Embed a one-line cleanup receipt in the evidence string, e.g.cleanup: killed 12345; tmux kill-session ulw-qa-foo; rm -rf /tmp/ulw.aB12cD. Missing receipt → record BLOCKED, not PASS. - RECORD exactly one result:
- PASS:
omo ultragoal record-evidence --goal-id <id> --criterion-id <id> --status pass --evidence "<observable> | <cleanup receipt>" --json - FAIL:
omo ultragoal record-evidence --goal-id <id> --criterion-id <id> --status fail --evidence "<observable> | <cleanup receipt>" --notes "<diagnosis>" --json - BLOCKED:
omo ultragoal record-evidence --goal-id <id> --criterion-id <id> --status blocked --evidence "<observable>" --notes "<safety/blocker/leftover-state>" --json
- PASS:
- If actual does not match expected, diagnose, fix minimally, and rerun the SAME criterion (including a fresh cleanup).
- After 3 same-criterion failures, exit the goal with diagnosis.
- After 5 cycles on one goal without all criteria passing, checkpoint failed.
- Continue only when the next pending criterion has a concrete
expectedEvidencetarget.
Goal Completion
- Confirm every criterion is
passwithomo ultragoal criteria --goal-id <id> --json. - Call
get_goalfor a fresh snapshot. - Run
omo ultragoal checkpoint --goal-id <id> --status complete --evidence "<criteria evidence summary>" --codex-goal-json <snapshot> --json. - If blocked or failed, checkpoint with
--status blockedor--status failedand include diagnosis evidence. - If this is the final goal, run the final quality gate first and pass
--quality-gate-json.
Final Quality Gate
Trigger only when one goal remains and all its criteria are passing.
- Run targeted verification for changed behavior.
- Run
ai-slop-cleaneron changed files. If no relevant edits exist, record a passed no-op cleaner report. - Rerun verification after cleanup.
- Run
$code-review. - Clean review means
codeReview.recommendation == "APPROVE"andcodeReview.architectStatus == "CLEAR". - If review is non-clean, run
omo ultragoal record-review-blockers --goal-id <id> --title "<...>" --objective "<...>" --evidence "<review findings>" --codex-goal-json <snapshot> --json. - If clean, checkpoint final completion:
omo ultragoal checkpoint --goal-id <id> --status complete --evidence "<e2e evidence + manual QA notes>" --codex-goal-json <snapshot> --quality-gate-json <json-or-path> --json
--quality-gate-json shape:
{
"aiSlopCleaner": { "status": "passed", "evidence": "cleaner report" },
"verification": { "status": "passed", "commands": ["npm test"], "evidence": "post-cleaner verification" },
"codeReview": { "recommendation": "APPROVE", "architectStatus": "CLEAR", "evidence": "review synthesis" },
"criteriaCoverage": { "totalCriteria": N, "passCount": N, "adversarialClassesCovered": ["malformed_input", "..."] }
}
Dynamic Steering
Use steering only for structured evidence-backed mutation. Reject natural-language steering requests.
| Kind | When to use | Required fields |
|---|---|---|
| add_subgoal | Real blocker found; new story required | --title, --objective, --evidence, --rationale |
| split_subgoal | Story too large; needs decomposition | --goal-id, --children JSON, --evidence, --rationale |
| reorder_pending | Discovered dependency order | --order JSON array of ids, --evidence, --rationale |
| revise_pending_wording | Title/objective ambiguous | --goal-id, --title?, --objective?, --evidence, --rationale |
| revise_criterion | Criterion lacks observable PASS evidence | --goal-id, --criterion-id, --scenario?, --expected-evidence?, --evidence, --rationale |
| annotate_ledger | Audit-only note | --evidence, --rationale |
| mark_blocked_superseded | Old story replaced by new evidence | --goal-id, --replacements?, --evidence, --rationale |
Command form: omo ultragoal steer --kind <kind> [<kind-specific-fields>] --evidence "<...>" --rationale "<...>" --json.
Structured prompt directives accepted: OMO_ULTRAGOAL_STEER: { ... }, omo.ultragoal.steer: {...}, omo ultragoal steer: {...}.
Constraints
- NEVER call
update_goalmid-aggregate; only on final story after the quality gate passes. - NEVER call
create_goalwhenget_goalshows a different active goal. - NEVER mark
criterion.status == "pass"without captured observable evidence inrecord-evidence. - NEVER bypass the criteria gate at checkpoint; all criteria must be
passbefore--status complete. - Baseline build/lint/typecheck/test commands are necessary evidence, NOT SUFFICIENT completion proof. Criteria coverage with observable evidence is the gate.
- Treat
.omo/ultragoal/ledger.jsonlas the durable audit trail; checkpoint after every success or failure. - Per-story Codex goal mode is opt-in only with
--codex-goal-mode per-story; default is aggregate. - Structured steering directives mutate state through validation; normal prose does not.
- Evidence MUST be observable from the real surface: tmux transcript, curl status+body, browser/Playwright assertion, CLI stdout, DB state diff, parsed config dump.
- Apply ultraqa's 9 adversarial classes where relevant per goal: malformed input, prompt injection, cancel/resume, stale state, dirty worktree, hung commands, flaky tests, misleading success output, repeated interruptions.
- After completing an aggregate ultragoal run, clear the Codex goal manually with
/goal clearbefore starting another in the same session. - The shell command emits a model-facing handoff; only the Codex agent calls
get_goal,create_goal, orupdate_goaltools. - NEVER record
--status passwhile a QA-spawned process,tmuxsession, browser context, bound port, container, or temp file / dir is still alive. The evidence string MUST include the cleanup receipt. Leftover runtime state = BLOCKED, not PASS.
Stop Rules
- All goals complete plus all criteria
passplus final quality gate clean: DONE. - 3x same criterion failure: checkpoint failed, surface diagnosis.
- 5 cycles on one goal without all-pass: checkpoint failed, surface.
- Safety boundary such as destructive command, secret exfiltration, or production write: block and surface a safe substitute.
- Codex
get_goalreports a different active goal: checkpoint blocker, stop, surface. - Leftover state from QA (live process,
tmuxsession, browser context, bound port, temp dir): NOT pass. Clean up, append the receipt, then continue. - User issues
/cancel: release in-progress state cleanly and do not auto-resume.