Commit Graph

125 Commits

Author SHA1 Message Date
YeonGyu-Kim 20efb79fbf test(runtime-fallback,prompt-gate): add red tests for event-shape and finish-marker blind spots (BUG-C+D)
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-20 13:12:06 +09:00
YeonGyu-Kim 847a8db2dc fix(runtime-fallback): preserve accepted pending retries 2026-05-20 11:42:46 +09:00
YeonGyu-Kim bcea4a9d28 fix(prompt-gate): harden sync and team prompt dispatch 2026-05-19 19:18:13 +09:00
YeonGyu-Kim 1492bffd20 fix(prompt-gate): harden internal prompt dispatch 2026-05-19 19:18:13 +09:00
YeonGyu-Kim 44ef5dec1d fix(runtime-events): honor OpenCode progress shapes 2026-05-19 13:48:45 +09:00
YeonGyu-Kim 67bd324987 fix(runtime-fallback): keep pending retry state on gate skip 2026-05-19 13:16:39 +09:00
YeonGyu-Kim 450c1f9e50 fix(runtime-fallback): ignore stale assistant errors during fallback wait
When OpenCode reports the original assistant error after a fallback retry has already been accepted, keep waiting for the pending fallback model instead of clearing the awaiting flag.

This prevents a duplicate stale session.error from advancing the fallback chain and dispatching a second assistant retry prompt.
2026-05-18 17:48:04 +09:00
YeonGyu-Kim 9ddf13107d fix(test/runtime-fallback): add git_master to config fixture
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-18 16:53:13 +09:00
YeonGyu-Kim 0f8fc548f1 fix(runtime-fallback): consume delegated bootstrap retry payload
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-18 13:42:57 +09:00
YeonGyu-Kim 989ab7171d refactor(hooks): use unified internal prompt dispatch
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-17 17:07:35 +09:00
YeonGyu-Kim ba648685d4 fix(runtime-fallback): carry delegated system and tools through bootstrap retry
When the first prompt fails before any durable user message persists,
runtime fallback retry was rebuilding the request from parts alone and
losing the delegated agent system prompt and tool gates. Now it threads
bootstrap.system and bootstrap.tools into the retry body alongside the
captured retry parts, so the retried prompt keeps the same scope as the
initial delegate launch.
2026-05-17 00:08:30 +09:00
YeonGyu-Kim 982fa81367 fix(delegate-task): start child prompts reliably
Preserve delegated child prompt/bootstrap metadata for early runtime fallback before OpenCode has persisted the first user turn. Bind prompt gate calls to the SDK session receiver and keep completed background task lookup visible across plugin manager instances.
2026-05-16 16:27:56 +09:00
YeonGyu-Kim 3e9b125fd5 test(runtime-fallback): cover localized provider errors 2026-05-15 22:25:20 +09:00
wjiuxing 149a83d703 feat: add Chinese quota patterns to classifyErrorType 2026-05-15 22:22:21 +09:00
wjiuxing adfa8befbb feat: add Chinese error patterns to RETRYABLE_ERROR_PATTERNS 2026-05-15 22:21:06 +09:00
Ravi Tharuma d9033d73ae fix(runtime-fallback): keep variant in equivalence
Treat parsed variant as part of runtime-fallback model equivalence so variant-only fallback hops remain distinct while preserving the existing Claude-family alias handling.

Constraint: Oracle verification flagged unresolved PR #3322 review concerns about variant equivalence and remote state
Rejected: Preserve provider identity in equivalence | contradicted the original live-loop fix for equivalent Claude aliases
Confidence: medium
Scope-risk: narrow
Directive: Any future equivalence broadening must prove both live retry-loop behavior and variant/provider semantics with targeted tests before merging
Tested: bun run typecheck
Tested: bun test src/hooks/runtime-fallback/index.test.ts src/hooks/runtime-fallback/error-classifier.test.ts src/plugin/event.model-fallback.test.ts
Not-tested: Full live end-to-end repro across all provider redundancy policies
2026-05-15 22:05:27 +09:00
Ravi Tharuma f501c47c49 fix(runtime-fallback): skip equivalent claude aliases
Prevent runtime fallback from cycling through provider aliases that resolve to the same underlying Claude family model. This keeps retry handling moving toward a genuinely distinct fallback model instead of appearing to fallback while staying on the same effective model.

Constraint: Live retry/fallback bug is in /Users/ravi/Code/personal/oh-my-opencode, while oh-my-openagent contribution work remains isolated to /Users/ravi/Code/forks/oh-my-openagent
Rejected: Change fallback chain precedence (category vs agent) first | lower-confidence root cause than equivalent-model retry
Confidence: high
Scope-risk: narrow
Directive: Keep alias-equivalence logic limited to model families that are intentionally interchangeable for runtime failover, and expand with targeted tests before broadening provider-family collapsing
Tested: bun run typecheck
Tested: bun test src/hooks/runtime-fallback/index.test.ts src/hooks/runtime-fallback/error-classifier.test.ts src/plugin/event.model-fallback.test.ts
Not-tested: Full live end-to-end session repro against external provider outages
2026-05-15 22:05:00 +09:00
YeonGyu-Kim 2bd4944bad fix(prompt-gate): scope reservation releases 2026-05-15 21:52:29 +09:00
Ivan Smetanin 3199bd3d90 fix(runtime-fallback): broaden watchdog progress detection + harden test timing
Addresses two issues identified by cubic on PR #3952.

1. Watchdog cancellation was too narrow — only `text`/`reasoning` parts
   counted as progress, so a subagent that immediately ran tools
   (Read/Bash/Edit) emitted `tool`/`tool_use`/`tool_result`/`tool-call`/
   `step-start` parts that the watchdog ignored, risking a false fire
   on actively-working subagents. Broaden to: any assistant part of any
   known type counts as progress (the model has started responding,
   whether or not visible text has arrived yet). `info.error` and
   `info.finish` continue to cancel.

2. Test timing margins were tight (15ms pre-cancel against a 40ms
   timer), risking CI flakiness on loaded runners. Bumped to a 100ms
   threshold with a 40ms pre-cancel window and a 250ms post-fire wait,
   giving a 60ms margin before the timer fires and ~2.5x the threshold
   after — robust against scheduler delay.

Refactor for testability: extracted the OpenCode-event→watchdog-signal
translation out of `hook.ts` into an exported `observeEventForWatchdog`
helper on the watchdog module. This let me add direct unit tests for
every part-type case (text, reasoning, tool, tool_use, tool_result,
tool-call, step-start, file) plus the error/finish/empty-parts branches
without spinning up the full hook. Net diff: hook.ts shrinks, watchdog
module gains a small pure function with parametrised coverage.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 20:47:03 +09:00
Ivan Smetanin a130fa70d1 fix(runtime-fallback): add first-prompt watchdog for stuck subagents
When a subagent is dispatched to a provider and the underlying SDK
enters a silent internal retry loop on a 429/quota error, no error
event is ever emitted back to OpenCode. The runtime-fallback hook —
which is fully reactive (listens to message.updated/session.error/
session.status) — has nothing to react to and never dispatches the
configured fallback. The subagent sits in `retry` status until the
parent's 30-minute poll timeout (DEFAULT_POLL_TIMEOUT_MS) gives up,
during which the parent's pending task tool call shows "waiting for
subagent" with no indication of failure.

This change adds a first-prompt watchdog that synthesises the missing
error-event trigger:

  - Armed when a user message lands in a subagent session
    (membership check via `subagentSessions`).
  - Cancelled on the first sign of progress: any assistant message
    with text/reasoning content, finish field, or an error field (any
    of which is something the existing handlers will deal with).
  - Cancelled on session terminal events (idle/stop/deleted/error).
  - On fire (90s default): aborts the in-flight request and routes
    into the existing dispatchFallbackRetry path — the same code that
    runs when a session.error arrives. No new fallback mechanism.

Design choices:

  - Dispatch fallback, do not abort the subagent outright. Network
    loss looks identical to a stuck retry from the hook's vantage
    point; with fallback-dispatch behaviour, network loss degrades
    to today's baseline (both attempts fail, 30-min outer timeout
    still ends things) rather than destructively aborting work.
  - Scope strictly to subagents. Parent/user sessions can legitimately
    take 90s+ to produce the first token; subagent dispatches in
    practice produce first content much faster, so a 90s ceiling is
    safe.
  - Threshold is tunable via the third arg to createFirstPromptWatchdog;
    DEFAULT_FIRST_PROMPT_WATCHDOG_MS = 90_000 in constants.ts.

Also adds a diagnostic log in session-status-handler when a
`session.status: retry` event arrives whose message does not match
RETRYABLE_ERROR_PATTERNS. This is the hook's other silent-return
spot for retry events; logging the raw retry message will let us
extend the patterns next time we hit a provider whose phrasing
we don't yet match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 20:45:46 +09:00
YeonGyu-Kim 0036c203a2 Merge pull request #3934 from Qihao0v0/fix/unifyllm-quota-classifier
fix(runtime-fallback): classify localized balance failures as quota exhaustion
2026-05-15 19:57:34 +09:00
YeonGyu-Kim 437a8edba6 Merge pull request #4007 from PeterPonyu/feat/runtime-fallback-internal-abort
fix(runtime-fallback): preserve attemptCount when our own abort is the cause
2026-05-15 19:26:53 +09:00
YeonGyu-Kim 3c7d1299a7 Revert "Merge pull request #3825 from tw-yshuang/fix/delegated-child-session-early-failure-fallback"
This reverts commit cd33f3a397, reversing
changes made to 521c99cf0b.
2026-05-15 19:13:48 +09:00
YeonGyu-Kim cd33f3a397 Merge pull request #3825 from tw-yshuang/fix/delegated-child-session-early-failure-fallback
fix(delegate-task): harden child-session first-prompt fallback recovery
2026-05-15 19:06:43 +09:00
YeonGyu-Kim 521c99cf0b Merge pull request #3950 from ismetanin/fix/surface-subagent-quota-error
fix(runtime-fallback): surface quota errors from stuck subagents
2026-05-15 19:05:34 +09:00
YeonGyu-Kim cb87385086 test(ci): isolate runtime and rules dependencies 2026-05-15 18:36:32 +09:00
YeonGyu-Kim 9f6d0d2281 docs(agents-md): refresh hierarchical knowledge base for v4.1.2
- Bump root AGENTS.md header: 2026-05-14 → 2026-05-15, commit 5ffbe0e2453a740636, release v4.1.1 → v4.1.2
- Update file counts: 2034 (1337+697) → 2041 (1340+701), LOC ~292k → ~294k
- Fix STRUCTURE: openclaw lives at src/openclaw/ (not src/features/); list more accurate feature modules in the parenthetical
- Clarify interactive_bash gate: tmux binary on PATH via isInteractiveBashEnabled() (not 'tmux enabled')
- Fix docs/reference/features.md hook counts: Tool Guard 14→16, Total base 52→54, total with team-mode 59→61
- Bump 'Generated' date on all 43 subdir AGENTS.md files to 2026-05-15
- Preserve promptAsync injection cautions verbatim (per request)
2026-05-15 13:48:11 +09:00
YeonGyu-Kim c2aa180e7e fix(prompt-gate): pin duplicate prompt dispatches
Keep prompt reservations briefly after successful dispatch so rapid idle/message/error transitions cannot inject the same follow-up twice.

Route all production session prompt calls through the shared gate, restore skipped background resume state, release holds after abort/recovery paths, and preserve Ralph/ULW loop state when a dispatch is deferred.

Add regression coverage for session routing, static prompt route auditing, team-mode live messaging, model suggestion retries, call-omo-agent reuse, background parent wakes, runtime fallback, compaction recovery, Atlas, and Ralph/ULW loops.
2026-05-15 13:19:10 +09:00
YeonGyu-Kim 439e72839b fix(runtime-fallback): gate retry prompts 2026-05-15 12:55:56 +09:00
YeonGyu-Kim 1e7a7600a2 docs(agents-md): regenerate hierarchical AGENTS.md knowledge base for v4.1.1
Refresh all AGENTS.md files to reflect codebase state at 5ffbe0e24 (was cd31d2a1a, 197 commits behind).

Key drift corrections across 45 modified + 1 new file:

Root AGENTS.md:
- TS file counts: 1967 -> 2034 in src/ (1337 source + 697 test)
- LOC: 278k -> 292k
- Barrel index.ts: 120 -> 122
- Hook tier composition: 52/59 -> 54/61 (base/with team-mode)
- Tool Guard hooks: 14 -> 16 (add fsync-skip-warning, bash-file-read-guard)
- Add boulder feature, agent-ordering schema, .agents/ directory, v4.1.1 release tag
- Add generated/ directory entry

src/AGENTS.md:
- Subsystem inventory: agents 96->102, hooks 570->581, tools 306->314,
  features 389->400, shared 258->278, cli 150->158, plugin 55->56
- LOC totals refreshed for every subsystem
- Schema files: 32 -> 30

src/hooks/AGENTS.md:
- Tier 2 (Tool Guard): 14 -> 16 hooks, add fsyncSkipWarning row
- Total: 52 base / 59 team-mode -> 54 base / 61 team-mode
- zauc-mocks count: 7 -> 5

src/features/AGENTS.md:
- background-agent: 47 -> 57 files, mention archive fallback
- opencode-skill-loader: 33 -> 30
- tmux-subagent: 34 -> 32

src/plugin/AGENTS.md:
- Tool Guard composer count: 14 -> 16
- Aggregator total: 43 -> 45

src/cli/AGENTS.md:
- Add new boulder subcommand (BoulderState inspector)
- Command count: 6 -> 7

NEW: src/features/boulder-state/AGENTS.md
- Document the new Boulder work tracking feature
- Schema v2 with BoulderState/BoulderWorkState/TaskSessionState
- Lifecycle, storage, integration points with atlas/ralph-loop hooks

All other AGENTS.md files: Generated date 2026-05-08 -> 2026-05-14.
2026-05-14 12:57:46 +09:00
PeterPonyu 5b99a87c6a fix(runtime-fallback): preserve attemptCount when our own abort is the cause (closes #4006)
When runtime-fallback aborts an in-flight request to swap in a fallback
model, opencode emits session.error{isAbort:true} as a consequence. The
existing event handler treated that as a user cancellation and called
resetRetryState — wiping attemptCount. Every subsequent provider
auto-retry signal then started over at attempt:1, never reaching
max_fallback_attempts, producing an infinite retry loop firing a new
fallback every ~2 seconds.

The bug only surfaces when the configured fallback target itself
silently fails (e.g. github-copilot quota exhausted): the original
model keeps re-emitting retry signals, our handler keeps "fixing"
them, the counter never advances. Reproducible on upstream/dev HEAD
(5ffbe0e24e).

Fix:
- New `internallyAbortedSessions: Set<string>` on HookDeps tracks
  sessions whose abort we triggered ourselves.
- abortSessionRequest in auto-retry.ts adds the session to the set
  when called with one of our internal sources:
  "session.status.retry-signal", "message.updated.retry-signal",
  "session.timeout". The "session.stop" source (user-initiated) is
  intentionally NOT marked — that path must still wipe state.
- handleSessionError in event-handler.ts checks the set before the
  cancellation branch. If the session is marked, consume the flag
  (delete it so a later user-abort still gets the reset) and skip
  resetRetryState. The state's attemptCount is preserved, so the
  next iteration progresses 1→2→3→... until max_fallback_attempts.
- dispose() clears the new set alongside the other per-session maps.

Tests: 3 new event-handler integration tests cover the fix
(internal-abort preserves state, external-abort still resets,
consecutive internal-abort cycles advance attemptCount). Existing
tests pass: 7/7 on event-handler. Pre-existing 2 dispose-test flakes
on the full runtime-fallback suite were verified to exist on
upstream/dev without this patch — unrelated.

bun run build: pass. bun run typecheck: pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 23:55:52 -04:00
YeonGyu-Kim 5ffbe0e24e fix(fallback): guard duplicate prompt injections 2026-05-14 01:03:19 +09:00
Ivan Smetanin 83c3379b5a Merge branch 'dev' into fix/surface-subagent-quota-error 2026-05-12 22:17:38 +01:00
tw-yshuang 12f5233877 merge(dev): resolve latest manager and runtime-fallback conflicts
Sync the PR branch with the newest dev branch and resolve the new import-level conflicts in background-agent manager and runtime-fallback tests. Preserve both the delegated bootstrap coverage from this branch and the newer upstream test utilities and runtime wiring changes, then re-verify the affected delegated fallback suites and typecheck.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-12 22:49:57 +08:00
YeonGyu-Kim d5fbada13d test: make unsafe test coercion explicit
Move test coercion out of a hidden global and require each test to import the helper so review tools and runtime scripts can see the unsafe boundary.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-12 15:51:31 +09:00
YeonGyu-Kim 23211159c9 test(hooks): remove unsafe test assertions
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-12 15:41:17 +09:00
YeonGyu-Kim 4da48555ee fix(plugin): normalize event session ids
Handle OpenCode session events that carry the session ID under properties.info.id or properties.info.sessionID so background tasks and continuation hooks do not miss idle/error/delete events.

Add regression coverage for nested session.idle events completing background tasks and waking continuation hooks.
2026-05-12 12:32:26 +09:00
tw-yshuang 1e1f574b95 merge(dev): resolve latest sync-task conflict for delegated fallback PR
Sync the PR branch with the latest dev branch and resolve the remaining conflict in sync-task.test.ts while preserving both the new upstream poll-recovery coverage and this branch's delegated bootstrap cleanup and isolation coverage. Re-verified the affected delegated fallback suites and typecheck after the merge resolution.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-12 00:37:01 +08:00
Ivan Smetanin ecb92c1e70 fix(runtime-fallback): abort stuck subagent on quota error with no fallback
When a subagent session (e.g. Momus on GPT) hits a quota/usage-limit
error and the agent's category has no `fallback_models` configured, the
runtime-fallback hook previously returned silently. The OpenCode session
stayed in `retry` status indefinitely while the SDK kept hitting the
limit, the sync-task poller treated `retry` as active work, and the
parent's pending `task` tool call never resolved — leaving a stuck
"waiting for subagent" indicator in the parent conversation.

Narrow fix: at the `fallbackModels.length === 0` exit point, if the
session is a known subagent AND the error classifies as
`quota_exceeded`, abort the subagent session. The existing
`getTerminalSessionError` path in `sync-session-poller.ts` then surfaces
the error via the parent's tool result, which is the persistent surface
the user is already watching.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 14:40:43 +01:00
Qihao 583fa2420f fix(runtime-fallback): classify localized balance failures 2026-05-11 11:06:06 +08:00
tw-yshuang 74171b9156 merge(dev): resolve background-agent delegated fallback conflicts
Reconcile the latest dev branch changes with the delegated child-session fallback work. Preserve the upstream background-agent updates while keeping the delegated bootstrap cleanup and compatibility wiring fixes intact, then re-verify the affected regression suites and typecheck.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-11 03:02:14 +08:00
wenghuayang863 f3f72fc96f fix(runtime-fallback): also classify Volcano Engine errors as quota_exceeded
- Add /exceeded.*quota/i and /usage\s*quota/i to classifyErrorType quota block
- Align /usage.?quota/i -> /usage\s*quota/i in RETRYABLE_ERROR_PATTERNS for consistency
- Strengthen auto-retry-signal test assertion
- Add classifyErrorType assertion to Volcano Engine regression test

Ensures Volcano Engine errors are both retryable AND logged as
errorType: quota_exceeded.
2026-05-11 01:03:02 +08:00
wenghuayang863 1c7881ec09 fix(runtime-fallback): match Volcano Engine 'exceeded the usage quota' errors
Volcano Engine sends quota exceeded errors with the words in reverse
order: 'You have exceeded the 5-hour usage quota'. The existing
patterns required 'quota' to precede 'exceeded', so they never matched.

- Add /exceeded.*quota/i and /usage.?quota/i to RETRYABLE_ERROR_PATTERNS
- Add exceeded.*quota and usage\s*quota to AUTO_RETRY_PATTERNS
- Add regression tests for both detection paths

Fixes: runtime-fallback not triggering on Volcano Engine quota errors
2026-05-11 00:43:58 +08:00
YeonGyu-Kim 838b5ae216 docs(agents): refresh hierarchical AGENTS.md with team-mode coverage
Update root + 43 directory-level AGENTS.md files to reflect current state:
- Root AGENTS.md rewritten with accurate counts (1967 TS files, 1304 source +
  663 test, 278k LOC, 120 barrel index.ts), 7-step init flow, 5-tier hook
  composition, and full Team Mode section (12 team_* tools, eligibility,
  storage layout, config gate)
- src/AGENTS.md adds team-mode init step, current per-subdir file/LOC table
- src/tools/AGENTS.md documents conditional gates (team-mode +12, task
  system +4, hashline +1, interactive_bash +1, look_at +1) with always-on
  baseline of 20
- src/hooks/AGENTS.md splits into 5 tiers + 4 conditional team-mode hooks
- src/features/team-mode/AGENTS.md surfaces 12 tools, eligible agents,
  spawn-race-safe invariants, and integration points
- src/features/builtin-skills/AGENTS.md tracks 10 skills incl. team-mode
- src/agents/AGENTS.md, src/plugin/AGENTS.md, src/config/AGENTS.md updated
  for team-mode awareness, accurate counts, and current schema field list
- All other AGENTS.md files refreshed to 2026-05-08 generation date
2026-05-08 12:08:42 +09:00
tw-yshuang fac90d69f8 fix(delegate-task): harden child-session fallback bootstrap and cleanup
Capture delegated child-session retry context before the first prompt so fallback recovery still works when session history is empty. Align background and sync launch paths around the same bootstrap contract, clear session-scoped fallback state on every terminal path, and lock the behavior with regression coverage for first-prompt retries, exhaustion, isolation, and cleanup.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-05-07 08:34:52 +08:00
MoerAI a22416d270 fix(error-classifier): classify credit balance too low as quota_exceeded for fallback (fixes #3571) 2026-05-06 17:17:23 +09:00
Paolo Notaro d800fec6b1 fix(runtime-fallback): prevent infinite loop when fallback model equals current model 2026-05-05 21:01:50 +02:00
MoerAI 328df08162 fix(test): update provider-matrix tests to expect fallback on quota exhaustion 2026-04-22 19:10:49 +09:00
MoerAI 59493002e4 fix(error-classifier): match insufficient balance/funds as quota_exceeded 2026-04-22 18:47:16 +09:00
MoerAI fbf130ec3a fix(test): update integration tests to expect fallback on quota exhaustion 2026-04-21 18:30:04 +09:00