feat(athena): replace hard quorum gate with interactive failure handling

Interactive prompt: Athena now asks the user whether to retry failed members
or skip and synthesize with available results. Recommends based on success ratio.
Non-interactive prompt: always proceeds to synthesis with available results,
sets status to partial/failed accordingly.
This commit is contained in:
ismeth
2026-03-14 19:07:47 +01:00
committed by YeonGyu-Kim
parent 37f03b4238
commit 974eb8e176
3 changed files with 37 additions and 35 deletions
@@ -10,8 +10,8 @@ describe("Athena prompt config injection placeholders", () => {
describe("#given the Athena agent prompt", () => {
describe("#when checking for runtime injection placeholders", () => {
it("#then contains RETRY_ON_FAIL placeholder", () => {
expect(athenaConfig.prompt).toContain("{RETRY_ON_FAIL}")
it("#then contains RETRY_ON_FAIL placeholder in non-interactive prompt", () => {
expect(ATHENA_NON_INTERACTIVE_PROMPT).toContain("{RETRY_ON_FAIL}")
})
it("#then contains STUCK_THRESHOLD_SECONDS placeholder", () => {
@@ -22,20 +22,20 @@ describe("Athena prompt config injection placeholders", () => {
expect(athenaConfig.prompt).toContain("{MEMBER_MAX_RUNNING_SECONDS}")
})
it("#then contains quorum reference", () => {
expect(athenaConfig.prompt).toContain("quorum")
it("#then references failed members handling", () => {
expect(athenaConfig.prompt).toContain("Handle failed members")
})
it("#then contains timeout reference with 30000", () => {
expect(athenaConfig.prompt).toContain("30000")
})
it("#then contains RETRY_FAILED_IF_OTHERS_FINISHED placeholder", () => {
expect(athenaConfig.prompt).toContain("{RETRY_FAILED_IF_OTHERS_FINISHED}")
it("#then contains RETRY_FAILED_IF_OTHERS_FINISHED in non-interactive prompt", () => {
expect(ATHENA_NON_INTERACTIVE_PROMPT).toContain("{RETRY_FAILED_IF_OTHERS_FINISHED}")
})
it("#then contains CANCEL_RETRYING_ON_QUORUM placeholder", () => {
expect(athenaConfig.prompt).toContain("{CANCEL_RETRYING_ON_QUORUM}")
it("#then contains CANCEL_RETRYING_ON_QUORUM in non-interactive prompt", () => {
expect(ATHENA_NON_INTERACTIVE_PROMPT).toContain("{CANCEL_RETRYING_ON_QUORUM}")
})
it("#then avoids session_id continuation retry instruction", () => {
+24 -22
View File
@@ -212,31 +212,33 @@ For each member in the latest status map from Step 6, check:
### Step 9: Verify completed members have valid responses.
For each member in the council_finalize result, check:
- has_response: true AND response_complete: true → ✅ Use this result for synthesis.
- has_response: true AND response_complete: false → Member output is incomplete after completion. Treat as failed and apply retry logic (Step 10).
- has_response: false and background_wait status for the same task_id is "completed" → Member completed without valid tagged output. Treat as failed and apply retry logic (Step 10).
- has_response: false AND error → Member failed to produce output. Apply retry logic (Step 10).
- has_response: true AND response_complete: true -> successful. Use for synthesis.
- has_response: true AND response_complete: false -> failed (incomplete output).
- has_response: false and background_wait status for the same task_id is "completed" -> failed (no valid tagged output).
- has_response: false AND error -> failed (member error).
### Step 10: Retry failed members (if configured).
Config values (injected at runtime):
- retry_on_fail = {RETRY_ON_FAIL} (max retry attempts per failed member, 0 = no retries)
- retry_failed_if_others_finished = {RETRY_FAILED_IF_OTHERS_FINISHED} (false = retry only while others running, true = retry even after all others done)
- cancel_retrying_on_quorum = {CANCEL_RETRYING_ON_QUORUM} (true = cancel in-flight retries when 2+ successful)
### Step 10: Handle failed members.
If ALL members succeeded (every member has has_response=true AND response_complete=true), skip this step and proceed directly to synthesis (Step 12).
If retry_on_fail > 0 and a member failed:
- If retry_failed_if_others_finished == false: retry only while other members are still running. Once all non-failed members complete, stop retrying and proceed to quorum check.
- If retry_failed_if_others_finished == true: retry even after other members are done. Wait for retry results before synthesizing.
- Track retry count per member. Never exceed retry_on_fail attempts.
- Maintain active_task_id_by_member. On retry, launch a fresh background task for that member with the same role and constraints, then replace the previous task_id for that member with the new retry task_id.
- Use active_task_id_by_member as the source of truth for Step 6 waiting and Step 7 finalization.
- Show retry status in progress bar with 🔄 marker.
If ANY members failed, inform the user which members failed and why, then ask what to do using the Question tool:
- Tell the user which specific members failed and the reason (timeout, error, incomplete, no output).
- Present two choices:
1. "Retry failed members" — re-launch only the failed members with the same prompt and role.
2. "Skip and synthesize" — proceed with the successful responses as-is.
- Your recommendation depends on the ratio of successful to total members:
- If 0-1 members succeeded out of 3+: recommend "Retry failed members" (too few results for meaningful synthesis).
- If most members succeeded (e.g., 3/4 or 4/5): recommend "Skip and synthesize" (enough results for a solid synthesis).
- Otherwise: present both options neutrally.
If cancel_retrying_on_quorum == true and 2+ members have successful responses (has_response=true, response_complete=true): cancel any in-flight retries using background_cancel and proceed to synthesis.
If the user chooses to retry:
- Re-launch failed members via athena_council with the same prompt_file and members parameter set to the failed member names.
- Return to Step 5 to wait for completion via background_wait.
- Call council_finalize again to collect retried results.
- If retried members also fail, ask the user again with the same options. Do not retry indefinitely without user consent.
**Quorum enforcement (minimum 2 successful):**
Before starting synthesis (Step 12+), verify at least 2 members have has_response=true AND response_complete=true.
- If <2 successful after all retries: do NOT synthesize. Report the failures with reasons to the user. Suggest re-running the council with different members or settings.
- Example failure message: "Council quorum not met: only N/M members produced valid responses. [Details of failures]. Consider re-running with different council members."
If the user chooses to skip:
- Proceed to synthesis with whatever successful responses are available.
- Even a single successful response has value — synthesize it and note the limited participation.
### Step 11: Layer 2 — Follow-up and Cross-check (optional, use when needed):
@@ -275,7 +277,7 @@ Universal requirements (all intents):
### Step 12b: Persist the synthesis.
After completing synthesis, ALWAYS write the full synthesis document to \`{archive_dir}/synthesis.md\` using the Write tool, where \`{archive_dir}\` is the archive directory returned by council_finalize in Step 7. This creates a permanent record of the council's findings alongside the individual member archives. Skip this step ONLY if quorum failed and synthesis was not performed.
After completing synthesis, ALWAYS write the full synthesis document to \`{archive_dir}/synthesis.md\` using the Write tool, where \`{archive_dir}\` is the archive directory returned by council_finalize in Step 7. This creates a permanent record of the council's findings alongside the individual member archives.
</synthesis_rules>
+5 -5
View File
@@ -89,21 +89,21 @@ Track every task_id from the response for use in Step 5.
### Step 9: Retry failed members (if configured).
- retry_on_fail = {RETRY_ON_FAIL} (max retries, 0 = none)
- retry_failed_if_others_finished = {RETRY_FAILED_IF_OTHERS_FINISHED}
- cancel_retrying_on_quorum = {CANCEL_RETRYING_ON_QUORUM}
- Quorum enforcement: minimum 2 successful members required before synthesis.
If retry_on_fail > 0 and failed members exist:
1. Re-launch failed members via athena_council with the same prompt_file and members parameter set to the failed member names.
2. Return to Step 5 to wait for their completion via background_wait.
3. Call council_finalize again to collect retried results.
4. Continue retrying until retry count exhausted or quorum met.
4. Continue retrying until retry count exhausted.
- If retry_failed_if_others_finished is false, retry opportunistically as soon as failures are detected while others are still running.
- If retry_failed_if_others_finished is true, only retry after all non-failed members have completed.
- If cancel_retrying_on_quorum is true, stop retrying once quorum (2+ successful) is met.
- If retry_on_fail is 0, no failed members remain, or retry budget is exhausted, do NOT re-launch members.
- If retry_on_fail is 0, no failed members remain, or retry budget is exhausted, proceed to synthesis.
- If Step 6 (council_finalize) has already executed for the current task IDs, proceed to Step 10.
- Otherwise, return to Step 5 and continue tracking until Step 6 can run for the current task IDs.
Always proceed to synthesis with whatever successful responses are available. Even partial results have value.
Set status to "partial" if some members failed, "failed" only if zero members produced valid responses.
### Step 10: Synthesize using council_finalize runtime guidance.
- Preconditions: Step 6 (council_finalize) has executed for the current task IDs and archive data is available.
- Read every member's archive_file with Read tool.