feat(runtime-fallback): add timeout toggle for quota retry detection
Make provider auto-retry signal detection respect timeout_seconds setting: - When timeout_seconds=0, disable quota-based fallback escalation - Only treat auto-retry signals as errors when timeout is enabled - Add test to verify behavior when timeout_seconds is disabled - Update documentation to explain timeout_seconds=0 behavior This allows users to disable timeout-based fallbacks while keeping error-based fallback functionality intact.
This commit is contained in:
committed by
YeonGyu-Kim
parent
8b2ae957e5
commit
68f5d982fc
@@ -738,7 +738,7 @@ Automatically switch to backup models when the primary model encounters retryabl
|
||||
| `retry_on_errors` | `[429, 503, 529]` | HTTP status codes that trigger fallback (rate limit, service unavailable). Also supports certain classified provider errors (for example, missing API key) that do not expose HTTP status codes. |
|
||||
| `max_fallback_attempts` | `3` | Maximum fallback attempts per session (1-10) |
|
||||
| `cooldown_seconds` | `60` | Cooldown in seconds before retrying a failed model |
|
||||
| `timeout_seconds` | `30` | Timeout in seconds for an in-flight fallback request before forcing the next fallback model |
|
||||
| `timeout_seconds` | `30` | Timeout in seconds for an in-flight fallback request before forcing the next fallback model. Set to `0` to disable timeout-based fallback and provider quota retry signal detection. |
|
||||
| `notify_on_fallback` | `true` | Show toast notification when switching to a fallback model |
|
||||
|
||||
### How It Works
|
||||
@@ -924,7 +924,7 @@ Automatically switch to backup models when the primary model encounters retryabl
|
||||
| `retry_on_errors` | `[429, 503, 529]` | HTTP status codes that trigger fallback (rate limit, service unavailable). Also supports certain classified provider errors (for example, missing API key) that do not expose HTTP status codes. |
|
||||
| `max_fallback_attempts` | `3` | Maximum fallback attempts per session (1-10) |
|
||||
| `cooldown_seconds` | `60` | Cooldown in seconds before retrying a failed model |
|
||||
| `timeout_seconds` | `30` | Timeout in seconds for an in-flight fallback request before forcing the next fallback model |
|
||||
| `timeout_seconds` | `30` | Timeout in seconds for an in-flight fallback request before forcing the next fallback model. Set to `0` to disable timeout-based fallback and provider quota retry signal detection. |
|
||||
| `notify_on_fallback` | `true` | Show toast notification when switching to a fallback model |
|
||||
|
||||
### How It Works
|
||||
|
||||
Reference in New Issue
Block a user