|
Wrap the shared provider HTTP choke point (doJSONRequest in llm/util.go,
used by openai, openrouter, anthropic and ollama) with two resilience
patterns implemented with the standard library only:
- Retry with exponential backoff + jitter (resilience.go): 3 attempts by
default, retrying transient network errors and retryable HTTP statuses
(429 and 5xx). Client errors (4xx) and successes are returned
immediately and never retried. Backoff waits are context-aware so
cancellation and deadlines are respected; retried response bodies are
drained and closed for connection reuse.
- Circuit breaker (circuitbreaker.go, own file as it has >3 methods):
classic closed/open/half-open breaker that trips after 5 consecutive
transient failures and stays open for a 30s cooldown, then allows a
single trial probe. Only transient failures count; 4xx never trips it.
A nil breaker is a valid no-op.
doJSONRequest now delegates to doJSONRequestResilient; the single-attempt
primitive is preserved as doJSONRequestOnce. Adds httptest-based unit
tests for retry/backoff/breaker logic with no real network calls;
new code coverage is >80%.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
