blob: 686c7ab821e9b8f685e211d6c8186fa0a1ea186f (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
|
# Add configurable request timeout for LLM calls
## Motivation
When using locally-hosted LLMs (via LM Studio, Ollama, or other OpenAI-compatible servers), the default 30-second HTTP timeout is often insufficient. Local models, especially larger ones like Gemma 3 27B, can take significantly longer to generate responses, resulting in "context deadline exceeded" errors.
This change allows users to configure a longer timeout to accommodate slower local inference.
## Changes
### New configuration option
Added `request_timeout` (in seconds) to the `[general]` section:
```toml
[general]
request_timeout = 120 # seconds, default 30
```
Or via environment variable:
```sh
export HEXAI_REQUEST_TIMEOUT=120
```
### Implementation
- Added `RequestTimeout` field to config structs with default of 30 seconds
- Each LLM client constructor now has a `*WithTimeout` variant that accepts the timeout
- Original constructors remain unchanged (delegate to `*WithTimeout` with default)
- `NewFromConfig` passes the configured timeout to clients
### Files modified
- `internal/appconfig/config.go` - Config field, parsing, merge logic, env var support
- `internal/llm/provider.go` - Added `RequestTimeout` to `Config`, calls `*WithTimeout` constructors
- `internal/llm/openai.go` - Added `newOpenAIWithTimeout`
- `internal/llm/ollama.go` - Added `newOllamaWithTimeout`
- `internal/llm/openrouter.go` - Added `newOpenRouterWithTimeout`
- `internal/llm/anthropic.go` - Added `newAnthropicWithTimeout`
- `internal/hexailsp/run.go` - Pass `RequestTimeout` to `llm.Config`
- `internal/llmutils/client.go` - Pass `RequestTimeout` to `llm.Config`
- `internal/lsp/server.go` - Pass `RequestTimeout` to `llm.Config`
- `config.toml.example` - Documented option
- `docs/configuration.md` - Added `HEXAI_REQUEST_TIMEOUT` to env vars list
### Test impact
No test files were modified. The original constructor signatures are preserved, so all existing tests continue to work unchanged.
## Testing
```sh
# Set timeout in config
[general]
request_timeout = 120
# Test with local LLM
cat somefile.go | hexai "review this code"
```
All existing tests pass:
```sh
HEXAI_TEST_SKIP_NET=1 go test ./...
```
|
