diff options
| author | Paul Buetow <paul@buetow.org> | 2025-09-03 16:00:26 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2025-09-03 16:00:26 +0300 |
| commit | ffe9ed5531b6e62706ea555c48964ea0e560b780 (patch) | |
| tree | 81974f771543827f4c0743f5f1d66f5fbd06a2bd /TODO.md | |
| parent | 71f0d04bd558433cebf1b05845c9fa0e2957eba8 (diff) | |
Phase 2: add configurable completion debounce\n\n- App config: completion_debounce_ms (default 200)\n- Server: wait until no input for debounce before LLM calls\n- Applies to chat and provider-native completion paths\n- Tests: add debounce and adjust to verify behavior\n\nAll unit tests pass.
Diffstat (limited to 'TODO.md')
| -rw-r--r-- | TODO.md | 4 |
1 files changed, 4 insertions, 0 deletions
@@ -17,6 +17,10 @@ Updated tests accordingly. Phase 2: Debounce completion requests: Introduce a configurable delay (e.g., 100–500 ms) before sending a completion request to the LLM. This prevents a flood of calls while typing. +Status: Done — added `completion_debounce_ms` (default 200). Server waits until +no recent input activity for at least this duration before LLM calls (both chat +and provider-native paths). Added unit test `TestCompletionDebounce_WaitsUntilQuiet`. + Phase 3: Throttle on the server side: Beyond debouncing, implement request throttling to cap the maximum rate of LLM calls (e.g., one per 500 ms). This is especially useful when debounce alone isn’t enough under rapid editing 2 . |
