diff options
| author | Paul Buetow <paul@buetow.org> | 2025-08-16 17:19:46 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2025-08-16 17:19:46 +0300 |
| commit | e62c851109892ed60a60926904b03bb4d2378fe4 (patch) | |
| tree | 7325477c1f7223de76545d47731a85c5c47a3f5c /README.md | |
| parent | 778a3591bd27ce49acb6f8596f3c714351c412dc (diff) | |
feat(llm): add Ollama provider + provider selection and CLI override; update README and logsv0.0.1
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 40 |
1 files changed, 34 insertions, 6 deletions
@@ -8,14 +8,34 @@ At the moment this project is only in the proof of concept phase. ## LLM provider -Hexai exposes a simple LLM provider interface and uses OpenAI by default for -code completion when `OPENAI_API_KEY` is present in the environment. +Hexai exposes a simple LLM provider interface. It supports OpenAI and a local +Ollama server. Provider selection and models are configured via environment +variables. -- Required: set `OPENAI_API_KEY` to your OpenAI API key. -- Optional: set `OPENAI_MODEL`. -- Optional: set `OPENAI_BASE_URL` to point at a compatible endpoint. +### Selecting a provider -If no key is configured, Hexai will fall back to a basic, local completion. +- Set `HEXAI_LLM_PROVIDER` to `openai` or `ollama` to force a provider. +- If not set, Hexai auto‑detects: + - Uses OpenAI when `OPENAI_API_KEY` is present. + - Uses Ollama when any `OLLAMA_*` variables are present. + - Otherwise, Hexai falls back to a basic, local completion. + +### OpenAI configuration + +- Required: `OPENAI_API_KEY` — your OpenAI API key. +- Optional: `OPENAI_MODEL` — model name (default: `gpt-4o-mini`). +- Optional: `OPENAI_BASE_URL` — override the API base (e.g., a compatible endpoint). + +### Ollama configuration (local) + +- Optional: `OLLAMA_MODEL` — model name/tag (default: `qwen2.5-coder:latest`). +- Optional: `OLLAMA_BASE_URL` or `OLLAMA_HOST` — base URL to Ollama + (default: `http://localhost:11434`). + +Notes: +- For Ollama, ensure the model is available locally (e.g., `ollama pull qwen2.5-coder:latest`). +- If you run Ollama in OpenAI‑compatible mode, you may alternatively use the + OpenAI provider with `OPENAI_BASE_URL` pointing to your local endpoint. ## CLI usage and configuration @@ -31,6 +51,7 @@ If no key is configured, Hexai will fall back to a basic, local completion. - `always-full`: always include the full file (may be slower/costly) - `-context-window-lines`: line count for the sliding window when `context-mode=window`. - `-max-context-tokens`: budget for additional context tokens. If the flag isn’t provided, `HEXAI_MAX_CONTEXT_TOKENS` is used when set. + - `-provider`: LLM provider override: `openai` or `ollama` (overrides `HEXAI_LLM_PROVIDER`). Notes: - Token estimation for truncation uses a simple 4 chars/token heuristic. @@ -48,3 +69,10 @@ Notes: | `-max-context-tokens` | `HEXAI_MAX_CONTEXT_TOKENS` | Token budget for additional context. | | `-log-preview-limit` | `HEXAI_LOG_PREVIEW_LIMIT` | Limit characters shown in LLM preview logs. | | `-no-disk-io` | `HEXAI_NO_DISK_IO` | Disallow any disk reads for context. | +| `-provider` | `HEXAI_LLM_PROVIDER` | Force LLM provider: `openai` or `ollama`. | + +### Environment quick reference (providers) + +- `HEXAI_LLM_PROVIDER`: `openai` | `ollama` (optional; otherwise auto‑detect). +- OpenAI: `OPENAI_API_KEY` (required), `OPENAI_MODEL`, `OPENAI_BASE_URL`. +- Ollama: `OLLAMA_MODEL`, `OLLAMA_BASE_URL` or `OLLAMA_HOST`. |
