summaryrefslogtreecommitdiff
path: root/README.md
diff options
context:
space:
mode:
authorPaul Buetow <paul@buetow.org>2025-08-16 17:19:46 +0300
committerPaul Buetow <paul@buetow.org>2025-08-16 17:19:46 +0300
commite62c851109892ed60a60926904b03bb4d2378fe4 (patch)
tree7325477c1f7223de76545d47731a85c5c47a3f5c /README.md
parent778a3591bd27ce49acb6f8596f3c714351c412dc (diff)
feat(llm): add Ollama provider + provider selection and CLI override; update README and logsv0.0.1
Diffstat (limited to 'README.md')
-rw-r--r--README.md40
1 files changed, 34 insertions, 6 deletions
diff --git a/README.md b/README.md
index c3c77c2..1649e1b 100644
--- a/README.md
+++ b/README.md
@@ -8,14 +8,34 @@ At the moment this project is only in the proof of concept phase.
## LLM provider
-Hexai exposes a simple LLM provider interface and uses OpenAI by default for
-code completion when `OPENAI_API_KEY` is present in the environment.
+Hexai exposes a simple LLM provider interface. It supports OpenAI and a local
+Ollama server. Provider selection and models are configured via environment
+variables.
-- Required: set `OPENAI_API_KEY` to your OpenAI API key.
-- Optional: set `OPENAI_MODEL`.
-- Optional: set `OPENAI_BASE_URL` to point at a compatible endpoint.
+### Selecting a provider
-If no key is configured, Hexai will fall back to a basic, local completion.
+- Set `HEXAI_LLM_PROVIDER` to `openai` or `ollama` to force a provider.
+- If not set, Hexai auto‑detects:
+ - Uses OpenAI when `OPENAI_API_KEY` is present.
+ - Uses Ollama when any `OLLAMA_*` variables are present.
+ - Otherwise, Hexai falls back to a basic, local completion.
+
+### OpenAI configuration
+
+- Required: `OPENAI_API_KEY` — your OpenAI API key.
+- Optional: `OPENAI_MODEL` — model name (default: `gpt-4o-mini`).
+- Optional: `OPENAI_BASE_URL` — override the API base (e.g., a compatible endpoint).
+
+### Ollama configuration (local)
+
+- Optional: `OLLAMA_MODEL` — model name/tag (default: `qwen2.5-coder:latest`).
+- Optional: `OLLAMA_BASE_URL` or `OLLAMA_HOST` — base URL to Ollama
+ (default: `http://localhost:11434`).
+
+Notes:
+- For Ollama, ensure the model is available locally (e.g., `ollama pull qwen2.5-coder:latest`).
+- If you run Ollama in OpenAI‑compatible mode, you may alternatively use the
+ OpenAI provider with `OPENAI_BASE_URL` pointing to your local endpoint.
## CLI usage and configuration
@@ -31,6 +51,7 @@ If no key is configured, Hexai will fall back to a basic, local completion.
- `always-full`: always include the full file (may be slower/costly)
- `-context-window-lines`: line count for the sliding window when `context-mode=window`.
- `-max-context-tokens`: budget for additional context tokens. If the flag isn’t provided, `HEXAI_MAX_CONTEXT_TOKENS` is used when set.
+ - `-provider`: LLM provider override: `openai` or `ollama` (overrides `HEXAI_LLM_PROVIDER`).
Notes:
- Token estimation for truncation uses a simple 4 chars/token heuristic.
@@ -48,3 +69,10 @@ Notes:
| `-max-context-tokens` | `HEXAI_MAX_CONTEXT_TOKENS` | Token budget for additional context. |
| `-log-preview-limit` | `HEXAI_LOG_PREVIEW_LIMIT` | Limit characters shown in LLM preview logs. |
| `-no-disk-io` | `HEXAI_NO_DISK_IO` | Disallow any disk reads for context. |
+| `-provider` | `HEXAI_LLM_PROVIDER` | Force LLM provider: `openai` or `ollama`. |
+
+### Environment quick reference (providers)
+
+- `HEXAI_LLM_PROVIDER`: `openai` | `ollama` (optional; otherwise auto‑detect).
+- OpenAI: `OPENAI_API_KEY` (required), `OPENAI_MODEL`, `OPENAI_BASE_URL`.
+- Ollama: `OLLAMA_MODEL`, `OLLAMA_BASE_URL` or `OLLAMA_HOST`.