diff options
Diffstat (limited to 'snippets/hyperstack')
| -rw-r--r-- | snippets/hyperstack/hyperstack_vm.rb | 4 | ||||
| -rwxr-xr-x | snippets/hyperstack/install.sh | 142 | ||||
| -rw-r--r-- | snippets/hyperstack/opencode-setup.md | 255 |
3 files changed, 4 insertions, 397 deletions
diff --git a/snippets/hyperstack/hyperstack_vm.rb b/snippets/hyperstack/hyperstack_vm.rb index 2219fbb..9743683 100644 --- a/snippets/hyperstack/hyperstack_vm.rb +++ b/snippets/hyperstack/hyperstack_vm.rb @@ -5,6 +5,10 @@ begin require 'bundler/setup' rescue LoadError nil +rescue Gem::Exception => e + # Ruby can ship with a Bundler library version whose matching executable + # is not installed locally. Fall back to direct gem loading in that case. + raise unless e.is_a?(Gem::GemNotFoundException) || e.is_a?(Gem::LoadError) end require 'json' diff --git a/snippets/hyperstack/install.sh b/snippets/hyperstack/install.sh deleted file mode 100755 index 3995a62..0000000 --- a/snippets/hyperstack/install.sh +++ /dev/null @@ -1,142 +0,0 @@ -#!/bin/bash -set -e - -# OpenCode + Codex + Ollama Installation Script -# Usage: ./install.sh [user@host] [opencode_version] -# Example: ./install.sh ubuntu@38.128.233.181 v1.1.32 - -HOST="${1:-ubuntu@38.128.233.181}" -OPENCODE_VERSION="${2:-v1.1.32}" -CODEX_VERSION="${3:-latest}" - -echo "Installing OpenCode, Codex, and Ollama on $HOST" -echo "OpenCode version: $OPENCODE_VERSION" -echo "Codex version: $CODEX_VERSION" - -# Function to run remote commands -run_remote() { - ssh "$HOST" "$1" -} - -# 1. Install Ollama -echo "Step 1: Installing Ollama..." -run_remote "curl -fsSL https://ollama.ai/install.sh | sh" || echo "Ollama may already be installed" - -# 2. Start Ollama service -echo "Step 2: Starting Ollama service..." -run_remote "sudo systemctl start ollama && sudo systemctl status ollama" || echo "Starting Ollama..." - -# 2.5. Configure Ollama to use /ephemeral for models (if available) -echo "Step 2.5: Configuring Ollama models directory..." -run_remote "if [ -d /ephemeral ]; then - sudo mkdir -p /ephemeral/ollama/models - sudo chown ollama:ollama /ephemeral/ollama /ephemeral/ollama/models - sudo mkdir -p /etc/systemd/system/ollama.service.d - sudo tee /etc/systemd/system/ollama.service.d/override.conf > /dev/null << 'EOF' -[Service] -Environment=\"OLLAMA_MODELS=/ephemeral/ollama/models\" -Environment=\"OLLAMA_GPU_OVERHEAD=2000\" -Environment=\"OLLAMA_NUM_PARALLEL=4\" -EOF - sudo systemctl daemon-reload - sudo systemctl restart ollama - sleep 3 - echo 'Ollama configured to use /ephemeral' -else - echo 'No /ephemeral directory found, using default' -fi" - -# 3. Kill unattended upgrade lock if it exists -echo "Step 3: Clearing package manager lock..." -run_remote "sudo pkill -f unattended-upgrade || true" -sleep 2 - -# 4. Install Node.js 20 (required for Codex), npm, and Python -echo "Step 4: Installing Node.js 20, npm, and Python..." -run_remote "curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - && sudo apt install -y nodejs python3 python3-pip" - -# 5. Download and install OpenCode CLI -echo "Step 5: Installing OpenCode CLI ($OPENCODE_VERSION)..." -run_remote "cd /tmp && wget -q https://github.com/anomalyco/opencode/releases/download/$OPENCODE_VERSION/opencode-linux-x64.tar.gz && tar -xzf opencode-linux-x64.tar.gz && sudo mv opencode /usr/local/bin/ && opencode --version" - -# 6. Create OpenCode config directory -echo "Step 6: Creating OpenCode config directory..." -run_remote "mkdir -p ~/.config/opencode" - -# 7. Create OpenCode configuration (qwen3-coder models) -echo "Step 7: Creating OpenCode configuration..." -run_remote "cat > ~/.config/opencode/opencode.json << 'EOF' -{ - \"provider\": { - \"ollama\": { - \"npm\": \"@ai-sdk/openai-compatible\", - \"name\": \"Ollama\", - \"api\": \"http://localhost:11434/v1\", - \"models\": { - \"qwen3-coder:latest\": { - \"name\": \"Qwen3 Coder Latest (Higher Quality)\" - }, - \"qwen3-coder:30b-a3b-q4_K_M\": { - \"name\": \"Qwen3 Coder 30B A3B Q4_K_M (Quantized)\" - } - } - } - } -} -EOF" - -# 8. Pull qwen3-coder models -echo "Step 8: Pulling qwen3-coder models (this may take a while)..." -run_remote "ollama pull qwen3-coder:latest" -run_remote "ollama pull qwen3-coder:30b-a3b-q4_K_M" - -# 9. Install Codex CLI -echo "Step 9: Installing Codex CLI..." -run_remote "sudo npm install -g @openai/codex || npm install -g @openai/codex" - -# 10. Create Codex config to use Ollama -echo "Step 10: Configuring Codex to use Ollama..." -run_remote "mkdir -p ~/.codex && cat > ~/.codex/config.json << 'EOF' -{ - \"provider\": \"ollama\", - \"model\": \"qwen3-coder:30b-a3b-q4_K_M\", - \"endpoint\": \"http://localhost:11434/v1\" -} -EOF" - -# 11. Install Aider (LLM-powered code editing) -echo "Step 11: Installing Aider..." -run_remote "pip3 install --user aider-chat && echo 'export PATH=~/.local/bin:\$PATH' >> ~/.bashrc" - -# 12. Configure Aider to use Ollama -echo "Step 12: Configuring Aider for Ollama..." -run_remote "mkdir -p ~/.aider && cat > ~/.aider/.aider.conf.yml << 'EOF' -model: ollama/qwen2.5-coder:14b-instruct -openai-api-base: http://localhost:11434/v1 -EOF" - -# 13. Verify installation -echo "Step 13: Verifying installation..." -run_remote "opencode --version && ollama list && codex --version && ~/.local/bin/aider --version" - -echo "" -echo "✓ Installation complete!" -echo "" -echo "To use OpenCode:" -echo " ssh $HOST" -echo " cd ~/your/project" -echo " opencode" -echo "" -echo "To use Codex:" -echo " ssh $HOST" -echo " codex --oss --local-provider ollama" -echo "" -echo "To use Aider (LLM-powered code editing):" -echo " ssh $HOST" -echo " cd ~/your/project" -echo " aider" -echo " aider <filename>" -echo "" -echo "Available models for all tools:" -echo " - qwen2.5-coder:14b-instruct (fast, ~9GB) - default" -echo " - qwen3-coder:30b-a3b-q4_K_M (slower, ~18GB)" diff --git a/snippets/hyperstack/opencode-setup.md b/snippets/hyperstack/opencode-setup.md deleted file mode 100644 index 4fd1361..0000000 --- a/snippets/hyperstack/opencode-setup.md +++ /dev/null @@ -1,255 +0,0 @@ -# OpenCode Setup on Hyperstack with Ollama - -This document outlines the steps to install and configure OpenCode to work with local Ollama models on a remote A100-80GB server. - -## Prerequisites - -- SSH access to the remote server (e.g., `ubuntu@IPHERE`) -- Ollama installed on the remote server - -## Installation Steps -### 0. Install Ollama - -Install Ollama on the remote server: - -```bash -ssh ubuntu@IPHERE "curl -fsSL https://ollama.ai/install.sh | sh" -``` - -Start Ollama service: - -```bash -ssh ubuntu@IPHERE "sudo systemctl start ollama && sudo systemctl status ollama" -``` - -### 1. Kill Unattended Upgrade Lock - -If the package manager is locked by unattended-upgrades: - -```bash -ssh ubuntu@IPHERE "sudo kill -9 3523 && sleep 2" -``` - -### 2. Verify Node.js is Installed - -```bash -ssh ubuntu@IPHERE "npm --version && node --version" -``` - -If not installed, install Node.js: - -```bash -ssh ubuntu@IPHERE "sudo apt install -y nodejs npm" -``` - -### 3. Download and Install OpenCode CLI - -Get the latest release from GitHub releases and install to `/usr/local/bin`: - -```bash -ssh ubuntu@IPHERE "cd /tmp && wget -q https://github.com/anomalyco/opencode/releases/download/v1.1.32/opencode-linux-x64.tar.gz && tar -xzf opencode-linux-x64.tar.gz && sudo mv opencode /usr/local/bin/ && opencode --version" -``` - -This installs OpenCode v1.1.32. Check [GitHub releases](https://github.com/anomalyco/opencode/releases) for the latest version. - -### 4. Create OpenCode Configuration Directory - -```bash -ssh ubuntu@IPHERE "mkdir -p ~/.config/opencode" -``` - -### 5. Configure OpenCode to Use Ollama - -Create `~/.config/opencode/opencode.json` with Ollama as the provider: - -```bash -ssh ubuntu@IPHERE "cat > ~/.config/opencode/opencode.json << 'EOF' -{ - \"provider\": { - \"ollama\": { - \"npm\": \"@ai-sdk/openai-compatible\", - \"name\": \"Ollama\", - \"api\": \"http://localhost:11434/v1\", - \"models\": { - \"qwen3-coder:30b-a3b-q4_K_M\": { - \"name\": \"Qwen3 Coder 30B A3B Q4_K_M\" - }, - \"qwen2.5-coder:32b\": { - \"name\": \"Qwen2.5 Coder 32B\" - }, - \"mistral-large\": { - \"name\": \"Mistral Large 123B\" - }, - \"deepseek-coder\": { - \"name\": \"Deepseek Coder\" - } - } - } - } -} -EOF" -``` - -**Key configuration fields:** -- `npm`: AI SDK package for OpenAI-compatible APIs -- `api`: Ollama endpoint (defaults to localhost:11434) -- `models`: Map of model IDs to display names - -### 6. Clean Up Disk Space (if needed) - -Check disk usage: - -```bash -ssh ubuntu@IPHERE "df -h / && ollama list" -``` - -For A100-80GB, ensure sufficient space (~120GB free). Remove unused models if needed: - -```bash -ssh ubuntu@IPHERE "ollama rm model_name" -``` - -### 7. Pull Models into Ollama - -Pull Qwen3 Coder 30B (~20GB): - -```bash -ssh ubuntu@IPHERE "ollama pull qwen3-coder:30b-a3b-q4_K_M" -``` - -Pull Qwen2.5 Coder 32B (~19GB): - -```bash -ssh ubuntu@IPHERE "ollama pull qwen2.5-coder:32b" -``` - -Pull Mistral Large (~73GB): - -```bash -ssh ubuntu@IPHERE "ollama pull mistral-large" -``` - -Pull Deepseek Coder (776MB - lightweight option): - -```bash -ssh ubuntu@IPHERE "ollama pull deepseek-coder" -``` - -Verify models are available: - -```bash -ssh ubuntu@IPHERE "ollama list" -``` - -## Using OpenCode - -Navigate to your project and start OpenCode: - -```bash -ssh ubuntu@IPHERE -cd ~/git/aitest -opencode -``` - -Then: -1. Press Tab to enter Plan mode (recommended for new features) -2. Use `/models` command to select which Ollama model to use -3. Ask OpenCode to help with your code - -## Configuration Details - -### Provider Configuration Schema - -The `provider` object in `opencode.json` uses this structure: - -```json -{ - "provider": { - "provider_id": { - "npm": "@ai-sdk/openai-compatible", - "name": "Display Name", - "api": "http://localhost:11434/v1", - "models": { - "model_id": { - "name": "Model Display Name" - } - } - } - } -} -``` - -### Available Models - -#### Qwen3 Coder 30B A3B Q4_K_M -- **Size**: ~20GB (optimized quantization) -- **Strengths**: Advanced coding capabilities, good tool calling, specialized for programming -- **Tool Support**: Function calling support -- **Use Case**: Primary coding assistant for complex tasks - -#### Qwen2.5 Coder 32B -- **Size**: ~19GB -- **Strengths**: Optimized for function calling and coding, better than earlier versions -- **Tool Support**: Strong function calling support -- **Use Case**: Reliable coding assistant for general-purpose tasks - -#### Mistral Large 123B -- **Size**: 73GB -- **Strengths**: Excellent tool/function calling support, strong reasoning, balanced for coding -- **Tool Support**: Native function calling -- **Use Case**: Most capable model for complex reasoning and coding tasks - -#### Deepseek Coder -- **Size**: 776MB (very lightweight) -- **Strengths**: Specialized for coding tasks, fast inference -- **Tool Support**: Function calling support -- **Use Case**: Fast responses for code generation and analysis on limited resources - -### Ollama Endpoint - -OpenCode communicates with Ollama via the OpenAI-compatible API: -- Default: `http://localhost:11434/v1` -- Ollama models endpoint: `http://localhost:11434/api/tags` - -## Troubleshooting - -### Configuration Error: "Unrecognized key" - -Ensure the config uses correct structure: -- Top-level key should be `provider` (not `providers`) -- Provider ID (e.g., `ollama`) is a key under `provider` object -- Each provider has `npm`, `name`, `api`, and `models` fields - -### Out of Disk Space - -Check available space: -```bash -df -h / -``` - -Remove models: -```bash -ollama rm model_name -``` - -For A100-80GB, budget ~120GB for models + system (account for OS and dependencies). - -### Ollama Connection Issues - -Verify Ollama is running and accessible: - -```bash -curl -s http://localhost:11434/api/tags | jq '.models[].name' -``` - -If using remote SSH, ensure: -- Ollama is listening on 0.0.0.0 (not just localhost) -- Port 11434 is not firewalled -- Use SSH port forwarding: `ssh -L 11434:localhost:11434 ubuntu@server` - -## References - -- [OpenCode Documentation](https://opencode.ai/docs) -- [OpenCode Providers Configuration](https://opencode.ai/docs/providers) -- [OpenCode GitHub](https://github.com/anomalyco/opencode) -- [Ollama Documentation](https://ollama.ai) |