summaryrefslogtreecommitdiff
path: root/snippets/hyperstack/.gitignore
diff options
context:
space:
mode:
authorPaul Buetow <paul@buetow.org>2026-03-18 13:14:26 +0200
committerPaul Buetow <paul@buetow.org>2026-03-18 13:14:26 +0200
commit5dc0878f9617b3472dc819b3662f52bfefad892a (patch)
treeed974db930596dc826305d04174ba47a018bd50e /snippets/hyperstack/.gitignore
parent2a2704fa4cac96a6754d4fea1bc341a27c5bb6c8 (diff)
Fix nemotron-super preset and test_vllm model detection
Replace cyankiwi/Llama-3_3-Nemotron-Super-49B-v1_5-AWQ-4bit with casperhansen/llama-3.3-70b-instruct-awq for the nemotron-super preset. The NAS model's config.json has num_key_value_heads=null by design for its heterogeneous per-layer attention architecture, which is incompatible with vLLM's pydantic ModelConfig validation (requires int). No working AWQ quant for this architecture exists; Llama-3.3-70B-Instruct AWQ is a proven drop-in for the extended-analysis use case. Also fix test_vllm to use the model reported by /v1/models instead of the static config default, so tests pass after a model switch. Add trust_remote_code support to vllm_install_script for future models that require custom HuggingFace model code. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'snippets/hyperstack/.gitignore')
0 files changed, 0 insertions, 0 deletions