diff options
| author | Paul Buetow <paul@buetow.org> | 2026-03-18 13:14:26 +0200 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2026-03-18 13:14:26 +0200 |
| commit | 5dc0878f9617b3472dc819b3662f52bfefad892a (patch) | |
| tree | ed974db930596dc826305d04174ba47a018bd50e /snippets/hyperstack/.gitignore | |
| parent | 2a2704fa4cac96a6754d4fea1bc341a27c5bb6c8 (diff) | |
Fix nemotron-super preset and test_vllm model detection
Replace cyankiwi/Llama-3_3-Nemotron-Super-49B-v1_5-AWQ-4bit with
casperhansen/llama-3.3-70b-instruct-awq for the nemotron-super preset.
The NAS model's config.json has num_key_value_heads=null by design for
its heterogeneous per-layer attention architecture, which is incompatible
with vLLM's pydantic ModelConfig validation (requires int). No working
AWQ quant for this architecture exists; Llama-3.3-70B-Instruct AWQ is
a proven drop-in for the extended-analysis use case.
Also fix test_vllm to use the model reported by /v1/models instead of
the static config default, so tests pass after a model switch.
Add trust_remote_code support to vllm_install_script for future models
that require custom HuggingFace model code.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'snippets/hyperstack/.gitignore')
0 files changed, 0 insertions, 0 deletions
