Qwen3.8-27B · hermes (LoRA, GGUF)
The GGUF version of harness-hub/Qwen3.8-27B-hermes-adapted, a LoRA adapter trained with reinforcement learning inside Hermes Agent, on Qwen/Qwen3.8-27B, for llama.cpp.
- SWE-bench Pro (hard 51): 29.4% → 43.1% (+13.7 points; 10 tasks newly solved, 3 lost).
- Terminal-Bench 2.1: 44.7% → 58.8% (+14.1 points; 19 newly solved, 7 lost; p = 0.029).
- Less work per task: under 40% of the base model's turns on the tasks both solve, with about 2× fewer generated tokens.
[!Tip] Thinking-level accuracy, at a quarter of the tokens. With thinking off, the adapter is level with the base model thinking at its highest reasoning effort (
xhigh) under the same setup, while generating 3.5–4.4× fewer tokens per task. That matters most locally, where every generated token is time: keep thinking off.
[!Note] These scores come from the BF16 adapter served by vLLM, under the Hermes Agent harness (one attempt per task, a 100-step budget, a 131,072-token context window, thinking off, temperature 0.8). With thinking on, Hermes Agent sends each turn's reasoning back to the model, and many thinking-on sessions ran out of that window: the thinking comparison holds under this setup, not necessarily with a larger window or compaction. This GGUF file has not been re-evaluated through llama.cpp, and a quantized base will move the results further. They are not comparable with the scores on the Qwen3.8-27B card, which uses other harnesses and thinking at maximum effort.
The original adapter card describes the training and the evaluation, and links every evaluation trajectory.
Qwen3.8-27B-hermes-lora-F16.gguf is the adapter only (about 319 MB). It runs on top of the base model's GGUF.
Run
1. Download the base model and the adapter
hf download ggml-org/Qwen3.8-27B-GGUF Qwen3.8-27B-Q8_0.gguf --local-dir .
hf download harness-hub/Qwen3.8-27B-hermes-adapted-GGUF Qwen3.8-27B-hermes-lora-F16.gguf --local-dir .
The adapter was trained on the BF16 weights. The closer the base is to them, the closer you get to the trained behavior.
Q8_0 is a good default; Q4_K_M is smaller, but drifts further from what the adapter learned against.
2. Serve
llama serve -m Qwen3.8-27B-Q8_0.gguf --lora Qwen3.8-27B-hermes-lora-F16.gguf \
--chat-template-kwargs '{"enable_thinking": false}' -c 65536 --port 8080 --alias qwen3.8-hermes
Keep thinking off, as in training and evaluation: with it on, the adapter is not significantly more accurate and generates about 3× more tokens. Hermes Agent needs at least 64,000 tokens of context.
3. Point Hermes Agent at it
Install Hermes Agent:
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
Add the server to ~/.hermes/config.yaml:
model:
default: qwen3.8-hermes
provider: custom
base_url: http://localhost:8080/v1
api_key: dummy
hermes chat -q "Find out why the tests in tests/test_parser.py fail and fix the bug."
Conversion
Converted from the PEFT adapter with llama.cpp's convert_lora_to_gguf.py, at F16:
python llama.cpp/convert_lora_to_gguf.py <adapter dir> --base <Qwen/Qwen3.8-27B config dir> --outtype f16
Limitations
- Not re-evaluated in GGUF. The scores on the original adapter card come from vLLM at BF16. This file has not been evaluated through llama.cpp, and a quantized base will move the results further.
- Thinking off. The adapter was trained without thinking; with thinking on it is not significantly more accurate, at about 3× the tokens.