LFM2.5-24B-A2B · Harbor · Hermes (LoRA, GGUF)

The GGUF version of harness-hub/LFM2.5-24B-A2B-hermes, a LoRA adapter for LiquidAI/LFM2.5-24B-A2B trained with reinforcement learning inside Hermes Agent, for llama.cpp.

In short: about the same accuracy as the base model, with 20% fewer tool calls on the tasks both solve. The original adapter card describes the training and the evaluation.

LFM2.5-24B-A2B-harbor-hermes-lora-F16.gguf is the adapter only (about 37 MB). It runs on top of LiquidAI's base GGUF.

Run

1. Download the base model and the adapter

hf download LiquidAI/LFM2.5-24B-A2B-GGUF LFM2.5-24B-A2B-Q8_0.gguf --local-dir .
hf download harness-hub/LFM2.5-24B-A2B-hermes-GGUF LFM2.5-24B-A2B-harbor-hermes-lora-F16.gguf --local-dir .

The adapter was trained on the BF16 weights. The closer the base is to them, the closer you get to the trained behavior. Q8_0 is a good default; Q4_K_M is smaller, but drifts further from what the adapter learned against.

2. Serve

llama serve -m LFM2.5-24B-A2B-Q8_0.gguf --lora LFM2.5-24B-A2B-harbor-hermes-lora-F16.gguf \
    -c 65536 --port 8080 --alias lfm-harbor

Hermes needs at least 64,000 tokens of context per session, so keep -c at 65536 or more, and raise it with the number of parallel slots (-np).

3. Point Hermes at it

Install Hermes:

curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash

Add the server to ~/.hermes/config.yaml:

model:
  default: lfm-harbor
  provider: custom
  base_url: http://localhost:8080/v1
  api_key: dummy

Then run hermes chat.

Conversion

Converted from the PEFT adapter with llama.cpp's convert_lora_to_gguf.py, at F16:

python llama.cpp/convert_lora_to_gguf.py <adapter dir> --base <LiquidAI/LFM2.5-24B-A2B config dir> --outtype f16

Limitations

Not re-evaluated in GGUF. The scores on the original adapter card come from vLLM at BF16. This file has not been evaluated through llama.cpp, and a quantized base will move the results further.
Small, single-domain evaluation. See the original card: 100 data-analysis tasks, about ±8 points of noise on a paired pass@1 difference.

harness-hub/LFM2.5-24B-A2B-hermes-GGUF