Run with the Decision 2.0 llama.cpp branch:
llama-server -hf vllm-sr/Decision-2.0-Vega-27B-GGUF:Q8_0 --embeddings --pooling none -c 8192 -b 8192 -ub 8192 -np 1
This is a decision model, to be used via the /v1/systemone API. Requires Decision 2.0 runtime support.
Available in BF16, Q8_0 and Q4_K_M.
[!NOTE] Conversion recipes: Decision 2.0 GGUF.