BF16 | FP8 | NVFP4 | GGUF | Blog
This repository holds GGUF quantizations of DaisyRose-27B for local inference with llama.cpp, Ollama and LM Studio.
For server deployment, use the BF16, FP8 or NVFP4 weights with SGLang. The files here are for laptops, workstations and single consumer GPUs.
| File | Size | Notes |
|---|---|---|
DaisyRose-27B-Q4_K_XL.gguf |
17.7 GB | Recommended. |
DaisyRose-27B-Q4_K_M.gguf |
16.6 GB | Slightly smaller 4-bit mix. |
DaisyRose-27B-Q3_K_XL.gguf |
13.2 GB | Recommended for 16 GB GPUs. |
DaisyRose-27B-Q2_K_XL.gguf |
10.0 GB | Smallest. Reasons longer and hits context limits more often; use a lower reasoning effort. |
DaisyRose-27B-Q5_K_M.gguf |
19.9 GB | |
DaisyRose-27B-Q6_K.gguf |
22.1 GB | |
DaisyRose-27B-Q8_0.gguf |
29.0 GB | |
mmproj-DaisyRose-27B-BF16.gguf |
0.9 GB | Vision projector, needed for image input. |
Ollama:
ollama run hf.co/RadixArk/DaisyRose-27B-GGUF:Q4_K_XL
llama.cpp server (OpenAI-compatible API on port 8080, text and images):
llama-server -hf RadixArk/DaisyRose-27B-GGUF:Q4_K_XL --jinja -ngl 999 -c 65536
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="none")
resp = client.chat.completions.create(
model="DaisyRose-27B",
messages=[{"role": "user", "content": "Write a binary search in Rust."}],
temperature=1.0, top_p=0.95,
extra_body={"top_k": 20, "min_p": 0.0},
)
print(resp.choices[0].message.content)
temperature=1.0, top_p=0.95, top_k=20, min_p=0.0.xhigh, which can produce tens of thousands of thinking tokens on hard problems. Give each request at least 64k of context (-c 65536 per slot), or lower the effort with --chat-template-kwargs '{"reasoning_effort":"medium"}'. Disable thinking with {"enable_thinking": false}.--jinja).convert_hf_to_gguf.py), commit 4d91760.llama-quantize --tensor-type-file. Thanks to the Unsloth team for publishing them.Apache-2.0. DaisyRose-27B is post-trained by RadixArk from Qwen3.8-27B.
RadixArk/DaisyRose-27B