Bojun-Feng/Qwen3.8-27B-llamafile

Run LLMs locally with a single file

Download the file and run it. Delete the file when you are done.

No environment setup. No library installation. No annoying clean up.

Compare uncensored variants ↓

How to use

These bundled files are too large to run on Windows. Use a separate llamafile binary with the source GGUF files, as shown in the llamafile docs.

1. Download the llamafile (e.g. Q4_K_M).

2. Open a terminal on Linux or macOS.

3. Grant execute permission.

   chmod +x Qwen3.8-27B.Q4_K_M.llamafile
   

4. Run the file.

   bash Qwen3.8-27B.Q4_K_M.llamafile
   
   # Or run directly if your system supports it.
   ./Qwen3.8-27B.Q4_K_M.llamafile
   

   • CLI: Chat directly in the current terminal.
   • Web UI: Open localhost:8080 in your browser.
   • API: Connect an OpenAI-compatible client to http://127.0.0.1:8080/v1.
     • The chat endpoint is /v1/chat/completions.
     • The server supports 4 parallel requests by default.

5. Press Control-C to stop the process.

6. For more details, run with --help or read the llamafile docs.

   bash Qwen3.8-27B.Q4_K_M.llamafile --help
   

expected output sample

ubuntu@ubuntu > ./Qwen3.8-27B.Q4_K_M.llamafile
██╗     ██╗      █████╗ ███╗   ███╗ █████╗ ███████╗██╗██╗     ███████╗
██║     ██║     ██╔══██╗████╗ ████║██╔══██╗██╔════╝██║██║     ██╔════╝
██║     ██║     ███████║██╔████╔██║███████║█████╗  ██║██║     █████╗
██║     ██║     ██╔══██║██║╚██╔╝██║██╔══██║██╔══╝  ██║██║     ██╔══╝
███████╗███████╗██║  ██║██║ ╚═╝ ██║██║  ██║██║     ██║███████╗███████╗
╚══════╝╚══════╝╚═╝  ╚═╝╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚══════╝╚══════╝
software: llamafile 0.10.5
model:    Qwen3.8-27B.Q4_K_M.gguf
compute:  Apple Metal GPU
server:   http://127.0.0.1:8080

A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.
>>>

Collection and comparison

Part of Qwen3.8-27B Uncensored Variant Llamafiles.

Chart source code

#@title Performance versus uncensored response rate
import matplotlib.pyplot as plt
from statistics import mean

# Source: [Abliterlitics](https://web.archive.org/web/20260909182157/https://abliterlitics.dev/models/qwen38-27b/)
# Archived FP8 results, not measurements of these GGUFs.
# Columns: MMLU-Pro, GPQA, BBH, MuSR, IFEval, HellaSwag, ARC-C, WinoGrande,
# PIQA, GSM8K, TQA-MC1, TQA-MC2, TQA-Gen, HumanEval, HarmBench judge ASR (%).
data = {
    "Official": [62.82, 28.28, 75.16, 45.90, 82.07, 74.79, 59.90, 71.11, 79.65, 91.21, 44.43, 66.98, 55.08, 79.3, 4.5],
    "Heretic ARA": [62.65, 27.27, 74.78, 45.90, 82.26, 74.19, 60.24, 71.27, 79.65, 91.51, 44.19, 65.74, 55.08, 82.3, 57.5],
    "Apostate": [62.81, 29.80, 74.22, 45.63, 82.99, 74.54, 60.84, 71.19, 79.87, 91.81, 44.19, 65.02, 53.37, 79.9, 78.7],
    "coder3101": [62.66, 28.79, 74.78, 46.16, 83.36, 74.53, 59.90, 70.09, 79.60, 91.21, 43.57, 64.63, 53.12, 80.5, 70.0],
    "OrcaRouter": [62.78, 27.27, 74.29, 45.37, 82.44, 74.16, 59.90, 71.27, 79.65, 90.75, 44.55, 65.48, 53.49, 78.7, 82.2],
    "Ultra Heretic": [62.25, 26.77, 74.92, 45.24, 82.26, 74.34, 58.28, 71.11, 79.22, 90.75, 39.90, 60.54, 48.84, 79.3, 70.5],
    "Huihui": [62.67, 32.32, 74.69, 45.63, 81.89, 74.10, 60.07, 71.35, 79.43, 91.74, 41.86, 63.71, 52.02, 78.7, 75.6],
    "OBLITERATUS": [56.49, 30.30, 70.44, 45.90, 77.63, 73.90, 62.03, 69.93, 80.69, 92.19, 38.19, 55.82, 45.04, 45.7, 63.9],
    "Blackfrost": [60.89, 25.76, 73.36, 42.72, 82.44, 73.31, 58.19, 68.11, 79.82, 89.76, 43.45, 64.08, 51.04, 70.7, 68.5],
}
# Not in this collection; remove these filters to include all source rows.
del data["Apostate"]
del data["coder3101"]

# Equal mean of 12 families; average TQA's three scores first. Exclude perplexity.
performance = [mean(row[:10] + [mean(row[10:13]), row[13]]) for row in data.values()]
# Blog judge ASR excludes degenerate responses; it is not final-answer usability.
uncensored = [row[-1] for row in data.values()]
fig, ax = plt.subplots(figsize=(9, 6), layout="constrained")
ax.scatter(performance, uncensored, s=55)
for name, x, y in zip(data, performance, uncensored):
    ax.annotate(name, (x, y), xytext=(6, 6), textcoords="offset points")
ax.set(xlabel="Performance — mean of 12 benchmark families (%)",
       ylabel="Uncensored response rate — HarmBench judge ASR (%)",
       title="Qwen3.8-27B variants", ylim=(0, 100))
ax.margins(x=0.20)
ax.grid(alpha=0.2)
fig.savefig("qwen38-performance-vs-asr.png", dpi=180, bbox_inches="tight")
plt.show()

Other variants:

• Qwen3.8 27B (This one)
• Heretic ARA Uncensored (Best Performance)
• OrcaRouter Uncensored (Best Uncensored)
• Ultra Heretic Uncensored
• Huihui Uncensored
• OBLITERATUS Uncensored
• Blackfrost Uncensored

The suggestions are based on imperfect benchmarks and should not be trusted blindly. Try the variants yourself with your own use case.

All data came from the Abliterlitics review (snapshot), which also contains more detailed analysis of more uncensored model variants. I am not the author, nor did I replicate the experiments myself.

Settings for Qwen3.8 27B GGUF Llamafiles

• Model creator: Qwen
• Original model: Qwen/Qwen3.8-27B
• Quantized GGUF files used: mradermacher/Qwen3.8-27B-GGUF
  • Commit hash 860659a596e879987cee91b4c43ae605796f3ef8
  • Commit message: auto-patch README.md
• Llamafile version: 0.10.5, mozilla-ai/llamafile
  • Commit hash 486e6c5f9356eae50b851b07517bfae1f2420193
  • Commit message: docs(support): document the Vulkan GPU backend (#1029) (#1033)
• Quantizations: Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0.
• Validation: Linux text generation; details in validation/.
• Vision: text GGUF; upstream projector files remain separate.
• MTP: source tensors retained; validation uses standard decoding.
• .args contents:

-m
Qwen3.8-27B.Q4_K_M.gguf
...