GGUF release of the baked dense-search candidate based on Qwen/Qwen3.5-4B.
This folder contains:
qwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.f16.ggufqwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.Q8_0.ggufqwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.Q6_K.ggufqwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.Q5_K_M.ggufqwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.Q5_K_S.ggufqwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.Q4_K_M.ggufqwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.Q4_K_S.ggufmmproj-Qwen3.5-4B-F16.ggufRecommended default:
qwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.Q4_K_M.ggufqwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.Q4_K_M.gguf with mmproj-Qwen3.5-4B-F16.ggufQuant guide:
F16: highest fidelity, largest fileQ8_0: near-lossless quantQ6_K: high-quality smaller quantQ5_K_M: balanced quality/sizeQ5_K_S: slightly smaller than Q5_K_MQ4_K_M: practical defaultQ4_K_S: smallest file in this release../qwen35-4b-oblit-dense-bakedQwen/Qwen3.5-4B20, strength 1.50, targets attention, embeddings, linear_attention, mlp128/128 -> 11/1289/131 -> 5/131128/128 -> 28/1283/131 -> 4/1310.00059792760293930774.5933134970255196e-05llama-cli \
-m qwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.Q4_K_M.gguf \
-p "Explain overfitting in plain English."
llama-mtmd-cli \
-m qwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.Q4_K_M.gguf \
--mmproj mmproj-Qwen3.5-4B-F16.gguf \
--image example.jpg \
-p "Describe this image."
llama-server \
-m qwen3.5-4b-obliterate-dense-search-step1-no_decoder_input.Q4_K_M.gguf \
--mmproj mmproj-Qwen3.5-4B-F16.gguf
mmproj-Qwen3.5-4B-F16.gguf file is the matching projector for multimodal use.Q4_K_M file is the practical default. Q6_K and Q8_0 are there if you want to trade space for fidelity.F16 file is the highest-fidelity GGUF export in this folder.Have the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo amkkk/Qwen3.5-4B-abliterated-aggressive-GGUF ./model-folderQwen/Qwen3.5-4B