Austriani // ik_llama.cpp // GGUF

Qwen3.5-4B-3D-Coder

Holographic Quantization Archive for Three.js and Code 4B Parameters 32 Layers 1024 Imatrix Ctx 500 Cycles Base Architecture This repository hosts ik_llama.cpp-compatible GGUF quantizations of ccore/Qwen3.5-4B-3D-Coder. This is an independent fine-tune of Qwen/Qwen3.5-4B by CCORE, optimized for general-purpose coding and high-quality 3D scene generation. It writes self-contained Three.js HTML and JavaScript scenes directly. The architecture is a dense 4B parameter hybrid model. It uses a repeating layout of 3 Gated DeltaNet (linear attention) layers followed by 1 gated full-attention layer, stacked 8 times. The model features a hidden size of 2560 and a native 262,144-token context window (extendable to 1M with YaRN). No abliteration or behavioral modification has been applied. Note on MTP Layer: The Multi-Token Prediction (MTP) layer has been removed from these quantizations, as the current quantization pipeline (ik_llama.cpp) does not support it. However, if you have the MTP weights for Qwen3.5-4B installed and enabled in your inference setup, this layer can be manually added back to restore full speculative decoding capabilities. MaxOptima Custom Quant This release includes MaxOptima, a bespoke, surgical quantization mix designed specifically for the 3:1 hybrid attention architecture of Qwen3.5. It targets the full-attention blocks and boundary layers for high precision, while compressing the FFNs to save space. MaxOptima Strategy: Full-attention layers (blocks 3, 7, 11, 15, 19, 23, 27, 31) are kept at Q5_K to preserve global context and logic. The embeddings, output layer, and boundary blocks (0 and 31) are kept at Q5_K and Q6_K. The attention gates use IQ4_KSS, while the bulk FFN up/down are compressed to IQ4_KSS and the FFN gates to IQ4_KS. This results in a file significantly smaller than a standard Q4_K_M, but with the attention precision required for complex 3D math. View Full Custom Quant Configuration --custom-q "^output\.weight=Q6_K,token_embd\.weight=Q5_K,ssm_out\.weight=Q5_K,attn_qkv\.weight=Q5_K,attn_gate\.weight=IQ4_KSS,blk\.(3|7|11|15|19|23|27|31)\.attn_q\.weight=Q5_K,blk\.(3|7|11|15|19|23|27|31)\.attn_k\.weight=Q5_K,blk\.(3|7|11|15|19|23|27|31)\.attn_v\.weight=Q5_K,blk\.(3|7|11|15|19|23|27|31)\.attn_output\.weight=Q5_K,blk\.(0|31)\.ffn_down\.weight=Q5_K,blk\.(0|31)\.ffn_up\.weight=Q5_K,ffn_down\.weight=IQ4_KSS,ffn_up\.weight=IQ4_KSS,ffn_gate\.weight=IQ4_KS,.*=IQ4_KS" Available Quants MaxOptima // Custom Hybrid Mix MaxOptima Custom mix. Full-attention blocks at Q5_K, FFNs at IQ4_KSS/IQ4_KS. Best quality-to-size ratio for 3D coding. Standard IQK // High & Mid Fidelity IQ5_K Higher fidelity 5-bit. Best quality-to-size for systems with sufficient memory. IQ4_K Standard 4-bit IQK. Uses 256-value super-blocks for scaling. IQ4_KS "Small" 4-bit variant with tighter packing. Lower memory footprint. IQ4_KS_R4 Row-interleaved variant of IQ4_KS, optimized for ik_llama.cpp's matmul kernels. Trellis // Alternative 4-Bit IQ4_KT Dynamic programming trellis quant. Uses a fundamentally different error-minimization strategy than IQK. Imatrix Specification This release utilizes the exact same V3 Calibration Dataset used across previous Austriani releases. No datasets were changed. The calibration was executed at 500 cycles per chunk with a 1024-token context window, processing approximately 1000 chunks. Compatibility These GGUF files are built exclusively for ik_llama.cpp. IQK, Trellis, and MaxOptima quants will not load in mainline llama.cpp. Please refer to the official base repository for mainline GGUF compatibility. Austriani // Qwen3.5-4B-3D-Coder // ik_llama.cpp Base: ccore/Qwen3.5-4B-3D-Coder