!!! quantized for use with ik_llama.cpp and its derivatives !!!

!!! incompatible with mainline llama.cpp as of commit #34af94c !!!

What's that?

Skyfall is a heavy hitter for mainline consumer GPUs. I have an abusive relationship with it - the goal of fitting the most of it into 16 GBs of VRAM, even without KVO, eluded me twice. I made two publicly avaliable ik_llama.cpp-exclusive quants of Skyfall before, each with their own issues. Lessons were learned. Most were forgotten.

It's time for Round 3.

The goal: shove Skyfall into 16 GB VRAM with system overhead, AGAIN, using ik_llama, hopefully make it all smart enough for a mixed quant.
The result: another hybrid quantization of TheDrummer/Skyfall-31B-v4.2, hopefully the final iteration

Rationale

Having a very limited budget for a 31b model (I run the system off the same GPU I use for ik_llama.cpp inference, so I am only safe within a 14.5 GB margin), I decided to play chess - literally. The recipe involves two layer precision schemae. The higher precision schema was used in 8 edge layers, and as every third middle layer as a drift prevention checkpoint.
The idea is simple. Keeping edge layers in higher precision should stabilize input comprehension/output composition (keeping them as unharmed as we can afford, given the size limits), and having checkpoints in an otherwise aggressive mixed 3/4bit ffn flow should let the model compensate for SOME quantization-borne drift. That, together with ik's advancements with trellis quantization and a bulkier imatrix dataset with an expanded RP corpus should keep the model sane. In theory.
Will it have a measurable mechanistic effect? Will such a ratio of layers be enough? Idk. That's an experiment and a love letter to Skyfall at the same time.

Quant Recipe

Quant Recipe

Input 0 1 2 3 4 5 6 7 8 9 10 11 12 ⋯13–48 49 50 51 52 53 54 Output Input / Output Higher Precision Lower Precision

Embeddings

High-Precision Blocks

Low-Precision Blocks

lm_head

Pattern: Blocks 0–3 high · Middle: 2 low + 1 high repeating · Blocks 51–54 high  ·  Total: 54 layers

imatrix generated using bartowski's combined dataset coupled with a random conversation 200k-token prune of Squish42/bluemoon-fandom-1-1-rp-cleaned.

quantized with ik_llama.cpp as of commit 43afea46c25a12aae6db1e3105643267164898b4 (Fri Aug 14 07:55:11 2026 +0200)

Cheers

MistralAI - the beloved base model(s).
ikawrakow and contributors of ik_llama.cpp - I probably misused your creation.
TheDrummer - you cooked a legendary one.
bartowski - for the combined dataset + the myriad of quants we all benefit from.
EmanuelOverride - for sparking my interest in custom quantization as a concept.