🧠 Optimization: Custom APEX Quants
This repository contains custom APEX (Adaptive Precision for EXpert Models) quants built with the specific mixture-of-experts architecture in mind.
⚠️ Disclaimer: This repository is an independent open-source optimization project and is not officially affiliated with, endorsed by, or representing the ReadyArt team.
🔬 The Mixed-Precision Philosophy
- The APEX Edge: Uniform quantization compresses all layers equally, which degrades MoE routing logic. The APEX approach locks core routing blocks at high precision, protects vital shared experts, and offsets file size by compressing less sensitive mid-layer mixed expert blocks.
- The Custom Sweet Spot: Standard automated APEX scripts lack a balanced midrange option, forcing users to choose between a big
Q6or a smallQ3/Q4config. This project largely aims forQ5-equivalent precision at a roughQ4+memory footprint.
⚡ Standard APEX Suite
The default APEX configurations.
- i-Mini:
12,052.38 MiB MiB(3.89 BPW) — Extremely nimble, stable low-VRAM baseline. - i-Compact:
13,341.05 MiB(4.31 BPW) — The all-rounder baseline sweet spot. - i-Balanced:
19,436.36 MiB(6.28 BPW) — High-precision layer tracking with mid-expert compression. - i-Quality:
18,226.36 MiB(5.89 BPW) — Advanced Importance Quantization (iq4_xs/nl) core valley logic.
🚀 The 26B Custom Intermediary Suite
In my initial attempts at creating an intermediary size-tier between Compact and Balanced, I scaled up the basic logic of the Compact configuration by one quant level:
- i-Upscale.Premium:
14,600.94 MiB(4.85 BPW) — Performance King on legacy hardware. Employs a[Q5_K -> Q4_K -> Q3_K]stepped gradient across the mid-layer experts while shielding context gates. - i-Upscale.Balanced:
14,790.00 MiB(4.92 BPW) — FlatQ4_Kexpert mapping across all up/down tensors. Known and stable size-to-precision efficiency curve. - i-Upscale.Econ:
12,899.38 MiB(4.29 BPW) — AggressiveQ4_KtoQ3_Kmid-expert trimming. Extensive use of Q3_K and how the ratio influences lookup tables probably causes it to balloon in memory? My best guess. Probably non-viable econ solution and won't be done in the future.
🎵 The Harmonic Series (Pulsed Attention)
Following on from the Harmonic Resonance branch, the Harmonic layouts further optimize layer contiguity for performance, by following strict precision tiers across all 30 layers (High/Medium/Low), while priorizing high-precision tiers for Global Attention (GA) layers, preventing aggressive compression in order to anchor deep-context tracking. These are "HQ"-versions, meaning that routers stay uncompressed, while shared exps and attention layers shift up by one quant level overall.
- i-Harmonic.Minus.HQ:
14,685.61 MiB(4.88 BPW) — High-performance configuration. Condenses the architecture into 3 strict tiers (5/15/10). Shifts high-precision blocks to coincide with global attention pulses. In testing: Faster than Upscale.Premium, with better long-context detail attention. - i-Harmonic.Plus.HQ:
16,203.40 MiB(5.39 BPW) — Medium-High-precision/Performance configuration. Eliminates theQ3_Kcompression valley entirely to enforce a flat, 2-tier matrix (10/20). In testing: Replaces and outperforms Upscale.Balanced at slightly larger overhead, while shielding long-context detail attention.
🧠 The Mindful Dynamics Series (Asymmetric Shock-Absorption)
The Mindful layouts build on the Harmonic framework, balancing precision-tier distributions while implementing additional buffers around the GA-layers. These are "HQ"-versions, meaning that routers stay uncompressed, while shared exps and attention layers shift up by one quant level overall.
- i-Mindful.Minus.HQ:
16,091.27 MiB(5.35 BPW) — Experimental Semantic Anchor. Follows15/9/6distribution. Trades uniform end-precision for comprehensive buffering of all Global Attention layers. Expectations: Slight edge in context stability and logical coherence under heavy prompt load. - i-Mindful.Plus.HQ:
16,089.18 MiB(5.35 BPW) — Experimental Fluidity Engine. Built on a16/7/7topology. Trades early intermediate buffers for an uncompromised, clean high-precision runway across the final formatting layers. Expectations: Slight edge in inference speed and final execution.
💬 Join the Discussion & Co-Design!
- All configurations get uploaded to the repo. Everyone welcome to create or suggest their own designs, whether driven by specific hardware needs or fresh ideas.
- Observations are not gospel. Your hardware environment and prompt styles can be expected to create unique conditions for these layouts.
- Knowing when a custom quant is in demand saves it from being pruned. This is a gauntlet, where only a handful of layouts are expected to survive.
- Did you notice a specific variant handling a character's tone perfectly?
- Did one layout throw an odd loop that another avoided?
- Are you seeing different token-per-second (T/s) behavior on your rig?
Please open a thread, share your experience, and post your impressions.
✦ Melody1437-26B-A4B ✦
v2.0
✨ It's a fabulous day! ✨
⚡ Evolutionary Upgrade: This is the same dataset as Melody1437-26B-A4B, retrained at a higher LoRA rank (64) for enhanced expressiveness and capability. CONTENT ADVISORY: MATURE THEMES / ADULT INTERACTIONSThis model is designed for mature audiences. Viewer discretion is advised. Your browser does not support the video tag.
🔄 What Changed?
This model builds upon the foundation of Melody1437-26B-A4B, using the identical dataset but with an upgraded LoRA configuration. The higher rank allows the model to capture more nuanced patterns and deliver richer, more expressive outputs.
Original LoRA 24 ➡️ Evolved LoRA 64- 🧠 Same Dataset: Identical training data to Melody1437-26B-A4B
- ⚡ Higher LoRA Rank: 64 (vs original 24) for greater expressiveness
- 🎭 Enhanced Nuance: Captures subtleties the lower rank couldn't fully express
🧬 Synthetic Life Engine
The dataset was generated using our advanced Character Engine and Emotional Engine within the synthetic dataset generator, breathing genuine life into every interaction.
🎭 Character Engine
Ensures consistent personality traits, speech patterns, and behavioral logic across all contexts. No more broken personas.
💓 Emotional Engine
Injects dynamic emotional states into responses, creating depth, empathy, and realistic reactions beyond simple pattern matching.
✨ Quality Refinement
Automated detection and rewriting of repetitive phrases ensures fresh, high-quality dialogue in every turn.
💬 Dialogue Integrity
Advanced quote normalization ensures perfectly balanced dialogue markers, preventing formatting errors in roleplay.
🔮 Training Process
The model weights were subjected to iterative refinement during data creation. Each conversation underwent multiple checks to ensure stability and alignment with the desired interaction style.
- 🔄 Multi-Turn Generation: Conversations were built turn-by-turn, allowing context to flow naturally.
- 🛡️ Refusal Filtering: Automated systems detected and removed unwanted refusals during dataset creation.
- 🧹 Slop Cleaning: Undesirable phrases were identified and rewritten by dedicated assistant models.
🛠️ Training Parameters
This model was fine-tuned using LoRA (Low-Rank Adaptation) with the following configuration:
| Parameter | Value |
|---|---|
| Training Method | LoRA (Low-Rank Adaptation) |
| LoRA Rank (r) | 64 |
| Epochs | 3 |
📚 Dataset Overview
The model's knowledge base was imprinted upon a Explicit Adult ERP dataset, weaving a vast tapestry of knowledge into its weights during the initial training. This ensures a rich foundation for all future interactions.
- 🔞 Content Rating: Strictly 18+ (Adults Only)
- 🎭 Focus: Erotic Roleplay, Mature Themes, Uncensored Dialogue
- 🌈 Diversity: Generated using variable system prompts and emotional states for varied responses.
📜 Version Notes
- ⚠️ Content Warning: This model is capable of generating NSFW content. Use responsibly.
- 🧹 Clean Output: Reasoning tags and excessive markdown have been stripped for cleaner roleplay.
- ⬆️ Evolution: Same dataset as Melody1437-26B-A4B, retrained at LoRA 64 for enhanced performance.
🎭 Usage Guidelines
To unlock the full potential of this model, a structured roleplay system prompt is required. This model is optimized for immersive character interaction rather than standard instructional queries.
- 📝 Initialization: Begin sessions with a clear character definition and scenario context.
- 🎭 Format: Use asterisks for actions
*like this*and quotes for dialogue"like this". - ⚠️ Limitation: Standard query-based prompting may result in refusals.
- 💡 Tip: The model expects balanced quotes. Ensure your input dialogue is properly formatted.
⚠️ Technical Note
Only text layers were trained on.
⚙️ Configuration
🎛️ Sampling Config
top_p: 0.95
temp: 0.8
min_p: 0.03
📦 Quantized Files
GGUF Quants Below
View Repository💖 Credits
- GECFDO – Data Generation & iMatrix Quants
- Sleep Deprived – Dataset Generator
- FrenzyBiscuit – Fine-Tuning & Dataset Creation
🔖 License & Usage
⚠️ USAGE AGREEMENT:
- 📜 This model is built upon the Apache 2.0 license.
- 🛡️ You accept full responsibility for all outputs generated.
- 🔞 You confirm you are at least 18 years old.
- 🌍 The creators bear no responsibility for how the model is used.
- 🏡 To the extent legally allowed under the original decree this model is for personal use only (non-profit/non-commercial).