Chibi Image Nexus v0.2
Experimental compression architecture for Logolabs/agate-preview-003.
Chibi Image Nexus v0.2 combines:
SVD low-rank decomposition + a small SwiGLU residual network
inside selected Linear layers of the Agate Generator.
The goal is to recover part of the representation lost by aggressive low-rank decomposition while keeping the Generator smaller.
Architecture
A normal Linear layer is:
x -> W -> y
Plain low-rank SVD approximates it as:
x -> B -> z -> A -> y
Chibi Image Nexus v0.2 inserts a small nonlinear SwiGLU residual inside the bottleneck:
Wu
|
v
x -> B -> z ------> SiLU -----
| |
| *
| |
| Wv
| |
| v
| Wo
| |
+------- residual ---+
|
v
A
|
v
y
Mathematically:
z = Bx
delta = Wo(
SiLU(Wu z) * Wv z
)
h = z + alpha * delta
y = A h + b
The output projection Wo is zero-initialized.
Therefore training starts from the ordinary SVD approximation.
Training strategy
The original Agate Linear layer is used as the teacher.
Real activations are collected from the original Generator.
Teacher:
x -> W -> y_teacher
Compressed student:
x -> B -> SwiGLU residual -> A -> y_student
Training minimizes reconstruction error between the teacher output and compressed output.
Training procedure:
- Initialize A and B using SVD
- Freeze A and B
- Train the small SwiGLU residual
- Optionally fine-tune A and B using a smaller learning rate
Compression result
Original Generator parameters:
194,270,952
Compressed Generator parameters:
189,030,740
Parameter reduction:
2.70%
Compressed Linear layers:
12
Visual comparison
The following comparisons use the same prompts and seeds for the original Agate model and Chibi Image Nexus v0.2.
Fox logo
Red robot
Blue ceramic cup
The compressed model does not reproduce the original image exactly.
However, semantic content and generation quality are retained substantially better than in the earlier plain fixed-rank SVD experiment.
Quantitative results
Same-seed comparison against the original model:
- Mean pixel MAE: 0.1204
- Mean PSNR: 13.58 dB
Layer reconstruction:
Average MSE recovery after SwiGLU training: 15.61%
Best layer MSE recovery: 31.58%
These metrics measure similarity to the original model output.
They should not be interpreted as general perceptual image-quality scores.
Why SwiGLU?
The v0.1 experiment used plain fixed-rank SVD.
v0.1
x
|
B
|
A
|
y
At aggressive rank reduction, generation quality degraded significantly.
v0.2 adds a small nonlinear correction inside the low-rank representation.
v0.2
x
|
B
|
z
|
+-------------------+
| |
| SwiGLU
| |
+------ residual ---+
|
A
|
y
The SwiGLU block attempts to recover part of the information that was lost during low-rank decomposition.
Model files
- generator.safetensors
- manifest.json
- nexus_loader.py
- layer_training.csv
- quality_metrics.csv
- comparison_prompt_00.png
- comparison_prompt_01.png
- comparison_prompt_02.png
- compressed_prompt_00.png
- compressed_prompt_01.png
- compressed_prompt_02.png
Usage
Example:
from nexus_loader import load_nexus
pipe = load_nexus(
"mikun0202/chibi-image-nexus-v0.2",
device="cuda",
)
image = pipe(
"a minimalist logo of a fox head, orange, flat design, white background",
seed=0,
steps=20,
watermark=False,
)[0]
image.save("output.png")
Base model
Base model:
Logolabs/agate-preview-003
This experiment compresses selected Linear layers inside the image Generator.
The text encoder and VAE are not compressed.
Status
This is an experimental research checkpoint.
The purpose of Chibi Image Nexus v0.2 is to investigate whether a small nonlinear SwiGLU correction can recover representations lost through aggressive low-rank SVD compression.
Future experiments may include:
- larger rank fractions
- different SwiGLU hidden widths
- layer-specific rank selection
- adaptive compression ratios
- automatic architecture search
- global end-to-end fine-tuning