Second model of the GPT-2 join suite: DistilGPT2 (6 layers, 768-d) distilled onto the same frozen shared basis P as kofdai/vera-gpt2-matryoshka.
Honest framing: prior-art model-stitching probe at personal scale — not a universal coordinate dictionary. P is reused from model A (reuse bias). Join ≫ random/shuffle is a weak lineage / shared-rep signal (python -m vera audit). Scale-up unverified (python -m vera about).
Used for cross-model coordinate join demos (Step 5 JOIN_VIABLE at GPT-2 class): mid-layer coords from GPT-2 → DistilGPT2 stay nearer solo than random/shuffle controls.
| File | Role |
|---|---|
model.safetensors |
Fine-tuned DistilGPT2 weights |
vera_basis.npz |
DistilGPT2 per-layer means + shared P (same dictionary as GPT-2 Matryoshka) |
config.json |
kind=hook_container, role=join_partner |
python -m vera download-gpt2-suite
# pulls:
# kofdai/vera-gpt2-matryoshka
# kofdai/vera-distilgpt2-join
Or select GPT-2 (Vera suite) in python -m vera ui → Convert/Chat — both repos are fetched.
| Direction | Site | Real join vs solo | Random control |
|---|---|---|---|
| GPT-2 → DistilGPT2 | late | ~1.01× solo ppl | tens–hundreds× worse |
| DistilGPT2 → GPT-2 | early | ~1.07× | collapse |
Details: CROSS_MODEL_JOIN.md in the Vera repo.
from vera.runtime import load_from_hf
partner = load_from_hf("kofdai/vera-distilgpt2-join", rank=256)
Must be paired with the Matryoshka GPT-2 bundle that shares the same P columns.
Research partner checkpoint for stereo-cross join — not a standalone chat model.
model.safetensors 328 MBvera_basis.npz 3 MBdistilbert/distilgpt2