Escarda-86M-Identity

Identity-tuned chat variant of Escarda-86M (SFT epoch 3). ~86M SpikeWhaleLM (JEPA + HRM refine), ChatML-aware SpikeTokenizer. It knows it is "Escarda" and answers like a clean assistant.

Usage

Custom architecture + tokenizer. trust_remote_code=True:

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Quazim0t0/Escarda-86M-Identity", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Quazim0t0/Escarda-86M-Identity", trust_remote_code=True)

ChatML (<|im_start|>role\n…<|im_end|>). Generation starts after a trailing <|im_start|>assistant\n.

Architecture

SpikeWhaleLM (~86M, 16 layers, hidden 640, 4096 context, 16,512 vocab, tied embeddings): MLA (LoRA-rank-128 Q/O, decoupled RoPE-16 + NoPE-48, multi-query, QK-norm), engram n-gram memory, ×2 hash-lookup, hyper-connections, HRM refine, MTP training head, and JEPA. Escarda uses both (use_hrm_refine=True, use_jepa=True).

Tokenizer

SpikeTokenizer - byte-level length-max (greedy longest-match), 16,512 vocab, ChatML atomic specials. PreTrainedTokenizer. Load with AutoTokenizer + trust_remote_code.

Evaluation

Zero-shot, full val/test splits (acc = raw continuation log-likelihood, acc_norm = byte-length-normalized).

ArithMark-2.0 (AxiomicLabs)

• official metric is raw acc: 0.3628 (strongest of the Escarda family).

Language modeling: WikiText-2 byte-ppl ↓ 2.7062 · BLiMP ↑ 0.7133.

Live demo: Escarda-86M-Chat Space.

Citation

If you use this model, please cite:

@misc{escarda86midentity,
  title        = {Escarda-86M-Identity: A ~86M-parameter SpikeWhaleLM},
  author       = {Dean Byrne (Quazim0t0)},
  year         = {2026},
  howpublished = {HuggingFace, \url{https://huggingface.co/Quazim0t0/Escarda-86M-Identity}},
  note         = {Quazim0t0/Escarda-86M-Identity}
}

Update: format-blended SFT on the engram-repaired base

This revision applies the (behavior-preserving) engram repair, then a short instruction/format SFT on a 60/25/15 blend of HuggingFaceTB/smoltalk, GSM8K-train (with '#### N' reasoning), and MMLU-style ('Answer: ') examples -- so chat fluency improves while the benchmark output-formats are preserved rather than overwritten. Held-out (test-split) before->after:

MMLU acc 0.188->0.284, format 0.700->0.958; GSM8K '####' 0.530->0.735

Note: these are fluency + output-format gains. Benchmark accuracy remains near the floor for a model this size -- the SFT does not add reasoning ability.