Loom Spark 1.8 Flash

Experimental micro variant · Textile Labs

Naming note: "Flash" here means the same thing it does for Gemini Flash — a smaller, faster variant of the family, not a shrunk copy of the numbered model. This is a separate model trained from scratch, not a distillation of Loom Spark 1.8 (18.85M params). Loom Spark 1.8 Flash is 2.62M parameters — about a seventh the size — and its smaller sibling is Loom Spark 1.5 Flash (1.35M).

Built in the same under-2-hour experiment as 1.5 Flash, at roughly double the capacity. Where 1.5 Flash asks "how small can this go and still work," 1.8 Flash asks "how much does doubling the size actually buy back" — quite a lot, on these numbers.

The headline result

Every prior full-size Loom release (v1, 1.5, 1.8) only learned <|endoftext|> at the very end of a whole training document — mid-conversation, nothing told the model a turn had ended, so on a runtime without exactly the right stop-token setup it would keep generating and invent your next message itself. This happened to the founder testing Loom Spark 1.8 in Ollama the day it shipped.

This model's curriculum was rebuilt so <|endoftext|> follows every single reply, not just document end. Verified: 442,333 / 442,333 model turns in the training corpus end in EOS. Tested with zero configuration — Ollama's stock chat template, no Modelfile, no stop tokens set by hand: 0 self-dialogue turns, both through the default chat endpoint and through the real agent harness with live web search. It stops because it learned to.

How it compares to 1.5 Flash

Both were trained on the identical 70MB curriculum and tokenizer. The only difference is capacity (128d×4L×4H vs 192d×4L×4H) and training budget, matched by step count (2,625 vs 2,666 steps) rather than wall-clock minutes — an earlier version of this model was trained for equal minutes instead of equal steps, which under-trained it relative to 1.5 Flash despite being bigger. That run was discarded; these are the numbers from the properly step-matched retrain.

Honest limitations — read this first

Still a very small model. Expect wrong or garbled answers to most factual questions — there's more room here than 1.5 Flash for identity and structure to hold up, but a real fact-core still doesn't fit. What holds up well: identity, restraint, and emotional register — 0/12 identity probes leaked, matching the best of the full-size models.

Prompt format

<tools:off>
<tools:off><user> who are you
<loom>

No trailing space after <loom>. With tools on, a reply may end in <lookup>query</lookup><|endoftext|>; your harness splices in <result>…</result> before continuing.

Usage — transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Spark-1.8-Flash")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Spark-1.8-Flash")

prompt = "<tools:off>\n<tools:off><user> who are you\n<loom>"
ids = tok(prompt, return_tensors="pt", add_special_tokens=False).input_ids
out = model.generate(ids, max_new_tokens=100, do_sample=True,
                     temperature=0.8, top_k=50, pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False))

Usage — Ollama

ollama create loom-spark-1.8-flash -f ollama/Modelfile
ollama run loom-spark-1.8-flash

Safe to run with no Modelfile at all — ollama run hf.co/textilelabs/Loom-Spark-1.8-Flash will not talk to itself, though output quality is better with the correct template.

Usage — the agent harness

pip install ./harness
loom-chat --model textilelabs/Loom-Spark-1.8-Flash

Harness v0.2.2+ required — auto-detects this model's format via an explicit flag in config.json rather than guessing from parameter count.

Training

• Hardware: CPU-only Dell OptiPlex 9020, i5-4690, 4 cores, no GPU
• 2,666 steps, batch 32 × 256 tokens, step-matched to 1.5 Flash's budget
• Corpus: same 70MB curriculum as Loom Spark 1.8, EOS after every model turn
• Final validation loss: 0.3332
• Architecture: 192d × 4 layers × 4 heads, vocab 4096 (fresh BPE, shared with 1.5 Flash, not shared with any full-size Loom generation)

Files

config.json / model.safetensors       transformers weights
tokenizer.json / tokenizer_config.json  4096-token custom BPE (Flash-specific)
loom-spark-1.8-flash-f32.gguf         GGUF for llama.cpp / Ollama
ollama/Modelfile                      correct template + stop tokens
harness/                              agent harness v0.2.2 with web search
evaluation/                           acceptance logs, comparison vs 1.5 Flash

License

MIT. See LICENSE.