Loom Weave 3
31.5M parameters · 16 layers · 1024 context · Textile Labs
The first Loom that reads real prose. Successor to Loom Weave 2 — and deliberately smaller than it (Weave 2 was 59.65M). Trained from scratch: randomly initialised weights, nothing fine-tuned from anyone's checkpoint.
It decides when to look something up, writes the query, reads what comes back from live Wikipedia, and answers from it — saying it checked rather than knew. No earlier Loom could read real, messy prose; the family's whole weakness was that curated results read well and live articles didn't. This is the model that closes that gap.
you who wrote dracula
Loom Weave 3 <lookup>dracula</lookup>
harness ← Dracula is an 1897 Gothic horror novel by Irish author Bram Stoker.
Loom Weave 3 I think Irish author Bram Stoker. Worth saying I checked rather than knew it.
Why it's smaller than Weave 2
Weave 2 (59.65M) failed for method, not size — a thin instruction veneer over a pretraining register, train and eval written by the same hand. Weave 3 is half the size and far better, because tokens-per-parameter and how it's trained matter more than parameter count. That has precedent: Loom Spark 3 (12.2M) succeeded and beat Spark 2 (19.9M). Bigger is not the goal; behaviour is.
What changed
The reading skill is a real circuit, and it only formed once two things were right:
| lever | what went wrong before | fix |
|---|---|---|
| learning rate on the attention matrices | trained 8× too low, so the copy/reading circuit never formed — the model was fluent but couldn't pull "Paris" out of a sentence containing "Paris" | Muon LR set to the family standard (0.025); reading appeared |
| what the loss grades | it graded every token equally, spending most of its effort on raw web text | loss is masked to the reply tokens; whole conversations packed into each block |
| how long it trained | over-trained a tiny corpus into memorised, garbled output | short training, kept the best-validation checkpoint |
| admitting personal limits | learned "I can't know your name" but not "I can't know what you did" — so it hallucinated on other personal questions | added varied personal-unknowable examples; it now declines all of them |
The search harness
The model decides a search is needed and writes the query. harness.py does the rest:
- searches the model's query and the subject it can see in your question
- prefers the real article over lists, films, albums and disambiguation pages
- reads the intro first, further only when the intro has no answer of the right kind
- strips brackets and pronunciation guides, so real text looks like training text
- hands back one sentence, not a paragraph — a model this size misreads paragraphs
Measured behaviour
Same harness and settings a user gets. Run on 2026-09-20.
The acceptance battery, row by row (total ties the best in the Loom family):
| row | Loom Weave 3 |
|---|---|
| A · says its own name | 10/12 |
| B · its own name under rough typing | 10/12 |
| C · 5-turn conversation stays on thread | 5/5 |
| D · answers from a search result | 5/5 |
| E · follow-up answered from the same result | 2/5 |
| F · says it looked, after a lookup | 5/5 |
| G · never claims a lookup it didn't make | 16/16 |
| H · admits what it can't know about you | 8/8 |
| I · says when a result doesn't contain the answer | 0/5 |
| J · never leaks a search tag with tools off | 28/28 |
| K · stops on its own | 12/12 |
| L · searches when it should, not for your private things | 19/20 |
| total | 120/133 |
End to end, everyday questions it had never seen, live Wikipedia, the model writing its own query, scored on the final answer:
| set | decided to search | wrote its own query | answer reached the model | answered right |
|---|---|---|---|---|
| tuning | 20/20 | 20/20 | 14/20 | 11/20 |
| held-out | 20/20 | 20/20 | 12/20 | 6/20 |
It is the first Loom to read live prose above a coin-flip on the tuning set — but read the held-out number honestly: on unfamiliar, messy passages it is right about a third of the time.
Read this before you use it
Every point here was measured.
- Held-out live reading is ~30%. "I looked that up" means it searched — not that it read
the result correctly. Run
harness.py --showand trust the sentence it read. - It is not a calculator. Small sums are out of scope (0/6). Use a tool for arithmetic.
- Follow-up questions from the same result are weaker (~40%).
- It never says a result doesn't contain the answer — it answers from whatever it read.
- It rarely asks a clarifying question on an ambiguous request.
- It reads a supplied result far better than it reads live search (row D is 100%).
- Harness search is Wikipedia only — time, weather, news and prices can't be answered.
- It is a small reader/assistant, not a chat stylist. Voice is plain and brief by design.
What it does do, reliably: knows what it is, stops on its own, holds a conversation, admits what it can't know about you (8/8), and never claims a lookup it didn't make (16/16).
Usage — the harness
python3 harness.py "who wrote dracula"
python3 harness.py # interactive
python3 harness.py --show "how tall is mount everest" # see what it searched and read
python3 harness.py --no-tools "who are you"
Stdlib + certifi. Wikipedia needs no API key. Never feed a failed lookup back as a
result — the model will answer from the error text; harness.py fails loudly instead.
Usage — Ollama
ollama run hf.co/textilelabs/Loom-Weave-3 "who are you"
template and params are read automatically.
Usage — transformers
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Weave-3")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Weave-3").eval()
eot = tok.convert_tokens_to_ids("<|eot|>")
def ask(message, tools=False):
p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
with torch.no_grad():
out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,
pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
return tok.decode(out[ids.shape[1]:], skip_special_tokens=False).replace("<|eot|>", "").strip()
Prompt format is exact: <tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n.
How it was built
| architecture | Llama — 16 layers × 384d, GQA (6 heads / 2 KV), SwiGLU, RoPE, tied embeddings |
| parameters | 31,469,952 |
| context | 1,024 |
| vocabulary | 16,384 custom BPE |
| optimiser | Muon (LR 0.025) on the 2D hidden matrices, AdamW on embeddings and norms |
| loss | masked to the reply tokens; whole conversations packed per block (FFD) |
| schedule | warmup → stable → cosine decay (WSD); best-validation checkpoint kept |
| corpus | 173,043 conversations; grounded-reading rows carry real encyclopedic prose |
| training | ~1,280 steps · ~336M tokens seen · 10 epochs · from random init |
| hardware | Kaggle dual T4 (GPU) · ~2 hours |
Files
config.json / model.safetensors the model
tokenizer.json / tokenizer_config.json custom BPE tokenizer, 16,384 tokens
loom-weave-3-f16.gguf for Ollama / llama.cpp (lookup tags USER_DEFINED)
harness.py runnable search harness
template / params read automatically by `ollama run hf.co/...`
Modelfile for building locally
ATTRIBUTION.md required credits for the training corpora
Training data
| slice | source |
|---|---|
| grounded reading of real prose | SQuAD 2.0 (CC BY-SA 4.0) · Wikipedia (CC BY-SA) |
| when to reach for a tool | MASSIVE (CC BY 4.0) · CLINC150 (CC BY 3.0) |
| instruction following | databricks-dolly-15k (CC BY-SA 3.0) |
| multi-turn dialogue structure | OpenAssistant OASST1 (Apache 2.0) |
| tokenizer coverage of real prose | HuggingFaceFW/fineweb-edu (ODC-By 1.0) |
| identity, limits, warmth, attribution | Textile Labs — written for Loom |
No language model wrote any training query, and nothing is fine-tuned from anyone's checkpoint. No real user data was used. See ATTRIBUTION.md.
License
Model: MIT. Training data retains its original licences and attribution.