Loom Tapestry 3
69.2M parameters · 40 layers · 1024 context · Textile Labs
The best Loom yet, and the first one grown. We took our 31.5M Loom Weave 3, stacked its 16 layers into 40 (function-preserving, so the bigger model started exactly where Weave 3 left off), and continue-trained it for two hours on a richer corpus. It reads live prose, reasons across two facts (multi-hop), and stays a disciplined Loom — it never turned into a story-generator.
Trained from-scratch lineage (no third-party checkpoint), on dual-T4 gpus in ~2 hours. Runs offline via Ollama; llama.cpp-compatible.
you which magazine was started first, Arthur's or First for Women?
Loom Tapestry 3 <lookup>arthur's magazine</lookup>
harness ← Arthur's Magazine was first published in 1844. First for Women started in 1989.
Loom Tapestry 3 Arthur's Magazine. I had to look that up.
How it was grown
Cold-training a big model on little data fails (undertrained). Instead we grew a trained small one: interleave its 16 layers among 40 positions, initialise the 24 new layers as no-ops (zeroed output projections), so the 40-layer model is identical to Weave 3 at step 0 — then it improves from there. Init loss was 2.2, not 9.7. All two hours went into getting better, not starting over.
Measured behaviour vs the family (same harness, 2026-09)
| model | params | battery /133 | live search (held-out) | reads real prose |
|---|---|---|---|---|
| Loom Tapestry 2 | 22.8M | 107 (80.5%) | — | curated only |
| Loom Spark 3 Flash | 7.18M | 119 (89.5%) | 5/20 | curated only |
| Loom Spark 3 | 12.2M | 120 (90.2%) | 7/20 | curated only |
| Loom Weave 3 | 31.5M | 120 (90.2%) | 6/20 | yes |
| Loom Tapestry 3 | 69.2M | 123 (92.5%) | 12/20 | yes + multi-hop |
The acceptance battery, row by row:
| row | Loom Tapestry 3 |
|---|---|
| A · says its own name | 12/12 |
| B · its own name under rough typing | 11/12 |
| C · 5-turn conversation stays on thread | 5/5 |
| D · answers from a search result | 5/5 |
| E · follow-up from the same result | 2/5 |
| F · says it looked, after a lookup | 5/5 |
| G · never claims a lookup it didn't make | 16/16 |
| H · admits what it can't know about you | 8/8 |
| I · says when a result doesn't contain the answer | 2/5 |
| J · never leaks a search tag with tools off | 28/28 |
| K · stops on its own | 12/12 |
| L · searches when it should, not for your private things | 17/20 |
| total | 123/133 |
End to end, everyday questions it had never seen, live Wikipedia, model writing its own query, scored on the final answer: tuning 12/20 (60%), held-out 12/20 (60%) — double Weave 3's held-out reading.
Read this before you use it
Every point measured.
- "I looked that up" means it searched — not that it read perfectly. Held-out reading is
~60%; run
harness.py --showand trust the sentence it read. - Follow-up questions from the same result are weak (~40%). The family's oldest gap.
- It is not a calculator. Arithmetic is out of scope; use a tool.
- It rarely asks a clarifying question on an ambiguous request.
- Harness search is Wikipedia only — no time, weather, news or prices.
- It is a small reader/assistant, not a chat stylist. Voice is plain and brief by design.
What it does reliably: knows what it is, stops on its own, holds a conversation, admits what it can't know (8/8), never claims a lookup it didn't make (16/16), reads a supplied result (5/5), and reasons across two facts.
Usage — the harness
python3 harness.py "who wrote dracula"
python3 harness.py --show "how tall is mount everest"
python3 harness.py --no-tools "who are you"
Stdlib + certifi. Wikipedia needs no API key.
Usage — Ollama
ollama run hf.co/textilelabs/Loom-Tapestry-3 "who are you"
template and params are read automatically.
Usage — transformers
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Tapestry-3")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Tapestry-3").eval()
eot = tok.convert_tokens_to_ids("<|eot|>")
def ask(message, tools=False):
p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
with torch.no_grad():
out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,
pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
return tok.decode(out[ids.shape[1]:], skip_special_tokens=False).replace("<|eot|>","").strip()
Prompt format is exact: <tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n.
How it was built
| architecture | Llama — 40 layers × 384d, GQA (6 heads / 2 KV), SwiGLU, RoPE, tied embeddings |
| parameters | 69,237,120 |
| grown from | Loom Weave 3 (16 layers) via function-preserving layer stacking |
| context | 1,024 |
| vocabulary | 16,384 custom BPE |
| optimiser | Muon (LR 0.025) on 2D hidden matrices, AdamW on embeddings and norms |
| loss | masked to the reply tokens; whole conversations packed per block (FFD) |
| corpus | Loom curriculum + HotpotQA multi-hop + unanswerable rows (real prose in results) |
| training | ~500 steps · ~67M tokens · best-validation checkpoint · from random-init lineage |
| hardware | Kaggle dual T4 (GPU) · ~2 hours |
Files
config.json / model.safetensors the model
tokenizer.json / tokenizer_config.json custom BPE tokenizer, 16,384 tokens
loom-tapestry-3-f16.gguf for Ollama / llama.cpp (lookup tags USER_DEFINED)
harness.py runnable search harness
template / params read automatically by `ollama run hf.co/...`
Modelfile for building locally
ATTRIBUTION.md required credits for the training corpora
License
Model: MIT. Training data retains its original licences and attribution.