Queen v5 “matriarch d20” — Crawlnet

A small language model trained from scratch. Pretraining data: pages the Crawlnet crawlers read on the crypto web (real browsers; 59% of the tokens trained), public market, funding and on-chain records written out as plain-text pages (6%), study notes a general-purpose LLM wrote for 113,078 of those pages (27%) and question/answer pairs a general-purpose LLM generated from those pages (8%); details in Pretraining data below. Tokenizer: Queen v4's, reused unchanged (trained on its crawler pages and record pages only). No pretrained weights, no FineWeb, no other corpus. Chat tuning: question/answer pairs generated by a general-purpose LLM from those pages (with the study notes' own Q&A), pairs cut from the pages and open-book conversations that answer from retrieved pages with a quote (details below).

Evaluation

Crypto knowledge (plain-frame log-likelihood multiple choice, acc_norm %; Queen rows are the chat-tuned releases, the other models their base checkpoints):

Every model is scored the same way: the question in a plain frame, each option's log-likelihood normalised by its length (acc_norm); 95% bootstrap intervals are about ±7 points on cryptomc/cryptoterms/projectid and ±1.7 on DMind. queen-v5 d20 vs queen-v4: better on all four, McNemar p < 0.001 each. cryptomc (150), cryptoterms (150) and projectid (200) come from crawled pages and were checked against the training data (matching documents left out); DMind (DMindAI/DMind_Benchmark objective set, 2,874 questions after removing ambiguous ones) was not part of that check, and on DMind picking the longest option scores 47.2, so treat it as a rough signal.

Reading pages from sites it never saw (400 questions about pages from 169 held-out sites (no page of those sites was trained on); the page and one page from another held-out site are put in front of the question in the model's own open-book format; % of answers holding every content word of the reference answer, and word F1):

queen-v5 d20 quoted a sentence in 91% of its answers (89% verbatim in the pages) and said "The pages don't say." in 9%. Without the pages it scores 17.8% (queen-v4: 11.3%).

LLM-graded answers (the 100 fixed crypto questions of queen_train/judge_questions.json, one seeded answer each (temperature 0.7, top-k 50), graded by a general-purpose LLM: mean correctness 0-2, share fully correct, share repetitive):

With pages: the site's search over the crawl, as served. When queen-v5's open-book reply is "The pages don't say." (45 of 100 here: the search returned no page answering a general question) the question is asked again without pages (Queen v5's serving code does the same). Lower is better for repetitive.

Pretraining data

Unique tokens: every document trained at least once, counted once (with its BOS token and source line). Tokens trained: every pass of it, plus the Q&A documents regrouped for the end of pretraining. The run trained 4,462 steps of 524,288 tokens = 2,339,373,056 of the stream's 2,339,400,113.

Contributors

Queen v5 was trained on pages found by these crawlers. Owners spawned theirs by burning $CRAWLNET; the wallets are the ones that made the burn, public on-chain. Since 9 Oct 2026 every crawl.md NFT is a crawler of its own: its pages are credited to the wallet that holds the NFT (its credits name when one is set).

Pages and tokens count what made it into this version's dataset after cleaning and dedupe. Machine-readable: contributors.json. The 45,627 record pages (see below) were not found by a crawler and are not in this table.

What went in besides crawler pages

• The nanochat training code, architecture and hyperparameter rules (no weights, no data).
• Six hand-written question templates used to turn page headings into questions (see SFT above).
• Record pages: 45,627 pages, about 10,839,462 tokens, of public market, funding and on-chain records (projects and their funding rounds, investors, token unlocks and token facts, DeFi protocols, chains and stablecoins, DEX and on-chain token statistics), written out as plain text from public data. Only factual fields were kept: no third-party labels, ratings or risk scores, and no social-media posts. They are in the pretraining data, the tokenizer and the generated chat pairs; the pairs cut from page headings use crawler pages only.
• Study notes: a general-purpose LLM was given 113,950 crawled pages and wrote study notes from each: an explanation, an FAQ, a list of facts and entities and a Q&A list. A fact checker dropped every item with a number, address, URL or ticker that is not on its page (15,567 items); no page of a held-out site was given to it, and 5,050 notes that matched a benchmark question were left out. The notes of 113,078 pages are in the pretraining data and their FAQ and Q&A pairs in the chat tuning; their wording, and any mistake the checker could not catch, is in this model.
• Generated Q&A pairs in pretraining: 964,943 question/answer pairs a general-purpose LLM generated from crawler pages and record pages for Queen v4's chat tuning, written as Q&A documents. 39,964 more (a fixed 2% of the questions and the earlier validation split) and 303 that matched a benchmark question never entered pretraining.
• Chat-tuning question/answer pairs generated by a general-purpose LLM from crawler pages and record pages, with the study notes' FAQ and Q&A pairs and open-book conversations built from them (see SFT above).
• The open-book frame of the chat tuning: two fixed replies, "The pages don't say." and "The pages disagree.", and the Quote: / Answer: reply lines.
• Nothing else. nanochat's default SFT mixture (SmolTalk, MMLU, GSM8K) and its CORE eval were switched off.

What it says

Fixed prompts, seeded sampling (temperature 0.7, top-k 50), unedited:

What is Bitcoin?

Bitcoin is a decentralized, peer-to-peer digital currency that operates on a public network without central authorities. It is the first and most widely adopted cryptocurrency, created in 2009 by an anonymous person or group under the pseudonym Satoshi Nakamoto.

Who created Bitcoin?

Bitcoin was created by an individual or group known as Satoshi Nakamoto, who published the Bitcoin whitepaper in 2008 and released the software in 2009.

What is a blockchain?

A blockchain is a distributed ledger where transactions are recorded in chronological blocks, maintained by a network of validators who use consensus algorithms to agree on and record transactions.

Explain proof of work.

Proof of work (pronounced "woot"[1]) is a consensus algorithm for permissionless blockchains that uses mining to decide on the ledger’s state. The mining process involves performing complex calculations that require significant computational power and energy. It is designed to be difficult to produce a hash value that meets the network’s difficulty target, allowing miners to be rewarded with block rewards. The concept of Proof of Work (PoW) was first formally introduced in 1993 for the first cryptocurrency, Bitcoin.

Explain proof of stake.

The proof of stake (PoS) consensus mechanism has been around since 2012, when Sunny King and Scott Nadal created the first major cryptocurrency as a response to the energy consumption issue that PoW has had. PoS is a consensus mechanism whereby participants of the network are required to stake (lock up) or lock up (stake) coins in order to get the ability to validate transactions. Validators are chosen based on the size of their stake. The more you stake, the higher your chances of being selected as a validator.

What is Ethereum?

Ethereum is the second-largest cryptocurrency by market capitalization and the most popular blockchain platform for creating decentralized applications.

What is a smart contract?

A smart contract is a computer protocol intended for the issuance and transfer of tokens, and it is a type of blockchain application.

What is gas on Ethereum?

Gas is the unit measuring the computational work needed to execute a transaction or smart contract on Ethereum, paid for in Ether (ETH).

What is Solana?

Solana is a high-performance blockchain platform designed to support scalable applications and decentralized finance (DeFi) and other blockchain-based services.

Why is Solana fast?

Solana is fast because it is a high-throughput Layer 1 built for frequent transactions. In its typical architecture, validators process transactions in parallel and reach consensus quickly. This model allows the network to handle thousands of operations per second at low cost. The speed is further supported by Proof of History, a cryptographic clock that orders events before they reach consensus. This approach enables the network to process thousands of transactions per second without sacrificing security.

Use it

chat.py in this folder needs only karpathy/nanochat (MIT) at the commit above:

pip install torch tiktoken safetensors
git clone https://github.com/karpathy/nanochat && git -C nanochat checkout 92d63d4e8bb4
PYTHONPATH=nanochat python chat.py "What is staking?"

Or load it yourself: config.json has the nanochat GPTConfig; build nanochat.gpt.GPT, load model.safetensors; the tokenizer is tokenizer.tiktoken (tiktoken rank file) + the pattern and special tokens in tokenizer.json. Chat format: <|bos|><|user_start|>...<|user_end|><|assistant_start|>.

Limitations

Tiny model, narrow data. It makes things up, including prices, dates, addresses and links. It is not financial advice and must not be used as a source of facts.