Queen v6 “queen” — Crawlnet

A small language model trained from scratch. Pretraining data: pages the Crawlnet crawlers read on the crypto web (real browsers; 65% of the tokens trained), public market, funding and on-chain records written out as plain-text pages (4%), study notes a general-purpose LLM wrote for 153,245 of those pages (24%) and question/answer pairs a general-purpose LLM generated from those pages (7%); details in Pretraining data below. Tokenizer: trained on this dataset's pages only (crawler pages and record pages). No pretrained weights, no FineWeb, no other corpus. Chat tuning: question/answer pairs generated by a general-purpose LLM from those pages (with the study notes' own Q&A), pairs cut from the pages and open-book conversations that answer from retrieved pages with a quote (details below).

Evaluation

Crypto knowledge (plain-frame log-likelihood multiple choice, acc_norm %; Queen rows are the chat-tuned releases, the other models their base checkpoints):

Every model is scored the same way: the question in a plain frame, each option's log-likelihood normalised by its length (acc_norm); 95% bootstrap intervals are about ±7 points on cryptomc/cryptoterms/projectid and ±1.7 on DMind. queen-v6 vs queen-v5 d20: higher on DMind (McNemar p 0.02), cryptoterms (p 0.02) and projectid (p 0.08). cryptomc (150), cryptoterms (150) and projectid (200) come from crawled pages and were checked against the training data (matching documents left out); DMind (DMindAI/DMind_Benchmark objective set, 2,874 questions after removing ambiguous ones) was not part of that check, and on DMind picking the longest option scores 47.2, so treat it as a rough signal.

Reading pages from sites it never saw (400 questions about pages from 169 held-out sites (no page of those sites was trained on); the page and one page from another held-out site are put in front of the question in the model's own open-book format; % of answers holding every content word of the reference answer, and word F1):

queen-v6 quoted a sentence in 90% of its answers and said "The pages don't say." in 10%. Without the pages it scores 19.5% (queen-v5 d20: 17.8%).

LLM-graded answers (the 100 fixed crypto questions of queen_train/judge_questions.json, one seeded answer each (temperature 0.7, top-k 50), graded by a general-purpose LLM: mean correctness 0-2, share fully correct, share repetitive):

No pages: the model answers from what it learned, the way the site serves its drafts. Lower is better for repetitive.

Pretraining data

Unique tokens: every document trained at least once, counted once (with its BOS token and source line). Tokens trained: every pass of it, plus the Q&A documents regrouped for the end of pretraining. The run trained 6,312 steps of 524,288 tokens = 3,309,305,856 of the stream's 3,309,666,269.

Contributors

Queen v6 was trained on pages found by these crawlers. Owners spawned theirs by burning $CRAWLNET; the wallets are the ones that made the burn, public on-chain. Since 9 Oct 2026 every crawl.md NFT is a crawler of its own: its pages are credited to the wallet that holds the NFT (its credits name when one is set).

Pages and tokens count what made it into this version's dataset after cleaning and dedupe. Machine-readable: contributors.json. The 45,629 record pages (see below) were not found by a crawler and are not in this table.

What went in besides crawler pages

• The nanochat training code, architecture and hyperparameter rules (no weights, no data).
• Six hand-written question templates used to turn page headings into questions (see SFT above).
• Record pages: 45,629 pages, about 10,514,046 tokens, of public market, funding and on-chain records (projects and their funding rounds, investors, token unlocks and token facts, DeFi protocols, chains and stablecoins, DEX and on-chain token statistics), written out as plain text from public data. Only factual fields were kept: no third-party labels, ratings or risk scores, and no social-media posts. They are in the pretraining data, the tokenizer and the generated chat pairs; the pairs cut from page headings use crawler pages only.
• Study notes: a general-purpose LLM was given 154,643 crawled pages and wrote study notes from each: an explanation, an FAQ, a list of facts and entities and a Q&A list. A fact checker dropped every item with a number, address, URL or ticker that is not on its page (19,715 items); no page of a held-out site was given to it, and 7,296 notes that matched a benchmark question were left out. The notes of 153,245 pages are in the pretraining data and their FAQ and Q&A pairs in the chat tuning; their wording, and any mistake the checker could not catch, is in this model.
• Generated Q&A pairs in pretraining: 965,100 question/answer pairs a general-purpose LLM generated from crawler pages and record pages for Queen v4's chat tuning, written as Q&A documents. 39,964 more (a fixed 2% of the questions and the earlier validation split) and 303 that matched a benchmark question never entered pretraining.
• Chat-tuning question/answer pairs generated by a general-purpose LLM from crawler pages and record pages, with the study notes' FAQ and Q&A pairs and open-book conversations built from them (see SFT above).
• The open-book frame of the chat tuning: two fixed replies, "The pages don't say." and "The pages disagree.", and the Quote: / Answer: reply lines.
• Nothing else. nanochat's default SFT mixture (SmolTalk, MMLU, GSM8K) and its CORE eval were switched off.

What it says

Fixed prompts, seeded sampling (temperature 0.7, top-k 50), unedited:

What is Bitcoin?

Bitcoin is a decentralized peer-to-peer payment network that lets people transfer value directly over the internet without banks or intermediaries.

Who created Bitcoin?

Bitcoin was created by an anonymous person or group using the pseudonym Satoshi Nakamoto. The name is a reference to the creator of Bitcoin, who designed the system to be decentralised and independent from any central authority. Nakamoto's vision was to create an electronic cash system that uses cryptography to secure transactions and to solve the double-spending problem without the need for a central authority like a bank.

What is a blockchain?

A blockchain is a distributed database that records transactions between two parties in a secure, transparent, and immutable manner.

Explain proof of work.

The security of the blockchain is not guaranteed. The Bitcoin blockchain has never been successfully attacked and the only way to do so is to redo all the proof of work for the block containing the transaction. Since the transaction is in a block, the attacker must also redo the proof of work for that block and all subsequent blocks. A more complex attack occurs when someone is mining with an old, still valid block. For example, let’s say the latest block is 8. The attacker must also redo the proof of work for block 8. Then, the attacker must also redo the

Explain proof of stake.

In the case of Ethereum (ETH) a proof of stake consensus protocol was also proposed to increase the scalability of the network. But it was rejected by developers in the last core developers meeting of the Ethereum network.

What is Ethereum?

Ethereum is a decentralized platform and blockchain that enables the creation of smart contracts and decentralized applications (dApps), expanding beyond a peer-to-peer electronic cash system.

What is a smart contract?

A smart contract is a digital agreement written into code that executes automatically when conditions are met, without requiring a trusted middleman.

What is gas on Ethereum?

Gas is a unit that measures how much computational effort is required to execute operations on the network, with fees paid in ETH.

What is Solana?

Solana is a high-throughput blockchain platform designed for speed and low cost, using Proof-of-History alongside Proof-of-Stake.

Why is Solana fast?

Speed is one of the biggest reasons Solana is a top choice for traders and investors in 2023. Solana can process 65,000 transactions per second (TPS), which is a significant leap from Ethereum’s 15 TPS. This means that Solana is able to handle a large number of transactions in a short amount of time. In order to achieve this, Solana relies on a combination of technologies. One of the most important is Proof of History (PoH) and Tower BFT, which are used to ensure that the network is secure and efficient.

Use it

chat.py in this folder needs only karpathy/nanochat (MIT) at the commit above:

pip install torch tiktoken safetensors
git clone https://github.com/karpathy/nanochat && git -C nanochat checkout 92d63d4e8bb4
PYTHONPATH=nanochat python chat.py "What is staking?"

Or load it yourself: config.json has the nanochat GPTConfig; build nanochat.gpt.GPT, load model.safetensors; the tokenizer is tokenizer.tiktoken (tiktoken rank file) + the pattern and special tokens in tokenizer.json. Chat format: <|bos|><|user_start|>...<|user_end|><|assistant_start|>.

Limitations

Tiny model, narrow data. It makes things up, including prices, dates, addresses and links. It is not financial advice and must not be used as a source of facts.