Moon-Beam 16B-A3B

Tablinum's first trained model. Moon-Beam is Moonlight-16B-A3B-Instruct (16B total, 3B active, MIT), fine-tuned to use the Tablinum-2 engram well. It reads the retrieved passages, answers from them when they state the answer, and says it doesn't know when they don't. With one search over all 26 engram tables and no routing, the 8 GB IQ3 build answers 406 of 502 held-out EngramBench questions correctly with 38 wrong (index +73.3, from −46.8 closed-book), and gets 155 wrong on SimpleQA where the untrained model gets 371.

Quick links: GGUF files · Tablinum-2 engram · Moon-Beam collection · Lite-Beam family

What Moon-Beam is for

Moon-Beam is a private reference desk that runs on your own computer. Give it the Tablinum-2 knowledge library and it answers factual and how-to questions from 26 tables of reference text (Wikipedia, law and regulation, software documentation, home repair and DIY Q&A, philosophy, psychology, open textbooks, research papers), offline. When the passages don't hold the answer, it says it isn't sure instead of making one up.

Good uses:

• Looking things up without the internet: names, dates, definitions, what a law or regulation says, how an API or command works.
• Practical help at home: repairs, DIY, cooking and gardening, answered from real community Q&A.
• Studying and research: a careful companion that says what its sources say and admits gaps.
• Your own apps: an OpenAI-compatible endpoint (llama-server, mlx_lm.server) plus the retrieval step.

Not for:

• Live information. It cannot browse the web or check news, prices or weather. It knows its training and the passages it is given.
• Tools and agents. Moon-Beam 16B does not call tools. Moon-Beam 8B (in training) adds tool calls, web-page actions and thinking.
• Long multi-step math (GSM8K 71.8%). Use the base model or a calculator tool.
• Medical, legal or financial decisions. It can explain what a source says; it does not replace a professional.

One quirk to know: asked "can you search the web?" or "can you run this code?", Moon-Beam 16B sometimes says yes. It can't. It only has its training and the passages you give it. We tested default system prompts that tell it so, and every one cost accuracy on easy facts (20 of 24 right with the original prompt, 13 to 17 with the new ones), so the chat template is unchanged. Moon-Beam 8B is trained to answer questions about itself correctly.

The knowledge library is a separate download today (Tablinum-2). Without it Moon-Beam is a careful general chat model; with it, it is the reference desk above. A one-command setup that installs both together is coming with the next release.

The result

The index is 100 × (correct − incorrect) / n, so a wrong answer costs as much as a right one earns. The engram supplies the knowledge. The training adds calibration: with the passages in hand Moon-Beam finds the answer about as often as its base, but it stops guessing when the passages don't hold it. All arms are graded by Claude Haiku against the gold answers, paired on the same questions; questions the grader declined to grade are dropped for both models. None of these questions, or any passage retrieved for them, was used in training.

What the engram knows

EngramBench is ours: 24 questions per subject, written from passages held out of training, each with a short gold answer that appears in its passage. One retrieval over all 26 tables finds the right subject's passage in the top 3 for 95% of questions, so no per-domain router is needed. Searching everything costs about one index point against sending each question to its own table.

Files

The merged BF16 weights in the original Moonlight layout: 27 safetensors shards (31.9 GB), the Moonlight tokenizer (tiktoken.model) and its remote code. Quantized builds for llama.cpp are in GGUF files:

How Moon-Beam was made

Moon-Beam is a QLoRA fine-tune: rank 16 on the MLA attention, the shared experts and the dense layer, with the 64 routed experts frozen. It trained one epoch on one 12 GB GPU, on 2,198 examples written by Claude Haiku from passages of the 21 non-health Tablinum-2 tables:

• answer (72%): the question with the gold passage hidden among near-misses → a grounded answer
• open / near-open (17%): how-to questions with no references, or with related passages that don't answer → a real answer
• decline (11%): fact lookups (a name, number or date) the passages don't state → "I'm not sure of that exact date". Never on how-to questions.

Health tables never left the machine. It took three rounds: the pilot learned to decline everything, and round two answered everything briefly and confidently. Round three, the recipe above, keeps both the help and the honesty.

What it costs

The training data is short factual and how-to answers, and it costs some general skill. Moon-Beam's math working is shorter and it skips steps more often, and it sometimes answers a multiple-choice question in a sentence instead of a letter. On the questions it does answer with a letter, MMLU-Pro knowledge is unchanged. If you need multi-step math, use the base model or route math to a tool.

Run with transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("tablinum/Moon-Beam-16B-A3B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("tablinum/Moon-Beam-16B-A3B", torch_dtype="auto", device_map="auto", trust_remote_code=True)
msgs = [{"role": "user", "content": "..."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=512)[0][ids.shape[1]:], skip_special_tokens=True))

The tokenizer needs trust_remote_code (Moonlight's tiktoken tokenizer); recent transformers versions also load the model natively as DeepseekV3ForCausalLM. For llama.cpp use the GGUF repo.

Use the v1 injection prompt with retrieved passages:

Reference passages (retrieved automatically; they may be incomplete or irrelevant): … Use a passage only if it states the answer to this exact question. A passage about a related but different item, version, year, rule or case does not answer it: then rely on your own knowledge, or say you don't know.

What you need

• GGUF: about 9 GB of VRAM for IQ3 with a 32K context (about 6 GB for IQ2). MLA keeps the KV cache small.
• BF16: about 32 GB of memory (GPU, or CPU with offload).
• Optional: the Tablinum-2 tables and retrieval code. The engram runs locally.
• Native context is 8K, as in Moonlight. Longer contexts were not trained for this release.

License and attribution

MIT. Moonlight-16B-A3B-Instruct is by Moonshot AI (MIT). Moon-Beam is a fine-tuned derivative by Lightspeed / Tablinum. Keep both copyright notices with any redistribution. Training examples were written with Claude Haiku, and the character art is Tablinum's own. No endorsement by Moonshot AI or Anthropic is implied.

Citation

@misc{moonbeam16b2026,
  title  = {Moon-Beam 16B-A3B},
  author = {Lightspeed / tablinum},
  year   = {2026},
  note   = {Moonlight-16B-A3B fine-tuned to use the Tablinum-2 engram},
  url    = {https://huggingface.co/tablinum/Moon-Beam-16B-A3B}
}