Carbon-X — MLX MXFP8

MLX MXFP8 quantization of HuggingFaceBio/Carbon-X, a Qwen3ForCausalLM fine-tune (14B dense, 40 layers, hidden 5120, GQA 40/8 heads, standard architecture — no custom modeling code) extended with a hybrid DNA tokenizer: alongside the normal Qwen3 BPE vocabulary (151,643 tokens), the embedding table carries 5,515 additional rows for DNA 6-mer tokens (dna_start_id 151669), so the model can read <dna>ACGT...</dna>-tagged nucleotide sequences as single tokens per 6-base chunk instead of spelling them out letter-by-letter. Runs on Apple Silicon via mlx-lm.

⚠️ Important: DNA tokenization needs the Python tokenizer

The DNA 6-mer vocabulary (ids 151669–157183) exists only inside the repo's custom tokenizer.py (HybridDNATokenizer) — it is generated in Python at load time and is not present in tokenizer.json. This means:

• Plain-text chat / Q&A works exactly as expected through mlx-lm, LM Studio, or any OpenAI-compatible server using the stock tokenizer — this is what's covered by the verification below.
• Structured DNA tasks (variant scoring, <dna>...</dna> sequence analysis, the skills/carbon-dna workflow from the base model) require running the tokenizer with trust_remote_code=True so HybridDNATokenizer builds the DNA vocabulary — e.g. transformers.AutoTokenizer.from_pretrained( "sahilchachra/Carbon-X-MXFP8", trust_remote_code=True), or any serving stack (vLLM, etc.) that loads custom tokenizer code. mlx-lm's load() and LM Studio's indexer do not execute tokenizer.py, so a raw <dna>...</dna> string typed into either will silently fall back to byte/BPE tokenization of the literal characters instead of 6-mer DNA tokens — it will not error, but it is not doing genomics inference.
• This repo keeps tokenizer.py, dna_config.json, and tokenizer_config.json (with auto_map intact) exactly as shipped, so the hybrid tokenizer is available to any Python-side consumer that opts into trust_remote_code.

Other quantizations of this model

• MXFP4
• OptiQ-5bpw

Verification

Text coherence (native mlx-lm, deterministic generation): "What is the capital of France?" → "The capital of France is Paris."

DNA embedding path — tested by encoding a real reference/variant DNA pair with the actual HybridDNATokenizer (bypassing the plain tokenizer, which has no DNA vocab) and feeding the resulting ids directly into each MLX checkpoint to score the log-likelihood of a single-base substitution:

All three builds agree in sign and magnitude (the single-base substitution is consistently scored as markedly less likely than the reference), confirming the DNA token embeddings and the hybrid vocabulary survive quantization correctly — the DNA-specific weight rows are not degraded or dropped.

LM Studio — all three builds load cleanly via the MLX runtime (lms load) and were tested through the OpenAI-compatible /v1/chat/completions API at localhost:1234, producing correct, properly-terminated answers (no runaway generation). The upstream chat_template.jinja contains {% generation %}/ {% endgeneration %} markers (a transformers-only training-loss-masking extension); these are no-ops at inference time and were stripped from this repo's template so LM Studio's Jinja engine (which doesn't recognize them) doesn't error. The rendered prompt is otherwise byte-identical to the upstream template.

Usage (mlx-lm)

pip install -U mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("sahilchachra/Carbon-X-MXFP8")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "What is the capital of France?"}],
    add_generation_prompt=True, tokenize=False,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=200))

For DNA-specific tasks, use the base model's skills/carbon-dna tooling or a trust_remote_code=True transformers/vLLM load — see the original model card for the ask/compare workflow.

Run in LM Studio

Loads and runs out of the box — the qwen3 architecture is recognized natively by LM Studio's bundled MLX runtime. Works through both the UI and the OpenAI-compatible /v1/chat/completions API. This is a reasoning model: it emits a <think> block before its answer, so give it enough max_tokens (100+) or the final answer can be truncated while still thinking.

DNA input is not supported through LM Studio — see the warning above; LM Studio uses the plain BPE tokenizer and has no path to the custom 6-mer vocabulary.

Notes & limitations

• tie_word_embeddings=false — the ~804M-parameter lm_head is a separate matrix; it is kept at the highest available precision in every build here (8-bit in OptiQ-5bpw, native in MXFP4/MXFP8) since it is the most sensitivity-critical layer for a 157,184-entry vocabulary spanning both text and DNA tokens.
• Config fix: upstream config.json stores RoPE theta under a nested rope_parameters dict (newer transformers config format); mlx-lm 0.31.3 expects a flat rope_theta field. This repo's config.json carries both for compatibility.
• Inherits all capabilities and limitations of the base model. See the original model card.
• Quantized by @sahilchachra with MLX. Apache-2.0.