Netra-2B — MLX (4bit)
Native MLX conversion of botnick/Netra-2B
(v0.3) — a LoRA-merged fine-tune of
typhoon-ai/typhoon-ocr1.5-2b
(Qwen3-VL-2B architecture) that reads Thai business and accounting documents: page
reading to clean Markdown (tables as one-line HTML, <figure> descriptions in Thai,
<page_number> tags, ☐/☑ checkboxes) and accounting-field extraction to one JSON object
with a fixed 28-key schema. 4-bit-quantized language model, bfloat16 vision tower (~1.8 GB here).
How it was converted
Converted on 2026-11-15 with mlx-vlm 0.7.12, following the convention of the official typhoon-ocr2-MLX / mlx-community Typhoon OCR conversions and of fwgpiyawudk/OvisOCR3-mlx-{8,4}bit: affine quantization, group size 64 — the language model is quantized, the vision tower stays bfloat16:
python -m mlx_vlm convert --hf-path botnick/Netra-2B \
--mlx-path Netra-2B-4bit -q --q-bits 4 --q-group-size 64
One source-level adjustment was needed: this repo was saved with transformers 5.17,
which writes the text config's RoPE settings as a nested rope_parameters object and
labels the vision config qwen3_vl_vision, while mlx-vlm 0.7.12 expects the flat
rope_theta + rope_scaling keys and the qwen3_vl vision label used by the
official mlx-community/typhoon-ocr1.5-2b-8bit
conversion of the base model (same values: rope_theta 5000000, mrope_section
[24, 20, 20], interleaved). The config was normalized before converting; the weights
are untouched. The source repo also ships no preprocessor_config.json
(transformers-5.17-style processor_config.json only), so
preprocessor_config.json / video_preprocessor_config.json are mirrored verbatim
from the official typhoon-ocr1.5-2b-8bit MLX conversion (same Qwen3-VL-2B processor)
to make the repo loadable.
Use
Requires mlx-vlm (tested with 0.7.12):
pip install -U mlx-vlm
from mlx_vlm import load, generate
model, processor = load("fwgpiyawudk/Netra-2B-mlx-4bit")
result = generate(model, processor, prompt=OCR_PROMPT, image="page.png",
max_tokens=4096, temperature=0.0, repetition_penalty=1.05)
print(result.text)
Official settings from the source model card
(the shipped generation_config.json keeps the base's sampling defaults, do_sample=true, temperature=0.7 — pass them explicitly): greedy decoding, repetition_penalty=1.05,
image converted to RGB with the long side at most 1400 px (LANCZOS, downscale only —
the fine-tune was trained and evaluated at ≤ 1400 px; the base typhoon-ocr1.5 uses
1800 px), max_tokens 4096 for page reading and 2048 for the accounting-JSON (KIE)
prompt. The prompts below and the KIE prompt are copied verbatim from the card
("copy these exactly"):
OCR_PROMPT = """
Extract all text from the image.
Instructions:
- Only return the clean Markdown.
- Do not include any explanation or extra text.
- You must include all information on the page.
Formatting Rules:
- Tables: Render tables using <table>...</table> in clean HTML format.
- Equations: Render equations using LaTeX syntax with inline ($...$) and block ($$...$$).
- Images/Charts/Diagrams: Wrap any clearly defined visual areas (e.g. charts, diagrams, pictures) in:
<figure>
Describe the image's main elements (people, objects, text), note any contextual clues (place, event, culture), mention visible text and its meaning, provide deeper analysis when relevant (especially for financial charts, graphs, or documents), comment on style or architecture if relevant, then give a concise overall summary. Describe in Thai.
</figure>
- Page Numbers: Wrap page numbers in <page_number>...</page_number> (e.g., <page_number>14</page_number>).
- Checkboxes: Use ☐ for unchecked and ☑ for checked boxes."""
Loop guard (part of the recommended inference — v0.3's ThaiOCRBench numbers were
measured with it on): on a small share of pages the model repeats one line or phrase,
often until the token limit. degenerate() from the source repo's
loop_guard.py
(stdlib-only) detects it; regenerate once with repetition_penalty=1.25 and keep the
second answer. The card's transformers retry also passes no_repeat_ngram_size=16,
which mlx-vlm does not implement — this conversion therefore mirrors the card's vLLM
guard variant (retry with repetition_penalty=1.25 only):
from loop_guard import degenerate # loop_guard.py from botnick/Netra-2B
text = generate(model, processor, prompt=OCR_PROMPT, image="page.png",
max_tokens=4096, temperature=0.0, repetition_penalty=1.05).text
if degenerate(text):
text = generate(model, processor, prompt=OCR_PROMPT, image="page.png",
max_tokens=4096, temperature=0.0, repetition_penalty=1.25).text
OpenAI-compatible server — same settings per request, image content part first then the prompt text, like the card's chat example:
python -m mlx_vlm.server --model fwgpiyawudk/Netra-2B-mlx-4bit --port 8080
The KIE prompt (fixed 28-key accounting schema), the key-mapping prompt style, the loop-guard measurements and the trust-gate pipeline are documented in the source model card.
Quantization caveat
No official 4-bit numbers exist for Netra-2B. The official Typhoon OCR V2 MLX model cards measured roughly −10 points on Thai pages at 4-bit vs 8-bit — for Thai documents prefer the 8-bit conversion.
Licence
Apache-2.0, the licence of the base model (LICENSE in this repo; the training-data
attribution lives in the source repo).