Nemotron 3.5 Bolt + Nemotron 3.5 Parse: a two-stage document reader

Packaged and tested by Offline Base: private AI that runs on hardware you own.

An all-NVIDIA way to read documents and answer questions about them:

1. Nemotron 3.5 Parse (0.63B) reads a page image and writes it out as Markdown, with tables as HTML, using NVIDIA's own postprocessing.
2. Nemotron 3.5 Bolt E4B (8B total, 4B effective) answers questions from that Markdown.

Parse is a separate encoder-decoder model (RADIO vision encoder with an mBART-style decoder), not a projector, so it cannot be loaded into Bolt's llama.cpp process. The two run as two stages. If you want one model and one process, see the sister repo OfflineBase/Nemotron-3.5-Bolt-E4B-Vision-GGUF, which runs Bolt with Gemma 4 E4B's vision projector (unofficial: NVIDIA lists Bolt's input as text) and measures it on a Jetson Orin Nano.

Community packaging. Not made, reviewed or endorsed by NVIDIA. This repo holds code and documentation; both models are downloaded from NVIDIA's own repositories at pinned revisions.

Models used

Quick start

pip install -r requirements.txt                        # Python 3.12
llama-server -m Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf -c 8192 -ngl 99 -fa on --jinja --reasoning off   # stage 2

python3 pipeline.py page.png                           # print the page as Markdown
python3 pipeline.py page.png -q "What is the security deposit?"   # parse, then ask Bolt

pipeline.py picks CUDA, then Apple silicon, then CPU for Parse (--device overrides). On a Jetson Orin Nano, serve Bolt with run/serve-bolt-jetson-orin-nano.sh.

Use cases

• Structured extraction. Parse keeps tables as tables (HTML in the Markdown), so statements, invoices and inventories can go straight into a spreadsheet or database, and Bolt answers questions over the result.
• Building a searchable archive. Batch-convert a backlog of scanned pages on a workstation GPU into clean, layout-aware Markdown chunks for retrieval, then serve questions with Bolt on a small edge box.
• An all-NVIDIA stack. Both stages are NVIDIA Nemotron models under the same OpenMDW license, which matters for some partners and procurement rules.
• Layout-heavy documents. Forms, multi-part pages and pages with figures, where reading order and structure matter as much as the words.

Not a fit (yet): an 8 GB edge board running everything in one process, since Parse needs PyTorch and has not been tested on a Jetson. For that, use the single-model sister repo.

How Parse read our test pages

The same 50 made-up pages used for the sister repo (10 document types × clean, scan, heavy scan, phone photo, low-res; 305 key facts). Parse ran on a Mac M4 Max (PyTorch, Apple GPU, bf16) with NVIDIA's default prompt and postprocessing.

By condition, Parse found 100% of key facts on clean and low-res pages, 98.4% on scans and phone photos, and 88.5% on heavy scans. Its outputs are structured Markdown with real tables, which suits retrieval and citation. Reviewers who checked every miss against the page image found three real misreads, including a 4-digit gate code written as Japanese characters, and one dropped address line on a heavy-scan lease. Every output and the audit are in eval/.

What has and has not been tested

• Tested: Parse on 50 pages (Mac, NVIDIA's prompt and postprocessing, exactly the code path in pipeline.py); Bolt on Jetson Orin Nano boards as a chat and question-answering model.
• Not yet tested: the two stages run end to end through pipeline.py; Parse on a Jetson (it needs PyTorch for JetPack 7, which we have not verified); Parse's memory use beside Bolt on an 8 GB board.
• Memory: on Apple silicon, Parse's GPU memory peaked near 12 GB in our run. Plan for that on shared machines.

License

Both models are NVIDIA's, under the OpenMDW License Agreement, version 1.1; Parse's tokenizer is under MIT (see NVIDIA's repo). The code and documentation here are Apache 2.0. See NOTICE.md.