Nemotron 3.5 Bolt E4B + Gemma 4 E4B vision projector (GGUF)

Packaged and tested by Offline Base: private AI that runs on hardware you own.

In short: this is NVIDIA's Nemotron 3.5 Bolt E4B with Google's Gemma 4 vision part added, so it can read pictures and documents, not just text. Both files are the original releases, unchanged. We tested how well the pair reads documents on a small Jetson Orin Nano 8 GB board, against five other document readers.

Community packaging. Not made, reviewed or endorsed by NVIDIA or Google.

How this differs from NVIDIA's Bolt

NVIDIA published Bolt in two forms:

• Full precision (NVIDIA-Nemotron-3.5-Bolt-E4B-BF16, 15.9 GB). This still contains Gemma 4's vision and audio parts.
• A GGUF for llama.cpp (NVIDIA-Nemotron-3.5-Bolt-E4B-4GB, 6.18 GB). This has the language model only. llama.cpp keeps vision and audio in a separate file, and NVIDIA did not publish one.

NVIDIA's model cards list Bolt's input as text, because text is what NVIDIA trained and supports it for. This repo takes NVIDIA's GGUF unchanged and adds the missing vision file, so llama.cpp users get the vision and audio that the full release still contains.

Why it works. Bolt is NVIDIA's further training of Google's Gemma 4 E4B, and it keeps Gemma 4's design. So Gemma 4's own vision file plugs straight in. NVIDIA's technical report (section 7) also says Bolt keeps Gemma's ability to handle images and audio.

We changed nothing. No retraining and no merging. We put the two files side by side and measured them. On 49 of our 50 test pages, Bolt's output was identical, character for character, to stock Gemma 4 E4B with the same vision file. So for reading documents, Bolt matches Gemma 4.

Harder image questions are slightly weaker. NVIDIA's report finds "slight but consistent degradation in vision capabilities", which it puts down to training Bolt on text only. For example, Bolt scores 42.0 on MMMU-Pro Vision against Gemma 4 E4B's 47.6. Reading text from a page held up in our tests. Reasoning about images, such as charts, is where to expect small losses.

Demos

Try it live at findingnemotron.com, running on our Jetson Orin Nano boards.

Reading a receipt. Mantis, a Jetson Orin Nano running this repo's two files, lists what was bought and checks the math (27.87 + 2.30 = 30.17). It writes 13.7 tokens a second, and the answer starts after 1.5 seconds.

With and without a draft model. The same Bolt file on two boards, asked the same question. Marlin (left) also runs a DFlash draft model, which guesses ahead so Bolt can accept several tokens at once. It writes 23.5 tokens a second, and 24% of its guesses were accepted. Mantis (right) runs without one, at about 14. The draft model is not part of this repo (how to add it).

Quick start

Fastest start

Install a recent llama.cpp (brew install llama.cpp on a Mac, or a release build). Then run:

llama-server -hf OfflineBase/Nemotron-3.5-Bolt-E4B-Vision-GGUF --jinja --reasoning off --port 8080

This downloads both files (6.74 GB) and starts a server. Open http://localhost:8080 for a chat page, attach an image and ask about it.

• You need a computer with at least 8 GB of memory. A GPU makes it much faster.
• We tested this exact command with Homebrew's llama.cpp (build 11669) on an Apple silicon Mac. It read our sample receipt correctly at 31 tokens a second.
• Keep --port 8080. Recent llama.cpp builds use a different port by default.

Run on a Jetson Orin Nano 8 GB

Everything in this section was tested with llama.cpp release b11574 on a Jetson Orin Nano 8 GB (JetPack 7, L4T R39.2.1, CUDA 13, MAXN_SUPER power mode). Download both files first, then run:

llama-server -m Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf --mmproj mmproj-gemma-4-E4B-it-Q8_0.gguf \
  -ngl 999 -c 8192 -np 1 -fa on --jinja --reasoning off -b 512 -ub 512 --cache-ram 0 \
  --load-mode mmap --lazy-mode on -ot per_layer_token_embd=CPU

The same command is in run/serve-jetson-orin-nano.sh. The memory flags come from NVIDIA's Jetson AI Lab recipe. About 3 GB of the Bolt file (2.99 GB) is a per-layer lookup table that can stay in regular memory instead of GPU memory (-ot per_layer_token_embd=CPU).

About the "4GB" in NVIDIA's repo name: NVIDIA's report calls it the model's "target memory footprint of approximately 4 GB". It is not the file size, which is 6.18 GB.

How it went on the board:

• The server used at most 6.4 GB of the board's 7.5 GB of memory, on both the 50-page and the 20-page test.
• A typical page took 13 seconds.
• Giving each page more image detail (1,120 tokens) made it slower (16.2 seconds a page) and no more accurate.

On other computers, the short version works too:

llama-server -m Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf --mmproj mmproj-gemma-4-E4B-it-Q8_0.gguf \
  -c 8192 -ngl 99 -fa on --jinja --reasoning off

Send a page from code

run/ocr_page.py sends an image to the server and prints what it reads. It uses only Python's standard library.

python3 run/ocr_page.py page.png

Every result in this card used this prompt, at temperature 0, with "chat_template_kwargs": {"enable_thinking": false}:

Transcribe every word and number in this document exactly as written, in reading order. For tables, write one row per line with cells separated by ' | '. Output only the transcription.

Faster text with DFlash (optional)

A DFlash draft model makes Bolt write text faster. NVIDIA publishes an official one (nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash, BF16, 0.74 GB). On our 8 GB Jetson we use unsloth's smaller copy instead, because it leaves more memory free: dflash-NVIDIA-Nemotron-3.5-Bolt-E4B-Q8_0.gguf (0.40 GB, from unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF).

To use it, take the Jetson command above and replace --mmproj mmproj-gemma-4-E4B-it-Q8_0.gguf with:

-md dflash-NVIDIA-Nemotron-3.5-Bolt-E4B-Q8_0.gguf --spec-type draft-dflash --spec-draft-n-max 7 --spec-draft-ngl 999

On our boards, on a step-by-step answer, Bolt with the draft model wrote 23.5 tokens a second, against about 14 without it.

It is text only for now. With the draft model and the vision file loaded together, llama.cpp fails image requests. The fix is still open (llama.cpp PR #30220). That is why our demo uses one board for pictures and another, with the draft model, for text.

Audio (experimental)

The vision file also includes Gemma 4's audio encoder, so Bolt can listen too. Send a WAV file as an input_audio part instead of an image:

{"type": "input_audio", "input_audio": {"data": "<base64 WAV>", "format": "wav"}}

On the Jetson, with the same server command, Bolt transcribed two short clips word for word, writing numbers as digits ("The rent for unit 22 is $1,450, due on March 3rd. Please call me back at 555-0120."). It also answered questions about them correctly. That is two clips with synthetic voices, not a benchmark, and llama.cpp calls its audio input experimental.

Files

Total download: 6.74 GB. ggml-org also offers a larger BF16 vision file (0.99 GB). We tested only the Q8_0 one.

SHA-256 checksums, identical to the originals:

• Bolt: c8af2b8aea10a690a1d9491db744a9446660173cb38fc489612b556615a69612
• Vision file: 197f49a93027f9843772bd24a6a9e0be2a32a788de5a3def330e9c585d86edd1

Good uses

• Ask questions about your own documents, privately. Photograph a lease, bill, bank statement or school form. Bolt reads it, the text goes into a local search index, and answers point to the right page. Nothing leaves the device.
• One model for chatting and reading on a small board. On an 8 GB Jetson, the vision file adds only 0.56 GB to the Bolt you already run. A separate reading model would not fit beside it.
• Pull numbers out of paperwork. Amounts, dates, account and invoice numbers from receipts and invoices, into a spreadsheet. Check that each table has the right number of rows (see Limitations).
• Digitize records offline. For schools, clinics and nonprofits whose records cannot go to a cloud service.
• Voice notes (experimental). Transcribe a short recording and ask about it, with the same model.
• Read letters aloud. Pair it with a local text-to-speech engine for people who find printed text hard to read.

Have a person check the output for: legal, medical or financial decisions, dense real handwriting, and long multi-column scans (untested).

Results

We made 70 test pages ourselves, with invented names, 555 phone numbers and example.com emails. The page generators, questions, every reader's raw output, per-page scores and our review notes are all in eval/.

What the columns mean:

• Key facts: the share of must-get-right details (names, amounts, dates) read exactly.
• Lines exact: the share of lines copied with no errors at all.
• Invented numbers: numbers in the output that are not on the page. Counted on pages a person can read.
• Table rows intact: table rows that came out whole and in the right order.

Set A: 50 pages. 10 kinds of document, each as a clean page, a scan, a heavy scan, a phone photo and a low-resolution copy. 305 key facts.

Bolt missed 6 of 305 facts. Four were on one receipt that our heavy-scan effect made unreadable even to a person. The other two were dates whose slashes the effect erased (Bolt copied the digits exactly). On every other page it got 100% of the facts.

Set B: 20 harder pages. An 18-row bank statement, a two-column newsletter, 30 lines of fine print, a dense claim form and a handwritten list. Each as a clean page, a fax, a tilted phone photo and a low-resolution copy. 227 key facts, and every page is readable to a person.

Searching the text Bolt read. We asked 121 questions, each with one right page among all 70, and searched with EmbeddingGemma 2 (Q8, running in llama.cpp beside Bolt on the Jetson). Searching Bolt's text worked as well as searching the true text.

Limitations

• Unofficial. NVIDIA lists Bolt's input as text only and does not validate image or audio input. These are our own measurements, not NVIDIA's.
• Slightly weaker at reasoning about images. NVIDIA's own tests show small drops on harder image questions compared with Gemma 4 E4B (see How this differs). Our tests only covered reading text from pages.
• Made-up pages only. Our pages are short, in English and rendered from computer fonts. Real scans, real photos and real handwriting are untested.
• Tilted photos of long tables. On one tilted photo of an 18-row statement, Bolt shifted the amount column down by one row. Every value was right but sat next to the wrong transaction. The first row came out blank and an extra row appeared at the end, so checking the row count catches it. Straightening the photo first will probably help (untested).
• Pages no one can read. On the one unreadable page, Bolt wrote repeated filler ("111 111 ...") instead of saying it could not read it. Watch for repetition and missing content.
• One believable wrong value in 70 pages. Where a scan erased the ring of an "@", an email came out as aikoa@example.com instead of aiko@example.com.
• One run per reader, at temperature 0. On set A, the top readers' ranges overlap, so they are effectively tied.
• Tested on one platform. Every result here is from llama.cpp b11574 on a Jetson Orin Nano. Ollama, other llama.cpp builds and other hardware are untested, apart from the quick Mac check in Fastest start.
• Audio barely tested. Two short synthetic clips. Real voices, noise, accents and long recordings are untested.
• One request at a time. With -np 1, reading pages and chatting share one slot. Run big reading jobs when no one is chatting.

Related work

Checked on 2026-10-24. Others got Bolt reading images first. What this repo adds is the measurement above.

• NVIDIA. The model card lists text input only, but the technical report (section 7) says Bolt "retains the original model's multimodal capabilities" and reports image and audio scores. Bolt beats Gemma 4 E4B on two speech benchmarks (VoiceBench and Audio MultiChallenge), and its vision scores drop slightly. On OmniDocBench it scores Bolt below Gemma 4 E4B (0.42 against 0.21 with reasoning on, 0.47 against 0.24 with it off, lower is better), while on our pages the two read identically. The task, prompt, image detail and scoring all differ from ours, and we have not worked out why.
• Vision files on launch day (2026-10-23). Ollama v0.40.10 ships Bolt E4B as a vision model, with a DFlash draft model. Also Canonical's snap (with a Q8_0 vision file), unsloth, AtomicChat, and a day later bartowski and roman220220.
• Same vision weights as Gemma 4. goinfer and FastFlowLM found Bolt's vision and audio parts are Gemma 4 E4B's own, piece for piece. Our 49 identical pages out of 50 show the same thing in the output.
• Vision scores. keptn reports DocVQA 79.8 ANLS and AI2D 74.6% for the full-precision model, with thinking off, on one 32 GB AMD Radeon AI PRO R9700.
• Jetson. NVIDIA's Jetson AI Lab recipe (text only) introduced the memory flags our Jetson command uses. DJLougen measured DFlash on an Orin Nano 8 GB: 2.3 times faster in everyday use, and 3.4 times on short prompts.
• Audio and Q8_0 vision files. roman220220 reports that a Q8_0 vision file garbles audio. With ggml-org's Q8_0 file, our two short audio checks came out right. We have not tested that claim further.

Not found in this search: Bolt's reading accuracy on a Jetson compared with dedicated document readers, output matching stock Gemma 4, or search over Bolt's text.

License

• Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf: NVIDIA's file, unmodified, under the OpenMDW License Agreement, version 1.1.
• mmproj-gemma-4-E4B-it-Q8_0.gguf: Google's Gemma 4 E4B vision and audio encoders, converted to GGUF by ggml-org, unmodified, under Apache 2.0 (Gemma 4 license).
• Our test code and made-up test pages in eval/ and run/: Apache 2.0.

See NOTICE.md. Your use of each model must follow its own license.

Credits

NVIDIA for Nemotron 3.5 Bolt E4B. Google DeepMind for Gemma 4 and its vision and audio encoders. ggml-org and the llama.cpp contributors for the GGUF conversion and the runtime. Packaging and testing by Offline Base.