Sakura · GGUF quantization of xiaomi-research/Xiaomi-OCR-0

Independent community quantization. Not an official release of the upstream author.

Made with an importance matrix (imatrix). The weights were quantized with importance weights computed from a fixed calibration text; the exact method and the hashes are in the technical details of section 5.

1. Model Overview

Architecture: qwen35 · source formats: gguf.

Original: 1.5 GB (BF16 GGUF, the source of these quantizations).

2. Original Model / Attribution

• Original model: xiaomi-research/Xiaomi-OCR-0
• Submitted repository: prithivMLmods/Xiaomi-OCR-0-GGUF
• Declared base models: xiaomi-research/Xiaomi-OCR-0 (relation: quantized)
• Quantized from: prithivMLmods/Xiaomi-OCR-0-GGUF (file Xiaomi-OCR-0.BF16.gguf).
• Quantized by webmp3. Upstream authors retain their license and attribution.

3. Files

Xiaomi-OCR-0.mmproj-bf16.gguf is the vision projector of the original GGUF repository prithivMLmods/Xiaomi-OCR-0-GGUF, copied unchanged. Use it together with any of the files above to give the model images. Image input has not been tested for this release.

4. Download and Usage

llama-cli -m <file>.gguf -p "Hello" -n 32

# with images
llama-mtmd-cli -m <file>.gguf --mmproj Xiaomi-OCR-0.mmproj-bf16.gguf --image <picture>.jpg -p "Describe this image."

Needs a llama.cpp build that supports this architecture. If a quantization is split into numbered parts, keep them together and load the first one.

5. Verification

• GGUF structure valid: True
• Real inference test (every file loaded by llama.cpp and used to generate text once): yes
• No coding benchmark, perplexity or quality score is claimed unless a measured value is listed here.

Measured deviation from the BF16 source

Measured with the stock llama-perplexity on two fixed public-domain texts that were not used for calibration (English: Pride and Prejudice; Chinese: Hongloumeng; 64 windows of 512 tokens). The BF16 source itself has a perplexity of 166.304 on this text. Q2_K to Q5_K_M and Q6_K to Q8_0 come from two runs on different machines: the BF16 source measured 166.304 in the first run and 166.250 in the second, and each ratio is relative to the reference of its own run. Q6_K was made with its own importance matrix from the second run (SHA-256 3671c48605c0a88abcbba686ea6a54fa6edc80677677ddc0f5f248c0de5052a1, same calibration text); the matrix named in the technical details belongs to Q2_K to Q5_K_M. Q8_0 does not use importance weights.

A ratio close to 1, a small KL divergence and a high share of identical top tokens mean the quantized model predicts like the source. These numbers describe agreement with the source on this text only; they are not task benchmarks and say nothing about other languages or tasks.

Technical details: exact versions, hashes, how these were made

Exact versions

• Original model commit: 7bbfa072e0f3d70ac722e26fbccef5b9f64b7121 (model card at that commit)
• Vision projector: commit 7bbfa072e0f3d70ac722e26fbccef5b9f64b7121 of prithivMLmods/Xiaomi-OCR-0-GGUF, SHA-256 634d6bc95781ebf4a3f06570a59859fa5082d488db540fd474bc6b4061aec90e
• Quantized from: commit 7bbfa072e0f3d70ac722e26fbccef5b9f64b7121, SHA-256 d796fa2cd211ba39a40f37691fc0046aa49f59c7968ec844eb9e81f2ae476a52 (provenance verified: True)

File hashes

How these were made

This route performs existing-GGUF requantization from the pinned BF16 source GGUF named above. This job did not convert the upstream BF16/safetensors checkpoint.

Quantized with the stock llama.cpp tools and an importance matrix (imatrix): the matrix was computed once with llama-imatrix from the fixed calibration text hfqa-original-calibration-v2 (6144 bytes of English, German and Chinese text, code and data formats, public domain; 11 chunks of 128 tokens) and applied with llama-quantize --imatrix. Matrix SHA-256: 83ea7a10917a9ccdaa14e7c5d6e35fdaf0f437dba59feffd8f3fa1bf3240bd12. Calibration text SHA-256: bc7216ea05c8bd05d6b9e94d813ec9eb651a2375288c08c06135a3f874d886f2. The calibration text is short; no quality improvement is claimed without a benchmark.

The quant label names the llama.cpp preset. Stock llama.cpp keeps some tensors (for example normalization weights) in higher precision, so not every tensor is stored in the named type; the table shows what each file really contains.

Test builds

6. License Information

Source license: apache-2.0. apache-2.0 allows redistribution with attribution

中文说明 · 樱花 Sakura（点击展开）

1. 概述

独立的社区量化版本，并非上游作者的官方发布。

使用重要性矩阵（imatrix）量化。 量化时使用了由固定校准文本计算出的重要性权重；具体方法和哈希见第 5 节的技术细节。

架构：qwen35 · 源格式：gguf。

原始模型：1.5 GB（BF16 GGUF，这些量化版本的来源）。

2. 原始模型与署名

• 原始模型：xiaomi-research/Xiaomi-OCR-0
• 提交的仓库：prithivMLmods/Xiaomi-OCR-0-GGUF
• 声明的基础模型：xiaomi-research/Xiaomi-OCR-0（关系：quantized）
• 量化来源：prithivMLmods/Xiaomi-OCR-0-GGUF（文件 Xiaomi-OCR-0.BF16.gguf）。
• 量化者：webmp3。上游作者保留其许可证与署名。

3. 文件

Xiaomi-OCR-0.mmproj-bf16.gguf 是原始 GGUF 仓库 prithivMLmods/Xiaomi-OCR-0-GGUF 的视觉投影文件，原样复制。与上方任一文件配合使用即可输入图像。本版本未测试图像输入。

4. 下载与使用

llama-cli -m <file>.gguf -p "Hello" -n 32

# with images
llama-mtmd-cli -m <file>.gguf --mmproj Xiaomi-OCR-0.mmproj-bf16.gguf --image <picture>.jpg -p "Describe this image."

需要支持该架构的 llama.cpp 版本。如果某个量化被拆分为带编号的多个部分，请将它们放在一起并加载第一个。

5. 验证

• GGUF 结构有效：True
• 真实推理测试（每个文件都由 llama.cpp 加载并生成一次文本）：yes
• 除非此处列出了实测数值，否则不声称任何编程基准、困惑度或质量分数。

与 BF16 源的实测偏差

使用 llama.cpp 标准工具 llama-perplexity 在两段固定的公有领域文本上测量，文本未用于校准（英文：《傲慢与偏见》；中文：《红楼梦》；64 个窗口，每个 512 个词元）。BF16 源本身在该文本上的困惑度为 166.304。Q2_K 至 Q5_K_M 与 Q6_K、Q8_0 来自两次在不同机器上的运行：BF16 源在第一次运行中为 166.304，第二次为 166.250，各比值均相对于各自运行的参考值。Q6_K 使用第二次运行中单独计算的重要性矩阵（SHA-256 3671c48605c0a88abcbba686ea6a54fa6edc80677677ddc0f5f248c0de5052a1，校准文本相同）；技术细节中给出的矩阵属于 Q2_K 至 Q5_K_M。Q8_0 不使用重要性权重。

比值接近 1、KL 散度小、首选词元相同的比例高，说明量化模型的预测更接近源模型。这些数字仅说明在该文本上与源模型的一致程度，不是任务基准测试，也不代表其他语言或任务。

技术细节：确切版本、哈希、制作方式

确切版本

• 原始模型提交：7bbfa072e0f3d70ac722e26fbccef5b9f64b7121（该提交的模型卡）
• 视觉投影文件：prithivMLmods/Xiaomi-OCR-0-GGUF 的提交 7bbfa072e0f3d70ac722e26fbccef5b9f64b7121，SHA-256 634d6bc95781ebf4a3f06570a59859fa5082d488db540fd474bc6b4061aec90e
• 量化来源：提交 7bbfa072e0f3d70ac722e26fbccef5b9f64b7121，SHA-256 d796fa2cd211ba39a40f37691fc0046aa49f59c7968ec844eb9e81f2ae476a52（出处已验证：True）

文件哈希

制作方式

该路线从上方标明的固定版本 BF16 源 GGUF 进行现有 GGUF 的再量化。本任务没有转换上游的 BF16/safetensors 检查点。

使用 llama.cpp 标准工具和重要性矩阵（imatrix）量化：矩阵由 llama-imatrix 根据固定的校准文本 hfqa-original-calibration-v2（6144 字节，含英文、德文、中文、代码和数据格式，属公有领域；11 个块，每块 128 个词元）计算一次，再由 llama-quantize --imatrix 应用。矩阵 SHA-256：83ea7a10917a9ccdaa14e7c5d6e35fdaf0f437dba59feffd8f3fa1bf3240bd12。校准文本 SHA-256：bc7216ea05c8bd05d6b9e94d813ec9eb651a2375288c08c06135a3f874d886f2。校准文本较短，没有基准测试，因此不声称质量提升。

量化标签指 llama.cpp 的预设名称。标准 llama.cpp 会把部分张量（例如归一化权重）保留为更高精度，因此并非每个张量都以所标注的类型存储；下表列出每个文件实际包含的内容。

测试版本

6. 许可证信息

源许可证：apache-2.0。apache-2.0 allows redistribution with attribution

完整独立的中文版：README_zh.md