About GGUF
GGUF is the file format of llama.cpp, designed for portability and efficient inference on the edge. These are quantizations of Villanova2-15B-A3B-VL.
Quick usage
llama-cli -hf VillanovaAI/Villanova2-15B-A3B-VL-GGUF:Q8_0
Replace Q8_0 with any quantization in this repository. For an OpenAI-compatible server, use llama-server with the same arguments plus --port 8080.
Performance
Measured with llama-bench: batch 1, 512-token prompt (prompt processing, PP), 128 generated tokens (token generation, TG), 3 repetitions.
Batch 1 is one conversation at a time, the typical GGUF use case.
Snapdragon 8 Elite (Hexagon NPU)
LPDDR5X at 84.8 GB/s.
| Model | Quant | PP t/s | TG t/s |
|---|---|---|---|
| Villanova2-2B | Q4_0 |
913 | 27.4 |
| Villanova2-2B | Q8_0 |
883 | 17.6 |
| Villanova2-15B-A3B | Q4_0 |
378 | 15.3 |
2B on a Galaxy S25 (12 GB), 15B on a OnePlus 13 (16 GB). Same SoC in both.
0.60 J/token for the 15B on the NPU, from 9.24 W device-side at 15.3 t/s. Whole-device power, not NPU-only.
Weight footprint: 1.45 GB (2B Q4_0), 8.85 GB (15B-A3B Q4_0).
RTX 4080 SUPER (Vulkan)
GDDR6X at 736 GB/s.
Villanova2-15B-A3B Q4_K_M, Flash Attention on. The left column is the KV cache already filled when timing starts, i.e. a conversation already in progress.
| KV prefill | PP t/s | TG t/s |
|---|---|---|
| 0 | 5647 | 147.4 |
| 4096 | 5672 | 146.7 |
| 16384 | 4912 | 126.4 |
Images
llama-cli handles multimodal input natively. With -hf the projector is downloaded automatically; with local files, pass it explicitly:
llama-cli -m Villanova2-15B-A3B-VL-Q8_0.gguf --mmproj mmproj-Villanova2-15B-A3B-VL-F16.gguf \
--image invoice.png -p "Extract number, date and total from this invoice as JSON."
Citation
@techreport{villanova2_2026,
title = {Villanova2 --- Technical Report},
author = {{Villanova.AI}},
institution = {Villanova.AI},
year = {2026},
note = {Coming soon}
}