GGUF file of Olala-v1.0-summarise, the document summarization model that OVHai LLM fine-tuned from Olala-v1.0-mini-7A1B and publishes under the Apache 2.0 license. In November 2026 its authors publish it as weights in safetensors. This repository holds these weights in one quantized GGUF file, written for Albireo, the Windows program of Cydral Technology that runs language models on the computer where it is installed.
[!WARNING] The file needs a program that reads the
olalaarchitecture. Albireo reads it from version 1.0.26. The card of the GGUF files of Olala 1.0 mini states that upstream llama.cpp, Ollama, LM Studio and the other programs under "Use this model" do not support this architecture yet. The llama.cpp fork of the authors reads their own files; this file was not run with it.
| File | Size | Tensors |
|---|---|---|
Olala-v1.0-summarise-q6_k.gguf |
6.31 GB | 989, of which 3.29 GB in 6 bits (Q6_K), 2.89 GB in 8 bits (Q8_0), 0.08 GB in single precision and 0.05 GB in BF16 |
Olala-v1.0-summarise has the layers and the shapes of Olala 1.0 mini: 36 layers, 29 of them Mamba-3 layers and 7 attention layers, each with a mixture of 256 routed experts. The tensors of the file have the names and the order of the 6-bit file that the authors publish for Olala 1.0 mini, and its formats, five tensors apart:
| Tensors | Format |
|---|---|
Up projections of the routed experts (ffn_up_exps) |
Q8_0 |
Up projection of the latent space of each layer (ffn_latent_up) |
BF16 |
Down projection of the latent space of layers 7, 8, 9, 10 and 14 (ffn_latent_down) |
BF16 |
| Other weight matrices, the token embeddings and the output projection included | Q6_K |
| Biases, norms, scales and the parameters of the Mamba-3 layers | single precision |
The five down projections kept in BF16 hold values near 2e-5 in the weights. The scales of the 6-bit format are numbers in half precision, which round such values to zero: the 6-bit file of the authors for Olala 1.0 mini reads these five tensors as zeros. The converter keeps in BF16 a tensor its format cannot hold, so that every tensor of this file follows the weights. The answers do not depend on it: the other tensors of the routed experts of these five layers are as faint, and their part in the output of their block stays under 2e-18 of that of the shared expert, where it is about 1.7 times that part in layer 6.
The file was quantized without an importance matrix, where the documentation of the authors' fork describes one, computed on English text, for their own files of Olala 1.0 mini.
Albireo is a Windows 11 program built on dlib and Dlib-Transformer-extensions: an inference engine for quantized GGUF models, a Windows service that serves it on a local HTTPS address with an interface compatible with the OpenAI one, a web application served by that service, and an installer. The conversations, the documents and the models stay on the computer, and nothing asks for an account. A document given to this model for a summary is read by the engine of that computer and by nothing else.
Albireo computes the layers of this architecture with code of its own, written after the module
modeling_olala.py that the authors publish with the weights. It writes the prompts of the model as the chat
template of its authors does, and reads its answers channel by channel: the reasoning of the analysis channel,
which the web application shows while the model writes it and folds when the summary starts, then the summary of
the final channel. It saves the state of the Mamba-3 layers of a conversation between two requests: a second
question about a document reads the answer before it and that question, and takes the document from the saved
state.
Olala-v1.0-summarise-q6_k.gguf from this repository.A file attached to a question comes before the instruction in the prompt the web application writes, between
<document> tags. The layout the authors give for this model is the instruction first: pasting the text of the
document after the instruction follows it.
Another program asks for a summary through the local interface, here from PowerShell with a report saved as
report.txt:
$document = Get-Content -Raw -Encoding UTF8 report.txt
$body = @{
model = "albireo-general"
reasoning = $true
max_tokens = 4096
messages = @(@{ role = "user"; content = "Summarize the following report in five bullet points.`n`n" + $document })
} | ConvertTo-Json -Depth 6
$answer = Invoke-RestMethod -Method Post -Uri https://localhost:8443/v1/chat/completions `
-ContentType "application/json; charset=utf-8" -Body ([Text.Encoding]::UTF8.GetBytes($body))
$answer.choices[0].message.content
The summary is in content and the reasoning in reasoning_content.
The card of the authors gives the rules below, which their measures follow:
disabled, low, medium or high, and medium
is the setting of their measures. In Albireo the deep reasoning asks for medium, and a request names another
effort with reasoning_effort.slm_safetensors_to_gguf.py,
a standalone Python program of the examples of Dlib-Transformer-extensions (Python 3 and numpy), retrieved the
weights, wrote the file and compared it with the weights:
python3 slm_safetensors_to_gguf.py fetch OVHaiLLM/Olala-v1.0-summarise Olala-v1.0-summarise
python3 slm_safetensors_to_gguf.py convert Olala-v1.0-summarise --type q6_k --out Olala-v1.0-summarise-GGUF \
--license apache-2.0 --author "OVHai LLM" --source-url https://huggingface.co/OVHaiLLM/Olala-v1.0-summarise
python3 slm_safetensors_to_gguf.py verify Olala-v1.0-summarise Olala-v1.0-summarise-GGUF/Olala-v1.0-summarise-q6_k.gguf
convert measures each tensor it writes against its values and names those it keeps in BF16. verify then reads
each tensor of the file back as the values an engine computes with, measures its relative distance to the weights
it comes from, and accepts a file whose tensors all stay within the bound of their format:
F32 614 tensors, largest relative distance 0.0000 (blk.0.exp_probs_b), at most 0.0000
Q8_0 36 tensors, largest relative distance 0.0085 (blk.14.ffn_up_exps.weight), at most 0.0300
Q6_K 298 tensors, largest relative distance 0.0492 (blk.10.ffn_down_exps.weight), at most 0.0800
BF16 41 tensors, largest relative distance 0.0000 (blk.0.ffn_latent_up.weight), at most 0.0100
every tensor follows the checkpoint
The SHA-256 of the file is listed on its page of this repository.
The weights are the work of OVH LLM SAS (trading as OVHai LLM), published under the
Apache License 2.0. The files LICENSE and NOTICE of their
repository come with this file, unchanged. The notice names the tokenizer of the Qwen project (Alibaba Cloud),
under the same license. Changes made here: the weights were written in a GGUF container and quantized as the
table above lists. Nothing else was changed, retrained or merged.
The technical report of the authors describes the architecture: https://huggingface.co/OVHaiLLM/Olala-v1.0-mini-7A1B/blob/main/Olala-v1.0-mini.pdf
Cydral Technology is an open source initiative, independent of OVHai LLM and of OVHcloud. The sources of the converter and of Albireo are published under the Boost Software License 1.0.
Cydral Technology, France: https://www.cydral-technology.eu, contact@cydral-technology.eu