M4-ai/TinyMistral-248M-v2-Instruct-GGUF

Source: 🤗🏴‍☠️ Rescued by: Alogotronapache-2.01.0 GBGGUF✓ 6 checksumsupdated 1,016d ago

CommentsShare your thoughts and shape the future of sovereign AI.

@M4-ai is unclaimedSign in with the matching Hugging Face account or an organization member account to verify this handle.Claim @M4-ai

Files and sources

SourceAvailable on 🤗

Checksums6 recorded

TorrentFiles match 🤗

Swarm2 seeders reported

Source revision: 114104e33976…
Witnessed: 14 Oct 2026, 19:07 UTC
Swarm checked: 15 Nov 2026, 20:16 UTC
Weight files: 6
Model size: 1.0 GB
Downloads · last month: 60,922
Publisher: Unclaimed

Family

↑ Quantized from Locutusque/TinyMistral-248M-v2-Instruct

Speed on real hardware

LocalMaxxing

Output speed for one request at a time, measured by people running this model on their own machines.

2× RTX 3090llama.cpp · Q4_K_M · 1 run517 tok/s

LD_LIBRARY_PATH=/home/beta1/qwen38-release/tools/llama-b10442-cuda /home/beta1/qwen38-release/tools/llama-b10442-cuda/llama-bench -m /mnt/pf/pf-deep2/TinyMistral-248M-v2-Instruct-GGUF/TinyMistral-248M-v2-Instruct.Q4_K_M.gguf -p 512 -n 128 -ngl 99 -fa 1

View run on LocalMaxxing

Median of 1 community runs on 1 setups, from LocalMaxxing.

Run it yourself: setup guide

Description

GGUF version of Locutusque/TinyMistral-248M-v2-Instruct.

Recommended inference parameters

do_sample: true
temperature: 0.1
top_p: 0.14
top_k: 12
repetition_penalty: 1.1

Recommended prompt template

<|im_start|>user\n{user message}<|im_end|>\n<|im_start|>assistant\n{assistant message}<|endoftext|>