中文Sign up

second-state/SmolVLM2-500M-Video-Instruct-GGUF

🤗 Hugging Face sourceimage-text-to-textapache-2.04.7 GBGGUF✓ 14 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo second-state/SmolVLM2-500M-Video-Instruct-GGUF ./model-folder
Needs a seeder →

SmolVLM2-500M-Video-Instruct-GGUF

Original Model

HuggingFaceTB/SmolVLM2-500M-Video-Instruct

Run with LlamaEdge

  • LlamaEdge version: v0.21.0 and above

  • Prompt template

    • Prompt type: smol-vision

    • Prompt string

      <|im_start|>
User: {user_message_1}<image>
Assistant: {assistant_message_1}
User: {user_message_2}<image>
Assistant:

  • Context size: 2048

  • Run as LlamaEdge service

    wasmedge --dir .:. --nn-preload default:GGML:AUTO:SmolVLM2-500M-Video-Instruct-Q5_K_M.gguf \
  llama-api-server.wasm \
  --prompt-template smol-vision \
  --llava-mmproj SmolVLM2-500M-Video-Instruct-mmproj-f16.gguf \
  --model-name SmolVLM2-500M-Video-Instruct \
  --ctx-size 2048

Quantized GGUF Models

Name Quant method Bits Size Use case
SmolVLM2-500M-Video-Instruct-Q2_K.gguf Q2_K 2 245 MB smallest, significant quality loss - not recommended for most purposes
SmolVLM2-500M-Video-Instruct-Q3_K_L.gguf Q3_K_L 3 273 MB small, substantial quality loss
SmolVLM2-500M-Video-Instruct-Q3_K_M.gguf Q3_K_M 3 261 MB very small, high quality loss
SmolVLM2-500M-Video-Instruct-Q3_K_S.gguf Q3_K_S 3 245 MB very small, high quality loss
SmolVLM2-500M-Video-Instruct-Q4_0.gguf Q4_0 4 256 MB legacy; small, very high quality loss - prefer using Q3_K_M
SmolVLM2-500M-Video-Instruct-Q4_K_M.gguf Q4_K_M 4 303 MB medium, balanced quality - recommended
SmolVLM2-500M-Video-Instruct-Q4_K_S.gguf Q4_K_S 4 293 MB small, greater quality loss
SmolVLM2-500M-Video-Instruct-Q5_0.gguf Q5_0 5 301 MB legacy; medium, balanced quality - prefer using Q4_K_M
SmolVLM2-500M-Video-Instruct-Q5_K_M.gguf Q5_K_M 5 326 MB large, very low quality loss - recommended
SmolVLM2-500M-Video-Instruct-Q5_K_S.gguf Q5_K_S 5 319 MB large, low quality loss - recommended
SmolVLM2-500M-Video-Instruct-Q6_K.gguf Q6_K 6 418 MB very large, extremely low quality loss
SmolVLM2-500M-Video-Instruct-Q8_0.gguf Q8_0 8 437 MB very large, extremely low quality loss - not recommended
SmolVLM2-500M-Video-Instruct-f16.gguf f16 16 820 MB
SmolVLM2-500M-Video-Instruct-mmproj-f16.gguf f16 16 199 MB

Quantized with llama.cpp b5501