中文Sign up

meshllm/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL-layers

🤗 Hugging Face sourceimage-text-to-textapache-2.04B activated19 GBGGUF✓ 37 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo meshllm/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL-layers ./model-folder
Needs a seeder →

gemma-4-26B-A4B-it-qat-UD-Q4_K_XL

Distributed GGUF inference package for Mesh LLM

GGUF layer package for running gemma-4-26B-A4B-it-qat-UD-Q4_K_XL across a local Mesh LLM cluster.

This package is derived from unsloth/gemma-4-26B-A4B-it-qat-GGUF and keeps the original GGUF distribution split into per-layer artifacts for distributed inference.

Highlights

Run locally Pool multiple machines OpenAI-compatible Package variant
Private inference on your hardware Split layers across peers Serve /v1/chat/completions locally UD-Q4_K_XL layer package

Model Overview

Property Value
Source model unsloth/gemma-4-26B-A4B-it-qat-GGUF
Model id unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL
Family Gemma
Parameter scale 26B-A4B
Quantization UD-Q4_K_XL
Layer count 30
Activation width not recorded
Package size 0 B
Source file gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf
Package repo meshllm/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL-layers
License apache-2.0 from unsloth/gemma-4-26B-A4B-it-qat-GGUF

Recommended Use

  • Local and private inference with Mesh LLM.
  • Multi-machine serving when the full GGUF is too large for one host.
  • OpenAI-compatible chat/completions workflows through Mesh LLM's local API.

For upstream architecture details, chat template guidance, sampling recommendations, license terms, and benchmark notes, see the source model card: unsloth/gemma-4-26B-A4B-it-qat-GGUF.

Quickstart

# Run this on each machine that should contribute memory/compute.
mesh-llm serve --model "meshllm/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL-layers" --split

# Check the mesh and discover the OpenAI-compatible model name.
curl -s http://localhost:3131/api/status
curl -s http://localhost:3131/v1/models

# Send an OpenAI-compatible chat request.
curl -s http://localhost:3131/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL",
    "messages": [{"role": "user", "content": "Write a tiny hello-world function in Rust."}],
    "max_tokens": 128
  }'

Package Variant

Property Value
Format gguf
Canonical source ref unsloth/gemma-4-26B-A4B-it-qat-GGUF@7b92b5b28818151e8669af2e45e88d6086f490dd/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf
Source revision 7b92b5b28818151e8669af2e45e88d6086f490dd
Source SHA-256 a7c5bc715f5ff8e99a3e8901ce7d2b42b402c669bf24f7c5250747633d0f5891
Skippy ABI not recorded
Package manifest SHA-256 a6a10ae900f883a3e2b7a7ccaeebf6a142984c8f18404583599f22dd96dcc78a

What Is Included

Artifact Path Contents SHA-256
Manifest model-package.json Package schema, source identity, checksums a6a10ae900f883a3e2b7a7ccaeebf6a142984c8f18404583599f22dd96dcc78a

Validation

Generated by the Mesh LLM HF Jobs splitter from mesh-llm ref f932c4d1dc12b3e3a670d5f470cedd5cdcc5db39. Each artifact is checksummed as it is written, uploaded to this repository, and removed from the job workspace before the next artifact is produced.

skippy-model-package write-package "/hf-cache/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf" --out-dir "/tmp/meshllm-layer-job-meshllm_gemma-4-26B-A4B-it-qat-UD-Q4_K_XL-layers-1/package"

Links