Llama 3.3 70B Instruct — NVFP4 GGUF

NVFP4 quantization of meta-llama/Llama-3.3-70B-Instruct, a 70B parameter multilingual instruction-tuned language model from Meta.

About the Model

Llama 3.3 70B Instruct is an auto-regressive transformer optimized for multilingual dialogue use cases. It outperforms many open-source and closed chat models on common industry benchmarks.

• 70B parameters — dense transformer architecture
• 128K context window with Grouped-Query Attention (GQA)
• 15T+ tokens pretrained on publicly available data
• Multilingual — English, German, French, Italian, Portuguese, Hindi, Spanish, Thai
• Tool calling support with native function calling
• Text-only — no vision/multimodal capabilities

Quantization

This GGUF was quantized from Unsloth's official F16 GGUF using llama.cpp (build 537). The F16 weights were dequantized to F32 during quantization, then quantized to NVFP4 format.

NVFP4 (NVIDIA FP4) uses 4-bit floating point quantization optimized for NVIDIA Blackwell (B-series) GPUs.

Files

Note: This is a text-only model. No mmproj/vision files needed.

Usage

llama.cpp

# Server mode with OpenAI-compatible API
llama-server \
  -m llama-3.3-70b-instruct-nvfp4.gguf \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8080

# Direct inference
llama-cli \
  -m llama-3.3-70b-instruct-nvfp4.gguf \
  -ngl 99 \
  -p "Explain the theory of relativity in simple terms"

LM Studio

1. Download the GGUF file from this repository
2. Load the GGUF file in LM Studio
3. Set GPU offload layers to maximum

Architecture

• Parameters: 70B (dense transformer)
• Layers: 80
• Hidden dimension: 8192
• Attention heads: 64 (8 KV heads for GQA)
• Context: 128K
• Vocabulary: 128K tokens

Hardware Requirements

• Minimum: 40 GB VRAM for partial offload
• Recommended: 48+ GB VRAM for full GPU offload
• Disk: ~37.2 GB

Benchmarks

License

Llama 3.3 Community License — see LICENSE.

Note: If your product/service has >700M monthly active users, you need a separate license from Meta.