ArcticSpeculator

Build the fastest OSS vllm-based speculative decoding system for your own model, using ArcticTraining and ArcticInference!

Throughput (tokens/s) of gpt-oss-120b on 8xH100 using vLLM below:

For more details about ArcticSpeculator and how to use it:

• ❄️ Using Arctic-Inference and Arctic-Training for improving real-world speculative decoding Performance (blog)
• 🚀 Getting started guide using ArcticTraining

See all of the speculators we have released via our Speculators Collection