Agnes-3.0-Flash Preview

Model version clarification

This repository contains an earlier open-weight Preview checkpoint of Agnes 3.0 Flash. It is distinct from the newer production/API checkpoint listed on Artificial Analysis. The Preview release has 33B parameters and a context window of 262,144 tokens. The production/API model uses a different checkpoint and configuration, with a 1M-token context window. Its benchmark results should not be attributed to the Preview weights released here. This repository was initially published as Agnes-3.0-Flash without the Preview suffix. The model card now explicitly identifies this release as Agnes-3.0-Flash Preview to clarify the distinction between the open-weight release and the production/API model. The specifications and Agnes benchmark results below refer to the Preview checkpoint. Hello! 👋 Today we are introducing Agnes-3.0-Flash Preview, an open-weights multimodal preview model built for people who want flagship-class reasoning without flagship-class hardware. Highlights:

• Competitive across core capabilities. Agnes-3.0-Flash Preview posts competitive results across reasoning, coding, and instruction-following evaluations.
• Built for demanding work. A 262 144-token context window, adjustable reasoning effort, tool calling, and text, image and video understanding.

Benchmarks

Benchmark scope: The Agnes results in the chart and table below belong to the Agnes-3.0-Flash Preview open-weight checkpoint released in this repository. They are not results for the production/API Agnes 3.0 Flash model listed on Artificial Analysis.

The Agnes-3.0-Flash Preview scores in the chart correspond to the open-weight checkpoint released in this repository. Reference results across contemporary models are shown below. The figures were compiled from different sources, harnesses, and model snapshots and do not constitute a controlled head-to-head comparison.

Higher is better for every row. Header parameter figures mix total and active counts, and harnesses and snapshot dates differ across sources, so treat cross-column comparisons as reference values rather than a controlled head-to-head evaluation.

Architecture

Agnes-3.0-Flash Preview is a hybrid-attention decoder: three of every four layers run a gated delta rule (recurrent, with per-layer state independent of sequence length), and the fourth runs standard global attention. Only 18 of the 72 layers therefore hold a KV cache that grows with context.

Quickstart

REMOTE CODE REQUIRED

Agnes-3.0-Flash Preview ships its own model implementation. Always load it with trust_remote_code=True.

Requirements

pip install "transformers>=5.12" torch torchvision accelerate

Tested on transformers 5.12.1. Image and video inputs go through the bundled processor, which needs torchvision.

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
path = "Agnes-AI/Agnes-3.0-Flash"
tok = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(
    path, dtype="bfloat16", device_map="auto", trust_remote_code=True
)
msgs = [{"role": "user", "content": "请用三句话解释什么是人工智能。"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

Images and video

Image and video inputs go through the bundled processor (also remote code):

from transformers import AutoProcessor
proc = AutoProcessor.from_pretrained(path, trust_remote_code=True)
msgs = [{"role": "user", "content": [{"type": "image", "image": "photo.jpg"},
                                     {"type": "text", "text": "描述这张图。"}]}]
inputs = proc.apply_chat_template(msgs, add_generation_prompt=True, tokenize=True,
                                  return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256)
print(proc.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])

Reasoning effort

The chat template exposes three reasoning levels — high (default), medium, low — plus a thinking-off switch:

ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt",
                              reasoning_effort="medium")   # or enable_thinking=False

Tool calling

The chat template renders tool definitions for you. The model emits calls as <tool_call><function=…><parameter=…>, and you feed results back as a tool role message:

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Look up current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string", "description": "City name"}},
            "required": ["city"],
        },
    },
}]
msgs = [{"role": "user", "content": "What's the weather in Beijing right now?"}]
ids = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True,
                              return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256)
reply = tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)

# <tool_call>

# <function=get_weather>

# <parameter=city>

# Beijing

# </parameter>

# </function>

# </tool_call>

# run the tool, append the result, generate the final answer
msgs += [{"role": "assistant", "content": reply},
         {"role": "tool", "content": "Clear, 26°C, light northeasterly wind"}]

Over the OpenAI API pass tools= the same way. The server returns the text above verbatim by default; to get structured tool_calls, configure sglang with a tool-call parser matching this format (likewise a reasoning parser, if you want the thinking span in reasoning_content).

SGLang

serve.sh starts a server from a stock public image, overlaying three files onto the image's sglang package and nothing else. See sglang_patch/README.md.

docker run --gpus all --shm-size 64g -p 30001:8080 \
    -v /path/to/agnes-3.0-flash:/model \
    lmsysorg/sglang:nightly-dev-20260908-20ca564b \
    bash /agnes-3.0-flash/serve.sh --served-model-name Agnes-3.0-Flash

serve.sh forwards extra command-line arguments to sglang, which is how --served-model-name takes effect; --tp 2 works the same way. The server listens on port 8080 inside the container:

from openai import OpenAI
client = OpenAI(api_key="EMPTY", base_url="http://localhost:30001/v1")
response = client.chat.completions.create(
    model="Agnes-3.0-Flash",
    messages=[{"role": "user", "content": "Design a fault-tolerant event processing architecture."}],
    temperature=1.0,
    max_tokens=2000,
)
print(response.choices[0].message.content)

Pass stream=True for streaming; tools= and reasoning_effort= are accepted the same way.

Hardware Requirements

Actual context length and concurrency depend on KV-cache allocation, runtime overhead, and tensor-parallel configuration; validate the target workload on the intended hardware.

Recommended Inference Settings

These are the checkpoint's own generation_config.json defaults.

Model Capabilities

License

Released under the Apache License 2.0.

Citation

@misc{agnes30flash2026,
  title        = {Agnes-3.0-Flash Preview},
  author       = {{Agnes AI}},
  year         = {2026},
  month        = sep,
  howpublished = {Open-weights preview checkpoint},
  url          = {https://agnes-ai.com/}
}