Comet 60B

Powered by CompactifAI

50% Fewer Parameters · Text, Image & Video Input · MTP Speculative Decoding

---

Table of Contents

• Model Overview
• License / Terms of Use
• Key Characteristics
• Quick Start
• Reasoning Control
• Tool Calling
• Training & Fine-Tuning
• Evaluation & Benchmarks
• Model Specifications
• Languages
• Safety & Limitations
• Ethical Considerations
• Model Information
• Citation

---

Model Overview

Comet 60B is a multimodal model based on NVIDIA Nemotron 3.5 Super VL 120B, developed by Multiverse Computing. The original model is a 124.5B-parameter hybrid Mamba2-Transformer Mixture-of-Experts model from NVIDIA, with about 12B parameters active per token. It takes text, image and video input, produces text, and supports configurable reasoning and tool calling.

Comet applies CompactifAI compression to halve the parameter count to 61.1B while keeping about 12B parameters active per token, like the original. The layer structure, tokenizer and chat template are unchanged, and the multi-token prediction (MTP) layer is kept. After compression, the model was healed by distilling the original Nemotron 3.5 Super on text, image and video data.

• Half the memory, faster serving: the BF16 checkpoint is 122.3 GB instead of about 249 GB for Nemotron 3.5 Super, so it runs on 2× H200 instead of 4 (the FP8 version on a single H200, the NVFP4 version is 40.5 GB). On the same 4× H200, Comet serves 20–23% more output tokens per second than Nemotron 3.5 Super on NVIDIA's chat and agentic traces; the FP8 version on 2× H200 delivers about 1.8× the original's throughput per GPU (see Serving Throughput).
• Stronger agents: ahead of the original on τ²-Bench (airline, retail and telecom), AutomationBench-AA and SWE-bench Pro.
• Close elsewhere: within 3 points on 28 of 35 scores, including GPQA Diamond, AA-LCR, SWE-bench Verified and most image, document and video benchmarks.
• Same serving stack: the same vLLM reasoning and tool parsers as the base model, with MTP speculative decoding.

Available formats:

• BF16 (122.3 GB)
• FP8 (66.3 GB, for NVIDIA Hopper and Blackwell; see FP8 Quantization)
• NVFP4 (40.5 GB, for NVIDIA Blackwell; see NVFP4 Quantization)

This card covers all three versions. Unless a section says otherwise, it describes the BF16 model.

---

License / Terms of Use

Model weights: Comet 60B is a derivative of NVIDIA Nemotron 3.5 Super and is governed by the OpenMDW License Agreement, version 1.1. Code: the model code shipped in this repository (modeling_*.py, configuration_*.py, image_processing_*.py) and the reasoning-parser plugin are NVIDIA code under the Apache License 2.0, as stated in their file headers.

This model is ready for commercial use under the terms above.

---

Key Characteristics

---

Quick Start

Use temperature=1.0 and top_p=0.95 for all tasks (reasoning, tool calling and chat).

vLLM Serving

Installation

All benchmarks on this card were run with the vLLM nightly 0.28.1rc1.dev628+g2a02f6efe (PyTorch 2.13, CUDA 13.0), with NVIDIA's MTP fix for hybrid Mamba models applied on top (bxyu-nvidia/vllm, branch bxyu/mtp-fix-try02, commit 64edb1d9; it changes vllm/v1/core/kv_cache_coordinator.py and vllm/v1/core/single_type_kv_cache_manager.py). The exact container we used is:

docker pull vllm/vllm-openai@sha256:96b234afa2867031ad0a226b149a06483861e613e3a3083da53398d347b5ffbd

The MTP fix is not required: the serving command below also works on the plain container, with the same answers and prefix-cache reuse in our tests.

Newer vLLM releases have not been validated with this model yet. If you use one, check that MTP speculative decoding and prefix caching work together (see the note under Serve).

Serve

Recommended serving command (4× H200, tensor parallel 4). It is the one used for the throughput numbers; the quality benchmarks used the same flags with 5 MTP draft tokens and --max-num-seqs 64, which do not change the outputs (MTP speculative decoding is lossless):

export VLLM_USE_V2_MODEL_RUNNER=0
export VLLM_SSM_CONV_STATE_LAYOUT=DS

vllm serve MultiverseComputingCAI/Comet_60B_A12B_BF16 \
  --served-model-name model \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --trust-remote-code \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.85 \
  --kv-cache-dtype fp8 \
  --mamba-ssm-cache-dtype float32 \
  --enable-prefix-caching \
  --no-disable-hybrid-kv-cache-manager \
  --block-size 128 \
  --mamba-cache-mode align \
  --enable-chunked-prefill \
  --max-num-seqs 128 \
  --max-num-batched-tokens 33920 \
  --async-scheduling \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser nemotron_v3 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' \
  --limit-mm-per-prompt '{"image": 128, "video": 1}'

The two environment variables are required for MTP speculative decoding with this hybrid model.

Do not combine MTP with --enable-mamba-fine-grained-prefix-cache: in our tests the pair produced garbled text in about 1% of long agentic sessions (either flag alone was fine).

FP8 version: serve MMultiverseComputingCAI/Comet_60B_A12B_FP8 with the same flags. It fits on a single H200: set --tensor-parallel-size 1 and drop --enable-expert-parallel (tested on H200 with 1 and 2 GPUs).

NVFP4 version: serve MultiverseComputingCAI/Comet_60B_A12B_NVFP4 with the same flags. On B200/B300 the 40.5 GB checkpoint fits on a single GPU: set --tensor-parallel-size 1 and drop --enable-expert-parallel.

Context length is 256K above. To go up to 1M, set VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 and --max-model-len 1048576 (tested: the server starts on 4× H200 with room for 31 concurrent 1M-token requests, and retrieves a fact placed anywhere in a prompt of up to ~980K tokens). Quality benchmarks above 256K were not run.

For video input, our Video-MME runs used 2 frames per second, capped at 256 frames:

  --media-io-kwargs '{"video": {"fps": 2, "num_frames": 256}}'

Request (text + image)

import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

with open("chart.png", "rb") as f:
    image_b64 = base64.b64encode(f.read()).decode()

response = client.chat.completions.create(
    model="model",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image_b64}"}},
            {"type": "text", "text": "What trend does this chart show?"},
        ],
    }],
    temperature=1.0,
    top_p=0.95,
    max_tokens=65536,
)
message = response.choices[0].message
print(getattr(message, "reasoning", None) or getattr(message, "reasoning_content", None))  # reasoning trace
print(message.content)                                                                     # final answer

A public image URL works too: pass it directly as {"url": "https://..."}. With thinking on, leave plenty of room in max_tokens: if the limit is hit before the reasoning ends, content comes back empty.

Request (video)

response = client.chat.completions.create(
    model="model",
    messages=[{
        "role": "user",
        "content": [
            {"type": "video_url", "video_url": {"url": "https://example.com/clip.mp4"}},
            {"type": "text", "text": "Summarize what happens in this video."},
        ],
    }],
    temperature=1.0,
    top_p=0.95,
    max_tokens=65536,
)
print(response.choices[0].message.content)

The server samples the video into frames; see the --media-io-kwargs note above to set the frame rate and the frame cap.

Recommended sampling: temperature=1.0, top_p=0.95. All benchmarks below used these settings.

Transformers

The examples below use the BF16 checkpoint. The FP8 and NVFP4 checkpoints are meant for vLLM or another engine that supports ModelOpt FP8 / NVFP4 checkpoints.

The model uses custom code (NemotronH_Omni_Reasoning_V3), so load it with trust_remote_code=True. The checkpoint was saved with Transformers 5.12.1.

Tested with Transformers 5.12.1 (text and image input).

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_id = "MultiverseComputingCAI/Comet_60B_A12B_BF16"

processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [{"role": "user", "content": "Write a haiku about GPUs"}]
inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=True, temperature=1.0, top_p=0.95)
print(processor.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Image input

The processor also returns bookkeeping fields (num_patches, num_tokens, imgs_sizes) that generate() does not accept; remove them before generating:

from PIL import Image

image = Image.open("chart.png")
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "Describe the trend in this chart."},
    ],
}]
inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
)
for key in ("num_patches", "num_tokens", "imgs_sizes"):
    inputs.pop(key, None)
inputs = inputs.to(model.device)

outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=True, temperature=1.0, top_p=0.95)
print(processor.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Video input

Pass one video as a list of PIL frames (sample them from the file first, for example at 2 frames per second). Use a {"type": "video"} placeholder in the message, build the prompt with the chat template, and give the frames to the processor:

from PIL import Image

frames = [Image.open(f"frames/{i:04d}.png") for i in range(16)]  # one video = list of PIL frames
messages = [{
    "role": "user",
    "content": [
        {"type": "video"},
        {"type": "text", "text": "Summarize what happens in this video."},
    ],
}]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=[prompt], videos=frames, return_tensors="pt")
for key in ("num_patches", "num_tokens", "imgs_sizes"):
    inputs.pop(key, None)
inputs = inputs.to(model.device)

outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=True, temperature=1.0, top_p=0.95)
print(processor.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

For long videos or batched requests, use vLLM.

---

Thinking (Reasoning) Control

Comet 60B keeps the base model's hybrid reasoning mode: it either thinks step by step before answering (reasoning mode) or replies directly (non-reasoning mode). The enable_thinking flag in the chat template controls which, and thinking is on by default. The template also takes two more flags:

This section is a short overview. For full details, see the official Nemotron 3.5 Super VL 120B model card

Transformers API

Pass enable_thinking through apply_chat_template:

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
    enable_thinking=False,   # True for reasoning mode
)

vLLM

Server-level default

Set the default for all requests at startup with --default-chat-template-kwargs:

vllm serve MultiverseComputingCAI/Comet_60B_A12B_BF16 \
  --served-model-name model \
  --reasoning-parser nemotron_v3 \
  --default-chat-template-kwargs '{"enable_thinking": false}' \
  ...

Per-request override

Individual requests can override the server default by passing chat_template_kwargs in the request body:

import requests

response = requests.post("http://localhost:8000/v1/chat/completions", json={
    "model": "model",
    "messages": [{"role": "user", "content": "Solve: x² - 5x + 6 = 0"}],
    "max_tokens": 65536,
    "temperature": 1.0,
    "top_p": 0.95,
    "chat_template_kwargs": {"enable_thinking": True},
})

With thinking on, a small share of responses can run very long. Set max_tokens to a limit that suits your workload. If a response hits the limit before the reasoning ends, it is returned as reasoning with empty content.

---

Tool Calling

Serve the model with --enable-auto-tool-choice --tool-call-parser qwen3_coder, as in the serving command above. The qwen3_coder parser turns the model's tool calls into standard OpenAI tool_calls:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["city"],
        },
    },
}]

messages = [{"role": "user", "content": "What's the weather in Paris?"}]
response = client.chat.completions.create(
    model="model", messages=messages, tools=tools,
    temperature=1.0, top_p=0.95, max_tokens=65536,
)
message = response.choices[0].message
for call in message.tool_calls or []:
    print(call.function.name, call.function.arguments)  # get_weather {"city": "Paris", "unit": "celsius"}

# Send the tool result back and let the model continue
messages += [message, {"role": "tool", "tool_call_id": message.tool_calls[0].id, "content": '{"temp_c": 18}'}]
final = client.chat.completions.create(model="model", messages=messages, tools=tools, max_tokens=65536)
print(final.choices[0].message.content)

The model can call several tools in the same turn. Tool results are passed back to the model inside <tool_response> blocks in a user turn; the chat template does this for you.

Under the hood, the model writes each call in this format (string values as plain text, objects and lists as JSON):

<tool_call>
<function=get_weather>
<parameter=city>
Paris
</parameter>
<parameter=unit>
celsius
</parameter>
</function>
</tool_call>

---

Training & Fine-Tuning

Base Model: Nemotron 3.5 Super

Nemotron 3.5 Super VL 120B is a multimodal hybrid Mamba2-Transformer Mixture-of-Experts model trained by NVIDIA. It is a single model for both reasoning and non-reasoning tasks, with reasoning set by a chat-template flag. See the original model card for details.

CompactifAI Compression

Comet 60B was compressed with CompactifAI, Multiverse Computing's compression technology based on quantum-inspired techniques. It halves the parameter count from 124.5B to 61.1B while keeping about 12B parameters active per token.

Healing

Compression is followed by knowledge distillation from the original Nemotron 3.5 Super. The original model writes the training responses, and Comet is trained toward its token-level output distribution. The healing data spans:

• single-turn text: math, science, code, reasoning, instruction following and multilingual chat;
• multi-turn agentic trajectories: terminal, software-engineering and tool-use sessions;
• images and video.

The MTP block is distilled together with the main head. All evaluations below ran with MTP speculative decoding on (5 draft tokens).

Reinforcement Learning

Reinforcement Learning targets what healing leaves behind. The model undergoes GRPO-style training on several environments to further improve on key areas like multi-turn conversation, multi-step tool use and mitigation of hallucinations, while controlling for chain of thought length and verbosity.

FP8 Quantization

The FP8 checkpoint is a post-training quantization of the BF16 model with NVIDIA Model Optimizer, following the FP8 layout of NVIDIA Nemotron 3 Super FP8 (NVIDIA has not published an FP8 version of Nemotron 3.5 Super):

NVFP4 Quantization

The NVFP4 checkpoint starts from a post-training quantization (PTQ) of the BF16 model with NVIDIA Model Optimizer, using the same mixed-precision layout as NVIDIA Nemotron 3.5 Super NVFP4:

The precision of every module matches NVIDIA Nemotron 3.5 Super NVFP4.

The PTQ checkpoint was then refined with quantization-aware distillation (QAD) from the original Nemotron 3.5 Super.

---

Evaluation & Benchmarks

All scores come from our internal runs. Nemotron 3.5 Super and Comet 60B were run with the same harnesses, prompts, judges and sampling settings (see Evaluation Methodology), so the columns are directly comparable. They can differ from scores published by NVIDIA or Artificial Analysis, which use their own setups.

Comet 60B is within 3 points of the original on 28 of the 35 scores below and within 5 points on 32. It is ahead on all three τ²-Bench domains, AutomationBench-AA, SciCode, SWE-bench Pro and BabyVision. The FP8 version stays within 3 points of the original on 27 of the 35 scores.

Text & Agentic

Vision & Video

NVFP4: Comet vs NVIDIA Nemotron 3.5 Super NVFP4

Comet 60B NVFP4 (40.5 GB) against NVIDIA Nemotron 3.5 Super NVFP4 (75.1 GB), both on 1× H200 with the same harness. Comet is within 3 points on 25 of the 35 scores and within 5 points on 33. It is ahead on all three τ²-Bench domains, τ³-Banking and AutomationBench-AA. NVIDIA Nemotron 3.5 Super NVFP4 ran without MTP speculative decoding (its MTP layer does not load in this vLLM build); MTP affects speed, not scores.

Serving Throughput

Output tokens per second on H200 with vLLM 0.28.1rc1.dev628+g2a02f6efe and the serving command above (MTP, 3 draft tokens), measured with NVIDIA AIPerf replaying NVIDIA's chat and agentic traces from its Dynamo recipe for Nemotron 3 Super in full. Both models ran with the same command and the same load per GPU (8 chat or 16 agentic requests per GPU).

• Chat: multi-turn conversations. Median request of about 8K input and 1K output tokens; many requests share the conversation history with earlier ones (NVIDIA's trace targets 70% KV-cache reuse).
• Agentic: agent sessions with tool calls, where each step re-sends the growing session context. Median request of about 64K input and 400 output tokens, with most of the prompt shared with the previous step (NVIDIA's trace targets 90% KV-cache reuse).

Latency per request, from the same runs (TTFT = time to first token, ITL = inter-token latency; p50 / p90 over all requests):

Against Nemotron 3.5 Super on 4× H200:

• Comet BF16 on the same 4 GPUs: TTFT p90 −23% (chat) and −16% (agentic); ITL −18% and −19%; tokens per second per user +23% and +24%.
• Comet FP8 on 2 GPUs: TTFT p90 −39% and −45%; ITL −49% and −43%; tokens per second per user +97% and +75%.

Comet's weights take half the memory, which leaves room for a larger KV cache: on 4× H200 it holds 24.0M tokens against 14.1M for Nemotron 3.5 Super, enough for 92 concurrent 256K-token requests instead of 54. More of each prompt is then served from the prefix cache.

Evaluation Methodology

Both models ran on the same stack, with the same harnesses, prompts, judges and sampling.

• Inference: vLLM 0.28.1rc1.dev628+g2a02f6efe with NVIDIA's MTP fix (see Installation), FP8 KV cache, MTP speculative decoding with 5 draft tokens. Nemotron 3.5 Super on 4× H200 (tensor parallel 4), Comet 60B on 2× H200 (BF16) or 1× H200 (FP8, NVFP4).
• Decoding: temperature 1.0, top_p 0.95, thinking on, no output cap beyond the 262,144-token context (exceptions in note 4).
• Throughput: vLLM 0.28.1rc1.dev628+g2a02f6efe (plain container, no MTP fix) with the recommended serving command; NVIDIA AIPerf 0.13 replaying the 15% subsets of NVIDIA's 8k_1k_70kv_chat (1,767 requests) and 64k_400_90kv_agent (3,410 requests) traces in full, with ignore_eos, requests longer than the 256K context removed and the prefix cache reset before each run; concurrency 32 (chat) and 64 (agentic) per 4 GPUs. Repeated runs differed by less than 2%.
• Scoring: mean over all rollouts (avg@N). No best-of-N or majority voting.
• Harnesses: text and agentic benchmarks use NeMo Gym (commit 4101fa5) with NVIDIA's Nemotron 3.5 Super settings. LLM judges and user simulators are called through OpenRouter.

Notes

1. AA-Omniscience and AutomationBench-AA use the public sets, not Artificial Analysis's private ones. GDPval-AA is NeMo Gym's rubric score, not AA's pairwise Elo. None of these are AA leaderboard numbers.
2. Some judges and simulators differ from NVIDIA's: AA-LCR v1.1 with gpt-5.6-luna (NVIDIA: v1.0 with a Qwen judge), and gpt-6-luna as the τ² and τ³ user (NVIDIA: gpt-5.2 and gpt-5.4-mini). These scores are not directly comparable with NVIDIA's published ones.
3. NVIDIA reports Video-MME with VLMEvalKit; we use lmms-eval.
4. Harness output caps: 131,072 tokens for AutomationBench and VLMEvalKit, 64,000 per call for the GDPval agent.
5. Single-repeat benchmarks move by 1–2 points between runs. Treat differences under about 3 points as noise.

Datasets, the NeMo Gym commit, the vLLM image and the MTP patch are all pinned, so every run can be reproduced.

---

Model Specifications

Input

• Input Type(s): Text, image, video (audio input is not supported)
• Input Format(s): String (text); RGB images (PNG, JPEG); video files sampled into frames
• Input Parameters: One-dimensional (1D) token sequences; two-dimensional (2D) images; three-dimensional (3D) video
• Other Properties Related to Input: Evaluated up to 262,144 tokens of context

Output

• Output Type(s): Text
• Output Format: String
• Output Parameters: One-dimensional (1D) token sequences
• Other Properties Related to Output: Reasoning trace (when thinking is on) followed by the final answer; tool calls in the format shown in Tool Calling

Software Integration

• Runtime Engine(s): vLLM 0.28.1rc1.dev628+g2a02f6efe (tested with and without NVIDIA's MTP fix)
• Supported Hardware: BF16: NVIDIA Hopper (H100, H200), NVIDIA Blackwell (B200, B300). FP8: NVIDIA Hopper (H100, H200), NVIDIA Blackwell (B200, B300). NVFP4: NVIDIA Blackwell (B200, B300) for native NVFP4; Hopper through weight-only FP4 kernels
• Test Hardware: benchmarks on H200 (Nemotron 3.5 Super on 4×, Comet BF16 on 2×, FP8 and NVFP4 on 1×); serving throughput on H200 (BF16, FP8); NVFP4 also served on 1× B300
• Operating System(s): Linux

Use Case

Comet 60B is a general-purpose multimodal reasoning and chat model for developers building assistants, agents, RAG systems and document, chart and video understanding applications, at about half the memory footprint of Nemotron 3.5 Super.

Model Version(s)

• v1.0.0

---

Languages

• Primary language: English

The healing data was mainly English, with a multilingual chat set added. Apart from OCRBench v2 (CN), we have not systematically evaluated languages other than English.

---

Safety & Limitations

Known Limitations

• Audio input is not supported. The processor keeps audio-related fields inherited from the Nemotron Omni architecture, but the model has no audio encoder.
• Tool calling depends on correct schema and tool design; exact parity with the original model is not guaranteed.
• Compression may affect some behaviors; evaluate for your use case.
• All limitations of the base Nemotron 3.5 Super model also apply.

Recommendations

• Validate tool outputs before running them
• Pair the model with retrieval or verification for factual, knowledge-intensive use cases
• Set max_tokens when thinking is on
• Human oversight for critical use
• Task-specific eval before production

---

Ethical Considerations

Multiverse Computing believes trustworthy AI is a shared responsibility. Developers should work with their internal model teams to make sure this model meets the requirements of their industry and use case, and addresses unforeseen product misuse. Compression and healing can change model behavior relative to the base model, including on safety-relevant prompts; evaluate before deployment and keep the guardrails of the base model in place.

All ethical considerations, safety and bias notes of the base Nemotron 3.5 Super VL 120 model also apply.

---

Model Information

---

Citation

If you use this model, please cite the base model and Comet 60B:

@misc{nemotron35super,
  title         = {TODO: NVIDIA Nemotron 3.5 Super technical report},
  author        = {{NVIDIA}},
  year          = {2026},
  url           = {TODO_NEMOTRON_35_SUPER_REPORT_URL}
}
@misc{comet1_60b,
  title         = {Comet 60B: Model developed from NVIDIA Nemotron 3.5 Super},
  author        = {Multiverse Computing},
  year          = {2026},
  url           = {https://huggingface.co/MultiverseComputingCAI/Comet-1.0-60B-BF16},
  note          = {Model developed based on NVIDIA Nemotron 3.5 Super using CompactifAI technology}
}

Built by Multiverse Computing · Report an issue · Discord