Antares-350M GGUF

Validated Q8_0 GGUF conversion of fdtn-ai/antares-350m, an Apache-2.0-licensed 350M-parameter model by Cisco Foundation AI for agentic vulnerability localization.

File

The source was pinned to 4bf5b0c49c0841c20c2289f86f0030d56ad1d1be. It was converted to a BF16 GGUF and then quantized to Q8_0 with llama.cpp commit fb0e6b621917488d623437349fb5361e0ac21c70. Q8_0 was chosen because this model is small enough that the near-lossless quant remains compact while avoiding the quality risk of aggressive low-bit quantization.

Validation

• GGUF architecture: granite
• Context length: 32768
• Blocks: 28
• Tensors: 254 ({"F32": 57, "Q8_0": 197})
• Embedded chat template: True
• BF16 GGUF llama.cpp load/generation: passed
• Q8_0 GGUF llama.cpp load/generation: passed
• Transformers reference: Paris. Paris is the capital of France. Paris is the capital of France.
• BF16 GGUF reference: The user is asking for the capital city of France. I should provide a clear
• Q8_0 GGUF reference: The question seems to be a simple factual request. I should provide the answer to

The machine-readable build and validation details are in build-manifest.json.

Use with llama.cpp

Use llama.cpp at commit fb0e6b621917488d623437349fb5361e0ac21c70 or newer:

llama-cli -m antares-350m-Q8_0.gguf --jinja -cnv

Antares is not a general-purpose chat model. It is designed to run in a sandboxed terminal-agent loop that accepts CWE information, explores a source repository, and submits candidate vulnerable files. Review the original model card for the prompt/tool protocol, intended use, limitations, and safety recommendations.

License and provenance

The source model and its IBM Granite 4.0 base model declare Apache License 2.0. This repository includes the full license in LICENSE and attribution/change notices in NOTICE. Quantization does not imply affiliation with or endorsement by Cisco or IBM.