Valen-4B-ONNX

English · 简体中文

Valen-4B-ONNX brings Valen-4B text, image and video decisions into your own application. Get categories, yes/no probabilities and scores for workflows such as ticket routing, image screening and video labeling.

Choose ONNX to deploy a Python service on CPU or embed inference in a Node.js application. Once downloaded, models can run from the cache offline and inputs are processed locally. Python inference uses ONNX Runtime without PyTorch or Transformers. Load and run the model with the Valen Python SDK or JavaScript SDK.

Try the free API and web demo with no installation, model download or GPU required.

Complete application tutorials

Complete workflows are maintained in the official Cookbook: API basics, local inference basics, ticket routing, product-image screening and video tagging. Pages import executable source and include actual outputs, uncertainty policies, media provenance and licensing notes.

This card owns format, installation, startup, limitations and minimal calls. Validation scope and same-worker performance record model revisions, SDK and runtime versions. Valen.local loads ONNX; connect to GGUF services over HTTP. Prebuilt GGUF 0.1.0 does not compile direct video support; historical source-build video tests do not qualify the prebuilt package. P1 additionally checks fixed images on the exact prebuilt binary with 0.8B Q8_0 only; see the linked evidence, without extending that scope to other sizes.

Which version should I choose?

For a first try, use the web experience or free HTTP API without downloading weights. The shared version guide covers Python/JavaScript applications, C++ local serving, GPU deployment and fine-tuning. 0.8B/2B/4B denote model size; native/ONNX/GGUF are formats; F32/Q8_0/ONNX q8-mlp are format-specific precision/quantization variants; Transformers, ONNX Runtime, custom llama.cpp/Ollama and SGLang/vLLM are backends. File size is not a RAM requirement; no unmeasured performance is promised.

Use ONNX SDKs for offline Python/JavaScript applications; GGUF with the Valen custom runtime for C++ Linux CPU services; native weights with the appropriate plugins, or Python ONNX CUDA, for GPU deployment; complete native Valen weights for fine-tuning. GGUF Q4/Q5 failed the quality gates: only qualified F32/Q8_0 are recommended. Support in official unmodified llama.cpp/Ollama is not claimed.

Examples

Click a preview to watch the full demonstration.

Choose a model

Start with 0.8B and the CPU example below. Try 2B or 4B when you want to evaluate a larger model on your own tasks.

Memory figures are measured Linux CPU text peaks, excluding vision. Images, video and longer inputs need more memory. Download sizes include vision components; see the variants below. Compare model results in the native Valen evaluations.

Quick start

The examples below use Python 3.11+ and a CPU; a GPU is not required. For Node applications, jump to JavaScript; for GPU deployment, see CUDA. Browser applications use the 0.8B WebGPU variant.

Both Decisions and SystemOne request formats are supported. Start with Decisions if you are new to Valen.

Install and load

python -m pip install "valen-sdk[onnx]==0.2.0"

from valen_sdk import Valen

client = Valen.local("Valen-Team/Valen-4B-ONNX", variant="fp32")

The published PyPI wheel does not require Git. The SDK downloads the selected model variant on first load and reuses the cache on later runs. See CUDA below for GPU installation and loading.

Minimal protocol calls

After installation/loading above, verify the path with one predicate. Complete multi-question workflows and image/video tutorials are maintained in Cookbook.

result = client.decisions(input="A cat is on the sofa.", questions=[
    {"name": "cat", "type": "predicate", "instructions": "A cat is present."},
])
print(result.by_name("cat").probability)
legacy = client.system_one("A cat is on the sofa.", {
    "cat": {"type": "noul", "instructions": "A cat is present."},
})
print(legacy.answers["cat"].noul)
client.close()

Cookbook · API

Choose a variant

Download sizes include vision components and are not runtime memory requirements. Start with fp32 on CPU. q8-mlp quantizes only the MLP to reduce download size and text memory; it was slower than FP32 in the measured CPU runs.

CUDA

Install GPU dependencies in a separate Python environment, then select cuda-fp16:

python -m pip install "valen-sdk[cuda,video]==0.2.0"

with Valen.local(
    "Valen-Team/Valen-4B-ONNX", variant="cuda-fp16",
    providers=[("CUDAExecutionProvider", {"use_tf32": "0"}), "CPUExecutionProvider"],
) as gpu:
    result = gpu.decisions(
        input="Please send my order to my new address.",
        questions=[{"name": "change_address", "type": "predicate",
                    "instructions": "Is the customer requesting a delivery address change?"}],
    )
    print(result.by_name("change_address").probability)

Local files and offline use

After the first online run, load from the cache offline, or point the SDK at an existing model directory. The two alternatives below use cached files and a local directory respectively; keep graph/data paths under onnx/ intact:

client = Valen.local("Valen-Team/Valen-4B-ONNX", variant="fp32", local_files_only=True)

client = Valen.local("./Valen-4B-ONNX", variant="fp32", local_files_only=True)

Local HTTP service

To call the model from an existing application over HTTP, start a local CPU service:

python -m pip install "valen-sdk[onnx,server]==0.2.0"
valen serve --model Valen-Team/Valen-4B-ONNX --model-name valen --port 8000

Send a request from another terminal:

curl http://127.0.0.1:8000/v1/decisions \
  -H 'Content-Type: application/json' \
  -d '{"model":"valen","input":"Please send my order to my new address.",
       "questions":[{"name":"change_address","type":"predicate",
                     "instructions":"Is the customer requesting a delivery address change?"}]}'

JavaScript / Node.js

CPU inference with Node.js 20+. Save the code as example.mjs, install the dependencies, then run node example.mjs.

npm install https://github.com/valen-team/valen-sdk-js.git @huggingface/tokenizers@0.2.0 @huggingface/jinja@0.5.10 @huggingface/transformers@4.3.1 onnxruntime-node@1.30.0

import { Valen } from '@valen-team/sdk';

const client = await Valen.local('Valen-Team/Valen-4B-ONNX', { variant: 'fp32' });
try {
  const result = await client.decisions({
    input: 'Please send my order to my new address.',
    questions: [{ name: 'change_address', type: 'predicate',
                  instructions: 'Is the customer requesting a delivery address change?' }],
  });
  console.log('Address change probability:', result.answers[0].probability);
} finally {
  await client.close();
}

Node images use the same Decisions message format; video decoding additionally requires FFmpeg and ffprobe.

Tested devices and performance

This model has been tested with text, images and video on Linux x86-64 CPU and Python H200 CUDA. macOS ARM tests cover 0.8B; this model has not been tested there. Windows real-model validation has not been run.

Measured on Linux CPU with 4 inference threads and the same text request, after warmup. Download and first model load are excluded. RSS excludes the lazily loaded vision module and is not a minimum RAM requirement. See validation.json for numerical and task regression results and the SDK documentation for full options.

API and fine-tuning

Use the free API guide for remote requests. To fine-tune, start from the native Valen model and follow the GitHub training guide. See the native model card for model benchmarks and leaderboards.

License

Apache-2.0. See LICENSE.

Citation

@misc{valen,
  author       = {{Valen Team}},
  title        = {Valen},
  howpublished = {GitHub - Valen-Team/Valen: A multimodal System One decision model for text, images and video.},
  note         = {GitHub repository},
  year         = {2026},
  url          = {https://github.com/Valen-Team/Valen}
}

User concepts, methods and references are maintained only on the website: Documentation · Probabilities.