rf-detr-p150 on Tenstorrent p150

Derived from Roboflow/rf-detr-base (RF-DETR-B) by Roboflow, with unmodified weights at commit 7b95b089. Paper: arXiv:2511.09554. Upstream code: roboflow/rf-detr. Port: changh95/tt-RF-DETR. See the original model card for training and evaluation details.

RF-DETR-base (RF-DETR-B), a 29.0M-parameter real-time detection transformer trained on COCO: a DINOv2-S/14 backbone with windowed attention, a C2f projector, two-stage query selection and a 3-layer deformable-attention decoder with 300 object queries. One PNG or JPEG image goes in and is squashed to 560x560 for the network. Out come labelled, scored boxes for the 80 COCO object categories, in the pixel coordinates of the original image: the best class of each query, kept when its score is above a threshold (0.5 by default).

Runs on Tenstorrent p150 (mesh P150).

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Intended use

Object detection of the 80 COCO categories in still images, one image at a time: research, prototyping, and offline analysis or labelling of image collections.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150

Quickstart

tt serve changh95/rf-detr-p150

tt serve downloads the Docker image and the Roboflow/rf-detr-base at 7b95b089788e6c7db56d5ea9b0a07ca08ea6ac0a weights (into your HF cache; they are not in the image), then starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve changh95/rf-detr-p150.

Send the demo image of this repo to /predict, on the port tt serve reported:

curl -sLO https://huggingface.co/changh95/rf-detr-p150/resolve/main/media/demo_source.png
printf '{"image":"%s"}' "$(base64 -w0 demo_source.png)" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json

Capabilities

This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API — its request and response shapes are the model's own. See the author's notes for the payload it expects.

POST /predict takes one image as JSON and returns the objects found in it. GET /health and GET /info report the server and device state; GET /v1/models is a stub, not a chat API.

Response for the demo image (media/demo_source.png, 640x480):

{"detections": [
   {"label": "cat", "label_id": 17, "score": 0.9597620368003845, "box": [6.88, 53.44, 318.12, 471.56], "box_cxcywh_norm": [0.253906, 0.546875, 0.486328, 0.871094]},
   {"label": "cat", "label_id": 17, "score": 0.929440438747406, "box": [341.88, 23.44, 638.12, 372.19], "box_cxcywh_norm": [0.765625, 0.412109, 0.462891, 0.726562]},
   {"label": "remote", "label_id": 75, "score": 0.8976953029632568, "box": [40.62, 72.66, 175.62, 117.66], "box_cxcywh_norm": [0.168945, 0.198242, 0.210938, 0.09375]},
   {"label": "remote", "label_id": 75, "score": 0.7256485819816589, "box": [334.14, 77.11, 370.86, 187.27], "box_cxcywh_norm": [0.550781, 0.275391, 0.057373, 0.229492]},
   {"label": "couch", "label_id": 63, "score": 0.6909762620925903, "box": [0.0, 0.94, 640.0, 473.44], "box_cxcywh_norm": [0.5, 0.494141, 1.0, 0.984375]}],
 "num_detections": 5, "image_size": {"width": 640, "height": 480}, "input_size": [560, 560], "threshold": 0.5, "preprocess_on_device": false, "timing_ms": {"decode": 13.46, "preprocess": 3.94, "inference": 3.28, "postprocess": 0.34, "total": 21.03}}

• box is [x1, y1, x2, y2] in pixels of the original image, clamped to the image; box_cxcywh_norm is the same box as [cx, cy, w, h] normalised to [0, 1]. Detections are sorted by score.
• label_id is the COCO category id of the weights' config.json (id2label).
• timing_ms gives the server-side stages of the request. Image decode, resize and normalisation run on the host CPU, so preprocess_on_device is always false.
• Errors: 400 for an image that cannot be decoded or is too large, 422 for invalid request fields, 503 while the server is starting.
• A Python API with the same outputs (rf_detr.RFDETRBase) is described in code/PYTHON.md, with an example in examples/quickstart.py. It needs a host ttnn build of the tt-metal commit in Provenance with patches/tt-metal-8b98410.local.patch applied.

Example output

The response above drawn on the demo image: two cats, two remotes and a couch. The photo is COCO val2017 image 39769 (CC BY-SA 2.0, see Licensing).

Expected performance

COCO val2017 box detection, the benchmark RF-DETR publishes, and agreement with the upstream model run in fp32 on the CPU:

The published figure is the RF-DETR-B row of the roboflow/rf-detr 1.1.0 README (COCO val2017, 5,000 images). The fp32 CPU reference is transformers RfDetrForObjectDetection with the same weights and the same 560x560 preprocessing, run on the first 1,000 val2017 images by image id; the PCC row covers the 489 of the first 500 with at least two foreground queries to match.

Measured with this package's image on one p150 in an 8-vCPU AMD EPYC-Rome host whose load average was about 9 to 12 from other jobs, with code/scripts/bench_p150.py (500 calls per Python row; the device row is 20 rounds of 500 replays) and code/scripts/bench_http.py (1,000 requests per served row, one at a time on one keep-alive connection); of the served PNG request, the PNG decode on the host takes 15.6 ms (median).

Limitations

• Not for safety-critical decisions: driving or other vehicle or robot control, security or surveillance decisions about people, or any use where a missed or false detection can cause harm, without independent validation for that use. It detects object categories; it does not identify individuals.
• Not for training or fine-tuning, and not for other RF-DETR checkpoints: inference only, with these COCO weights.
• Only the 80 COCO categories; no other domain or dataset was tested.
• One image per request, batch 1, requests served one at a time. Every image is squashed to 560x560 without letterboxing, so the aspect ratio is not kept; EXIF orientation is not applied.
• One p150 only (one chip): multi-chip meshes are not supported, and other boards were not tested.
• /predict and model() return at most one detection per query (its best class). Upstream RF-DETR takes the top 300 over all query-class pairs, so a query can also give a second class there: 51.8 against 52.5 mAP on COCO val2017 for the same device outputs.
• The device computes in bfloat16, with block-float8 weights in the projector: 0.9 mAP below the fp32 reference on 1,000 COCO images, most of it on small objects (AP_S 35.0 against 37.0). 100 of the reference's 4,708 detections above 0.5 have no match, and the scores of matched detections differ by a median 0.004 (p99 0.070).
• Image decode, resize and normalisation run on the host CPU in PIL and torch, so the latency of model() and /predict depends on the host's CPU and rises with its load; the device time does not.

Licensing

• Weights: Roboflow/rf-detr-base at commit 7b95b089788e6c7db56d5ea9b0a07ca08ea6ac0a, Apache-2.0. Downloaded by tt serve at that commit, not redistributed here.
• Port and serving code (code/): Apache-2.0. It includes files copied or adapted from tt-metal (Apache-2.0, Tenstorrent; named in their headers), and the CPU reference model in code/rf_detr/reference/ adapts code from huggingface/transformers and roboflow/rf-detr (both Apache-2.0).
• patches/tt-metal-8b98410.local.patch: the changes the image's tt-metal carries on top of the public commit in Provenance (Blackhole dispatch on Ethernet cores, a row-major reshape fix); tt-metal is Apache-2.0.
• media/demo_source.png, media/demo_detections.png and code/data/cats_000000039769.jpg are COCO val2017 image 39769 (Flickr photo, CC BY-SA 2.0) and this package's detections drawn on it; they keep that licence.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/changh95/rf-detr-p150/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image: