yolox-p150 on Tenstorrent p150
Derived from AutowareFoundation/tensorrt_yolox (tag
v1.0), the YOLOX-s+ seg16 network that TIER IV trained for Autoware'sautoware_tensorrt_yoloxcamera detector, with unmodified weights. YOLOX: Ge et al., arXiv:2107.08430. See the original model card for training details.
YOLOX-s+ seg16, a 15.5M-parameter single-stage camera detector (ELAN CSPDarknet backbone, PAFPN neck, anchor-free decoupled head on strides 8, 16 and 32) with a 16-class semantic-segmentation decoder, run on a 960x960 letterboxed input (121 GFLOPs per image). One camera image of any size goes in. Out come 2D boxes for 8 classes (UNKNOWN, CAR, TRUCK, BUS, BICYCLE, MOTORBIKE, PEDESTRIAN, ANIMAL), the Autoware ROI records and a 16-class semantic mask, with the pre- and post-processing of the autoware_tensorrt_yolox node (letterbox, decode, NMS, class remap, ROI overlay on the mask).
Runs on Tenstorrent p150 (mesh P150).
Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).
At a glance
| Hardware | p150 |
| License | apache-2.0 |
Intended use
Camera 2D object detection and semantic segmentation for Autoware-style perception: research, prototyping and offline processing of recorded driving images, with the outputs of one autoware_tensorrt_yolox node per camera.
Prerequisites
- The Tenstorrent CLI,
tt:uv tool install tenstorrent(or use tt-model alone, see the Quickstart) - Docker
- Tenstorrent hardware: p150
Quickstart
tt serve changh95/yolox-p150
tt serve downloads the Docker image and the AutowareFoundation/tensorrt_yolox at 424d6b9cb94c939ae9ce6a22380e46c13f76b1da weights (into your HF cache; they are not in the image), then starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.
Without tt-cli: tt-model serve changh95/yolox-p150.
Send the sample image shipped in this repo (Autoware's own test image) to /predict, on the port tt serve reported:
curl -sLO https://huggingface.co/changh95/yolox-p150/resolve/main/code/tt_yolox/samples/test_image.jpg
printf '{"images": [{"camera": "CAM_FRONT", "data": "%s"}]}' "$(base64 -w0 test_image.jpg)" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
Capabilities
This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API — its request and response shapes are the model's own. See the author's notes for the payload it expects.
POST /predict takes one camera image as JSON and returns its detections, ROIs and mask. GET /health and
GET /info report the server and device state; GET /v1/models is a stub, not a chat API.
| request field | meaning |
|---|---|
images |
exactly one entry, {"camera": "<name>", "data": "<base64 PNG or JPEG>"}; any image size; decoded to BGR8 as the Autoware node receives it |
params (optional) |
score_threshold (default 0.35), nms_threshold (0.7), is_roi_overlap_segmentation (true), overlap_roi_score_threshold (0.3), is_publish_color_mask (false) |
output_format (optional) |
json (default; the mask as a base64 PNG) or npz (the mask and the box arrays as a lossless npz) |
Response for the sample image (two of the nine detections and one ROI shown, the mask data elided):
{"model": "yolox-p150", "frame_id": "camera", "num_detections": 9,
"detections": [
{"label": "CAR", "label_id": 1, "score": 0.9356, "box_xyxy": [344.0, 376.0, 398.0, 410.0]},
{"label": "TRUCK", "label_id": 2, "score": 0.9307, "box_xyxy": [556.0, 308.0, 832.0, 465.0]}],
"semseg": {"format": "png", "key": "mask", "dtype": "uint8", "shape": [620, 960], "data": "<base64 PNG>"},
"rois": [{"x_offset": 344, "y_offset": 376, "width": 54, "height": 34, "label": "CAR", "label_id": 1, "autoware_label": 1, "score": 0.935619}],
"meta": {"variant": "clip6", "image_hw": [827, 1280], "mask_hw": [620, 960], "scale": 0.75},
"timing_ms": {"preprocess": 9.1, "device": 8.9, "postprocess": 1.5, "total": 48.5, "decode": 19.5, "model_call": 19.5}}
box_xyxyis in original-image pixels: the integer Autoware ROI, also returned inroiswith the AutowareObjectClassificationidautoware_label. Detections are sorted by score.semsegis the Autoware mask: one uint8 class id per pixel (the 16 classes ofsemseg_color_map.csvin the weights repo), at network resolution, cropped to the image area (620x960 for a 1280x827 image).- A Python API with the same outputs (
tt_yolox.YOLOX) is described incode/PYTHON.md; it needs a tt-metal build with the two patches in this repo'spatches/directory.
Example output
Boxes and segmentation computed by this package on 60 front-camera images of nuScenes scene-1094 (Singapore, night
after rain, about 12 Hz), also as MP4. Non-commercial use only: rendered from the nuScenes
dataset, © Motional AD Inc., CC BY-NC-SA 4.0 and the nuScenes Terms of Use;
Motional does not endorse this work. Attributions for every file in media/: media/ATTRIBUTION.md.
Expected performance
Agreement with the fp32 CPU reference of the same network (same weights, same pre- and post-processing):
| eval | this package | fp32 CPU reference | images |
|---|---|---|---|
| Detections matched one-to-one (same class, IoU ≥ 0.5), sample image | 9 of 9, max score difference 0.0024 | 9 | 1 |
| Detections matched one-to-one, public driving images | 3,370 of 3,394 (99.3 %) | 3,394 | 265 |
| Detections with score ≥ 0.40 that change class, public driving images | 1 | – | 265 |
| Detection-head output PCC, sample image / minimum over 24 public images | 0.99956 / 0.99885 | – | 1 / 24 |
| Semantic-mask pixels equal to the reference, sample image | 99.85 % | – | 1 |
| Semantic-mask pixels equal to the reference, public driving images: mean / minimum | 99.76 % / 98.40 % | – | 265 |
| 2D recall against projected 3D ground-truth boxes (IoU ≥ 0.3), cars | 448 / 462 (97.0 %) | 448 / 462 (97.0 %) | 238 |
| 2D recall against projected 3D ground-truth boxes (IoU ≥ 0.3), pedestrians | 421 / 571 (73.7 %) | 421 / 571 (73.7 %) | 238 |
Public driving images: all cameras of PandaSet sequences 005, 019 (day) and 065 (night) and of nuScenes v1.0-mini scenes (day and night). The recall rows use the 238 of them that have 3D labels; they check the detector against ground truth, not a 2D benchmark. The weights repo publishes no dataset-level accuracy to compare with.
| speed (batch 1, warm) | input | median | throughput |
|---|---|---|---|
| Device time per image, back-to-back trace replays | 960x960 network input | 6.49 ms | 154 images/s |
Python model() call (decode, letterbox, device, post-processing) |
1280x827 JPEG (sample image) | 30.1 ms (p99 31.3) | 33 images/s |
Python model() call |
1920x1080 JPEG (PandaSet) | 30.2 ms (p99 32.4) | 33 images/s |
Served POST /predict, server timing_ms.total |
1280x827 JPEG (sample image) | 48.9 ms | – |
Served POST /predict, client round trip on loopback |
1280x827 JPEG (sample image) | 55.0 ms | – |
Measured with this package's code on one p150 in an 8-vCPU AMD EPYC-Rome host: code/scripts/bench.py (100 warm
iterations per stage; device time over 5 rounds of 50 back-to-back replays) and 50 served requests of the sample
image. The model() and served times are host-bound: image decode and letterbox take about 18 ms of the call on
the sample image and 19.5 ms on the PandaSet image.
Limitations
- Do not use it to drive or control a vehicle or as any safety function: it is not a certified Autoware component and has no functional-safety qualification.
- Do not use its outputs as a YOLOX accuracy benchmark: the weights are TIER IV's T4 fine-tune, whose training data is not published, and accuracy on other cameras and domains is not characterized.
- Not a ROS 2 node: it serves HTTP and Python only. It runs the
seg16model of the weights repo; the detection-only, tiny and traffic-light models are not included. - Batch 1, one image per request; requests are served one at a time. The network input is fixed at 960x960.
- One p150 only. The image runs device dispatch on the chip's Ethernet cores, so multi-chip meshes are not supported.
- Numerics follow Autoware's deployed engine in saturation (every activation clipped at 6), not in int8 quantization; outputs were compared with the fp32 CPU reference, not with TensorRT.
- bf16 activations: scores and boxes differ slightly from the fp32 reference. An object near the 0.35 score threshold or a near-duplicate pair near the 0.7 NMS IoU can appear or disappear, and an object whose two best classes score almost equally can change class (1 object above score 0.40 in 265 public images).
- On public driving images the weights themselves (on CPU and on p150 alike) label San Francisco buses as
TRUCK, almost never predictsidewalk(it comes out asroad), predictcrosswalk_otherson road texture and give construction barriers no box.
Licensing
- Weights: AutowareFoundation/tensorrt_yolox at tag
v1.0(commit424d6b9cb94c939ae9ce6a22380e46c13f76b1da), Apache-2.0 per its model card. Downloaded bytt serveat that commit, not redistributed here. The upstream card does not document the training data of its T4 fine-tunes. - Pre- and post-processing ported from autoware_universe
perception/autoware_tensorrt_yolox(Apache-2.0). - Port and serving code (
code/) and the tt-metal patches inpatches/: Apache-2.0. - Sample image
code/tt_yolox/samples/test_image.jpg: the Autoware package's own test image (Apache-2.0). - Demo media in
media/are renders, not part of the package. The demo video andmedia/*_NC.jpgcome from nuScenes: non-commercial use only, CC BY-NC-SA 4.0, © Motional AD Inc. The PandaSet renders: CC BY 4.0 and the PandaSet Dataset Terms (Scale AI and Hesai). Neither dataset owner endorses this work. Full attributions:media/ATTRIBUTION.md.
Related packages
- The Autoware collection: the other Autoware perception and planning models packaged for the p150.
Feedback
Questions or problems with this package: open a discussion at https://huggingface.co/changh95/yolox-p150/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.
Provenance
The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:
| component | built from |
|---|---|
| tt-metal | 44d66500520fda9f2c7060c0f6b41ec48f7ab37e (dirty tree — the image includes uncommitted changes) |
code/ digest |
42fe41168085a734 (sha256, first 16 hex digits) |
| built | 2026-10-16T16:37:27+00:00 by tt-model 0.1.0 |