meteor-p150 on Tenstorrent p150

Derived from AutowareFoundation/meteor (tag v1.0, meteor_v157c3Z.onnx), the METEOR surround-view multi-task driving network that TIER IV trained and released through the Autoware Foundation, with unmodified weights. There is no Autoware package for METEOR; the pre- and post-processing follow METEOR's own runtimes in tier4/METEOR. See the original model card for training details.

METEOR v157, a 48.25M-parameter camera-only perception and planning network (a ResNet-34 + FPN image encoder over the eight cameras, 2D segmentation, depth, 2D detection and traffic-light heads, a depth-gated lift to one 800x500 bird's-eye view at 0.2 m, BEV heads for lanes, 3D detection, occupancy, agent futures and risk, and an ego planner), run on eight 768x432 camera images (2.58 TFLOPs per frame). Eight raw, unrectified surround-view images with their intrinsics and extrinsics and the ego speed go in. Out come 3D boxes (vehicle / VRU) with stationary flags and futures, per-camera 2D boxes, the ego-relevant traffic light, a BEV lane map and three ego paths with steer, acceleration and brake, with the pre- and post-processing of METEOR's C++ runtime (resize, calibration-dependent lift tables, thresholds, NMS, plan-mode hysteresis, temporal smoothing per stream).

Runs on Tenstorrent p150 — see the serving profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Intended use

Camera-only multi-task perception and end-to-end planning research for eight-camera rigs laid out like TIER IV's: prototyping and offline processing of recorded drives, with the outputs of METEOR's own runtime.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150
• Time for the first start: on an empty kernel cache the server compiles its kernels for about 12-15 minutes on an 8-vCPU host (per profile); later starts are ready in under a minute.

Quickstart

tt serve changh95/meteor-p150

tt serve downloads the Docker image and the AutowareFoundation/meteor at 01a5f6d71df5ecbbb5853ec600825481d57b9c6b weights (into your HF cache; they are not in the image), then starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve changh95/meteor-p150.

Send the synthetic eight-camera sample shipped in this repo to /predict, on the port tt serve reported. client.py (standard-library Python) builds the request from the sample's images, calibration preset and ego speed:

hf download changh95/meteor-p150 --include "code/tt_meteor/*" --local-dir meteor-p150 && cd meteor-p150
python3 code/tt_meteor/server/client.py --sample code/tt_meteor/samples/synthetic_8cam.json --out req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json

Serving profiles

Select a different serving profile using --profile.

Capabilities

This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API — its request and response shapes are the model's own. See the author's notes for the payload it expects.

POST /predict takes one frame of eight camera images as JSON and returns the boxes, the plan and the dense BEV outputs. GET /health and GET /info report the server and device state (/info lists the calibration presets); GET /v1/models is a stub, not a chat API.

Response of the default fast profile for the sample (three of the eight 3D boxes shown, lists and the lane PNG elided):

{"model": "meteor-p150", "frame_id": "base_link", "num_detections": 8,
 "detections": [
   {"label": "VEHICLE", "label_id": 0, "score": 0.9933, "center": [-12.86, 0.096], "size": [4.392, 1.742], "yaw": -0.0695, "stationary": false, "future_mode": 0, "future": [[-11.92, -0.09], [-10.74, -0.16], "..."]},
   {"label": "VEHICLE", "label_id": 0, "score": 0.9873, "center": [-7.173, -3.215], "size": [3.962, 1.729], "yaw": 0.0198, "stationary": false, "...": "..."},
   {"label": "VEHICLE", "label_id": 0, "score": 0.9626, "center": [10.84, 0.019], "size": [4.341, 1.762], "yaw": -0.0188, "stationary": false, "...": "..."}],
 "trajectory": [[0.73, 0.92], [0.83, 1.56], [0.38, 2.36], [0.28, 3.10], [0.57, 3.84], [0.23, 4.47]], "columns": ["x", "y"],
 "plan": {"mode": 0, "mode_probs": [0.235, 0.296, 0.469], "paths": ["..."], "steer": 0.0025, "accel": 0.019, "brake_prob": 0.127, "dt": 0.5},
 "detections_2d": {"CAM_FRONT_WIDE": [{"label": "car", "label_id": 1, "score": 0.5224, "box_xyxy": [412.86, 211.09, 438.41, 230.22]}, "..."], "...": "..."},
 "unknown_obstacles": [],
 "traffic_light": {"state": "red", "state_id": 3, "probs": [0.360, 0.139, 0.031, 0.470]},
 "lane": {"format": "png", "key": "lane", "dtype": "uint8", "shape": [800, 500], "data": "..."},
 "lane_classes": ["bg", "road", "sidewalk", "crosswalk", "laneline", "stopline", "road_edge", "marking", "parking"],
 "meta": {"present": [true, true, true, true, true, true, true, true], "v0": 8.0, "stream_id": "synthetic_8cam"},
 "timing_ms": {"decode": 81.37, "preprocess": 3.0, "device": 118.21, "postprocess": 9.32, "model_call": 130.61, "total": 216.24}}

• 3D boxes are in base_link (x forward, y left, metres): center [x, y], size [length, width], yaw in radians counter-clockwise from +x; METEOR predicts no height, z or velocity. future is the agent's best mode, 6 points at 0.5-3 s.
• trajectory is the selected ego path, 6 points at 0.5 s steps; plan holds the three modes with their probabilities, steer (rad), accel (m/s²) and brake_prob.
• detections_2d are per camera in pixels of the 768x432 input. lane is a uint8 PNG of 800x500 cells of 0.2 m (row 0 is 80 m ahead, column 0 is 50 m to the left).
• The sample is a test pattern (a red light and a car 11 m ahead), so its plan barely moves; the fp32 CPU reference gives the same path (within 0.06 m with fast, 0.03 m with accurate).
• A Python API with the same outputs (tt_meteor.METEOR) is described in code/PYTHON.md; it needs a tt-metal build with the two patches in this repo's patches/ directory.

Example output

All heads computed on the p150 for 40 consecutive frames of PandaSet sequence 029 (San Francisco, hilly residential streets, daylight, 5 Hz, shown at 8 frames/s): the front camera with projected 3D boxes, lane classes and the planned path, three front cameras with 2D boxes, and the bird's-eye view with lanes, occupancy, risk, 3D boxes with futures and the three plan modes; also as MP4. The video was rendered with this port's initial version (same weights). Rendered from PandaSet, CC BY 4.0 and the PandaSet Dataset Terms (Scale AI and Hesai); Scale AI and Hesai do not endorse this work. Attributions for every file in media/: media/ATTRIBUTION.md.

Expected performance

The two serve profiles differ in precision. fast (the default) runs the image branch, the 3D detection stem and the box refiner with single-term bf16 activations (fp32 weights at HiFi4; bf16 weights at HiFi2 in the 2D detection, traffic-light and 2D segmentation heads) and uses bf16 weights at HiFi2 in most BEV decoders and heads. accurate (--profile accurate) keeps fp32 activations in the image branch, the 3D detection path and the box refiner, every one of their convs as three bf16 terms, with fp32 weights at HiFi4. Both use bf16 activations in the BEV trunk and fp32 accumulation. Agreement with the fp32 CPU reference of the same network (same weights, same pre- and post-processing; the reference matches ONNX Runtime on meteor_v157c3Z.onnx, minimum PCC 0.9999999):

The public frames are 13 frames each of PandaSet sequences 019 and 090 and 12 key-frames of nuScenes scene-0103, with the six real cameras mapped onto METEOR's slots, a virtual FRONT_NARROW crop and an absent BACK_NARROW; the 45-frame rows add 7 in-domain METEOR frames and run every sequence as one stream with the host temporal state and rig changes. They are out of domain for the weights, so they measure agreement, not accuracy; the path ADE against the logged ego path is a sanity check. fast is accepted on task-level agreement over the 45 frames (3D box recall and precision >= 0.95, worst path deviation <= 0.3 m, plan mode and traffic-light agreement >= 0.95, lane agreement >= 0.98 on every frame); PandaSet 029 was not used to choose the precision settings. METEOR has no paper or public evaluation split, so no dataset metric is reported. Of the unmatched reference boxes of the 1 m row, 6 of 13 (fast) and 7 of 9 (accurate) score within 0.0015 of their class threshold.

Measured with this package's code on one p150 in an 8-vCPU AMD EPYC-Rome host: code/scripts/bench.py (30 warm iterations) and 20 served requests of the sample per profile to the package's server app run under uvicorn on the host (not in the container), at a load average of about 4.5-5. With fast the served call is no longer device-bound: of its 216 ms, 81 ms decode the request body and 118 ms are the device call (the upload, the 103 ms trace replay and about 15 ms more until the outputs are read back). Inside the container image, 2,100 frames per profile with a calibration change on every frame (four rigs, lift tables rewritten each time) gave bit-identical outputs per rig: 117 ms (fast) and 293 ms (accurate) per frame for upload, replay and readback, plus a 5 ms table write.

Limitations

• Do not use it to drive or control a vehicle or as any safety function: METEOR is a research model that must not control a vehicle on public roads, there is no Autoware node for it, and this is not a certified Autoware component.
• Do not use its outputs as a METEOR accuracy benchmark: the weights are trained on TIER IV's own eight-camera recordings in Japan, which are not public, and accuracy on other rigs and domains is not characterized.
• Not a ROS 2 node: it serves HTTP and Python only.
• A rig that puts more than 4 cameras over one 0.4 m lift cell is refused with HTTP 400. Every rig validated has at most 3.
• Batch 1, one frame per request; requests are served one at a time. Fixed shapes: eight 768x432 camera slots in METEOR's order and an 800x500 BEV. A calibration seen for the first time builds its lift tables on the host once (about 0.2 s); a known rig costs 5 ms.
• One p150 only. The image runs device dispatch on the chip's Ethernet cores, so multi-chip meshes are not supported.
• fast (the default) is reduced precision: against the fp32 reference it moves about 1 % of the dense-map pixels (lane, 2D segmentation, depth bins) and drops or adds near-threshold boxes more often (2 of the 28 boxes of the nuScenes gate frame, both 0.004 above the 0.15 threshold; 2D boxes recall / precision 0.977 / 0.991 on the 45 frames, 0.993 / 0.998 with accurate). On the held-out PandaSet 029 its worst planned-path deviation is 0.264 m (0.169 m with accurate). Use --profile accurate when per-output parity with the fp32 network matters; it runs 2.7 times longer on the device.
• bf16 and fp32 numerics on the chip in both profiles: scores, boxes and plan logits differ from the fp32 reference, so near-threshold boxes and plan modes can flip. Outputs were compared with the fp32 CPU reference, not with METEOR's TensorRT runtimes (fp16 or INT8).
• The released graph's input normalisation (/255) and log-spaced depth bins are kept for parity; the trained ImageNet normalisation and linear bins are a load-time option of the Python API.
• On rigs other than TIER IV's the weights themselves (on CPU and on p150 alike) lose accuracy, depth first: on PandaSet 019 the fp32 reference finds vehicles with recall / precision 0.46 / 0.91 within 20 m of the annotations, and its front-camera depth reads 1.65 times the LiDAR range.

Licensing

• Weights: AutowareFoundation/meteor at tag v1.0 (commit 01a5f6d71df5ecbbb5853ec600825481d57b9c6b), Apache-2.0 per its model card. Downloaded by tt serve at that commit, not redistributed here. Trained by TIER IV on its own recordings with auto-generated ground truth; no training data is public.
• Pre- and post-processing ported from METEOR's runtimes, tier4/METEOR deploy/cpp at dc193a81e9959de57b751e269127ae9b67e3f300 (Apache-2.0).
• Port and serving code (code/) and the tt-metal patches in patches/: Apache-2.0.
• Sample code/tt_meteor/samples/synthetic_8cam* and the calibration preset synthetic_8cam: generated by this package (code/scripts/make_synthetic_sample.py, no third-party data), Apache-2.0.
• Demo media in media/ are renders, not part of the package. The demo video and the other PandaSet renders: CC BY 4.0 and the PandaSet Dataset Terms (Scale AI and Hesai). The nuScenes renders media/*_NC*: non-commercial use only, CC BY-NC-SA 4.0, © Motional AD Inc. Neither dataset owner endorses this work. Full attributions: media/ATTRIBUTION.md.

Related packages

• The Autoware collection: the other Autoware perception and planning models packaged for the p150.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/changh95/meteor-p150/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image: