centerpoint-p150 on Tenstorrent p150

Derived from AutowareFoundation/lidar_centerpoint (tag v4.1), the CenterPoint and CenterPoint-tiny networks that TIER IV trained for Autoware's autoware_lidar_centerpoint LiDAR detector, with unmodified weights. CenterPoint: Yin et al., arXiv:2006.11275. See the original model card for training details.

CenterPoint, the LiDAR 3D object detector Autoware runs by default, and its CenterPoint-tiny variant: a PointPillars pillar encoder (PFN 9-16-32), a SECOND backbone, a SECONDFPN neck and a single-task CenterPoint head, on a 480x480 head grid (base) or a 240x240 one with a stride-2 first block (tiny, a quarter of base's compute). One LiDAR point cloud in base_link goes in (one sweep, or Autoware's 2-sweep densified cloud). Out come 3D boxes of CAR, TRUCK, BUS, TRAILER, BICYCLE and PEDESTRIAN equivalent to Autoware's DetectedObjects, with the pre- and post-processing of the autoware_lidar_centerpoint node (voxelization, decode, circle NMS, IoU NMS, area-based class remap).

Runs on Tenstorrent p150 — see the serving profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Intended use

LiDAR 3D object detection for Autoware-style sensor setups (concatenated clouds in base_link): research, prototyping and offline processing of recorded LiDAR data, with the outputs of Autoware's lidar_centerpoint node.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150

Quickstart

tt serve changh95/centerpoint-p150

tt serve downloads the Docker image and the AutowareFoundation/lidar_centerpoint at 494c8171def40bd36cc2feb323e0a5acbfab132b weights (into your HF cache; they are not in the image), then starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve changh95/centerpoint-p150.

Send the sample point cloud shipped in this repo (Autoware's own test.pcd, as an npz) to /predict, on the port tt serve reported:

curl -sLO https://huggingface.co/changh95/centerpoint-p150/resolve/main/code/tt_centerpoint/samples/test_pcd.npz
printf '{"points": {"format": "npz", "data": "%s"}}' "$(base64 -w0 test_pcd.npz)" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json

Serving profiles

Select a different serving profile using --profile.

Capabilities

This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API — its request and response shapes are the model's own. See the author's notes for the payload it expects.

POST /predict takes one point cloud as JSON and returns its 3D detections. GET /health and GET /info report the server and device state (/info lists every parameter default); GET /v1/models is a stub, not a chat API.

Response for the sample cloud (base profile; two of the ten detections shown, meta shortened):

{"model": "centerpoint-p150", "frame_id": "base_link", "num_detections": 10,
 "detections": [
   {"label": "TRUCK", "label_id": 2, "score": 0.7409, "center": [24.16, 11.304, 1.693], "size": [8.131, 2.85, 3.404], "yaw": -3.1406},
   {"label": "TRUCK", "label_id": 2, "score": 0.7322, "center": [21.25, -0.423, 1.694], "size": [7.609, 2.619, 2.993], "yaw": 0.0407}],
 "meta": {"variant": "base", "num_points": 48048, "num_pillars": 9416, "pillar_overflow": false, "counts": {"after_threshold": 38, "after_circle_nms": 12, "after_iou_nms": 10}},
 "timing_ms": {"preprocess": 4.807, "device": 13.846, "postprocess": 4.768, "total": 43.301, "decode": 19.532, "model_call": 23.452}}

• center, size ([length, width, height]) and yaw are in base_link metres and radians, in Autoware's convention; label_id is the autoware_perception_msgs ObjectClassification value; detections are sorted by score, and score is Autoware's existence_probability.
• No velocity is returned (both networks have no twist head, as in Autoware). TRAILER and remapped TRUCK labels come from Autoware's area-based class remapper; meta.label_before_remap keeps the network's class.
• A Python API with the same outputs (tt_centerpoint.CenterPoint) is described in code/PYTHON.md; it needs a tt-metal build with the two patches in this repo's patches/ directory.

Example output

Boxes computed by this package (base profile) on 50 LiDAR scans of the Autoware sample rosbag (a VLS-128 roof LiDAR and two VLP-16, each scan densified with the previous one, about 10 Hz), drawn from above, also as MP4. The video shows LiDAR points and boxes only, no camera pixels. The rosbag is the one Autoware's rosbag-replay demo plays; its publisher states no licence. Attributions for every file in media/: media/ATTRIBUTION.md.

Expected performance

Agreement with the fp32 CPU reference of the same network (same weights, same pre- and post-processing, identical voxel input); detections match when they have the same label and BEV centres at most 0.5 m apart:

Public driving frames: PandaSet 019 and 090 (100), nuScenes v1.0-mini mini_val (81) and three Argoverse 2 val logs (471). Held-out frames: 45 of them plus 9 frames of the Autoware sample rosbag. The mAP rows score the networks against the datasets' ground truth with re-implemented protocols (nuScenes devkit functions with Autoware's 5 classes; an AV2-style AP, not the official CDS) on small splits; the weights repo publishes no dataset-level accuracy to compare with.

Measured with this package's code on one p150 in an 8-vCPU AMD EPYC-Rome host: code/scripts/bench.py (100 warm iterations per stage; device time over 5 rounds of 50 back-to-back replays) and 50 served requests of the sample cloud per profile. The model() and served times are host-bound: on the sample cloud, host pre-processing (reading the input file plus voxelization) takes about 10 ms of the model() call, and the server's base64 and npz decode of the request another 19 ms.

Limitations

• Do not use it to drive or control a vehicle or as any safety function: it is not a certified Autoware component and has no functional-safety qualification.
• Do not use its outputs as a CenterPoint accuracy benchmark: the weights are TIER IV's, trained on nuScenes (base) or Argoverse 2 (tiny) plus TIER IV data, and accuracy on other LiDAR setups (mounting, beam count) is not characterized; the mAP rows above come from small splits.
• Not a ROS 2 node: it serves HTTP and Python only, and takes arrays or files, not PointCloud2 messages. It runs the base and tiny models of the weights repo; the sigma (variance heads) and short_range models are not included.
• Batch 1, one cloud per request; requests are served one at a time, one model per serve profile. Up to 40,000 pillars per frame on a fixed 480x480 canvas; above that the front-most 40,000 are kept.
• One p150 only. The image runs device dispatch on the chip's Ethernet cores, so multi-chip meshes are not supported.
• Deterministic voxelization: a fixed seeded point selection per pillar replaces Autoware's random one, points at exactly (0, 0, 0) and non-finite points are dropped, and the score sort before circle NMS is stable. Outputs were compared with the fp32 CPU reference, not with Autoware's TensorRT engine.
• fp32 weights and bf16 activations: scores and boxes differ slightly from the fp32 reference. An object near Autoware's decision thresholds (score 0.35, rotation-head norm 0.3, the circle-NMS neighbourhood, the 12.1 / 36 m² remap areas) can appear, disappear or come from a neighbouring cell: 346 (base) / 171 (tiny) reference detections in 652 public frames have no partner, 30 / 44 of them with a score of 0.40 or more.
• On public frames the weights themselves (on CPU and on p150 alike) give tiny less than half of base's mAP on nuScenes' 32-beam clouds, swap buses and box trucks in tiny on Argoverse 2, label many pickups TRUCK, and turn 12-14 m buses into TRAILER through Autoware's remapper.

Licensing

• Weights: AutowareFoundation/lidar_centerpoint at tag v4.1 (commit 494c8171def40bd36cc2feb323e0a5acbfab132b), Apache-2.0 per its model card. Downloaded by tt serve at that commit, not redistributed here. The upstream card's Legal Notice: base was trained on nuScenes (CC BY-NC-SA 4.0) plus TIER IV data, tiny on Argoverse 2 (CC BY-NC-SA 4.0) plus TIER IV data; read it before commercial use.
• Pre- and post-processing ported from autoware_universe perception/autoware_lidar_centerpoint (Apache-2.0).
• Port and serving code (code/) and the tt-metal patches in patches/: Apache-2.0.
• Sample cloud code/tt_centerpoint/samples/test_pcd.npz: Autoware's own test.pcd from autoware_universe perception/autoware_ground_segmentation/test/data/ (Apache-2.0).
• Demo media in media/ are renders, not part of the package. The demo video is a LiDAR-only render of the Autoware sample rosbag, whose publisher states no licence; it contains no camera pixels and no rosbag data ships here. The nuScenes and Argoverse 2 renders (media/*_NC.jpg) are non-commercial use only, CC BY-NC-SA 4.0, © Motional AD Inc. and © 2021 Argo AI, LLC. The PandaSet renders: CC BY 4.0 and the PandaSet Dataset Terms (Scale AI and Hesai). No dataset owner endorses this work. Full attributions: media/ATTRIBUTION.md.

Related packages

• The Autoware collection: the other Autoware perception and planning models packaged for the p150.
• changh95/pointpainting-p150 and changh95/transfusion-p150: Autoware's other LiDAR 3D detectors (camera-painted CenterPoint, and TransFusion).

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/changh95/centerpoint-p150/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image: