RTMPose3D

Real-time multi-person 3D whole-body pose estimation with 133 keypoints per person.

Model Description

RTMPose3D is a real-time 3D pose estimation model that detects and tracks 133 keypoints per person:

• 17 body keypoints (COCO format)
• 6 foot keypoints
• 68 facial landmarks
• 42 hand keypoints (21 per hand)

The model outputs both 2D pixel coordinates and 3D spatial coordinates for each keypoint.

Model Variants

This repository contains checkpoints for:

The model outputs both 2D pixel coordinates and 3D spatial coordinates for each keypoint.

Model Variants

This repository contains checkpoints for:

Installation

pip install git+https://github.com/b-arac/rtmpose3d.git

Or clone and install locally:

git clone https://github.com/b-arac/rtmpose3d.git
cd rtmpose3d
pip install -r requirements.txt
pip install -e .

Quick Start

Using the HuggingFace Transformers-style API

import cv2
from rtmpose3d import RTMPose3D

# Initialize model (auto-downloads checkpoints from this repo)
model = RTMPose3D.from_pretrained('rbarac/rtmpose3d', device='cuda:0')

# Run inference
image = cv2.imread('person.jpg')
results = model(image, return_tensors='np')

# Access results
keypoints_3d = results['keypoints_3d']  # [N, 133, 3] - 3D coords in meters
keypoints_2d = results['keypoints_2d']  # [N, 133, 2] - pixel coords
scores = results['scores']              # [N, 133] - confidence [0, 1]

Using the Simple Inference API

from rtmpose3d import RTMPose3DInference

# Initialize with model size
model = RTMPose3DInference(model_size='l', device='cuda:0')  # or 'x' for extra large

# Run inference
results = model(image)
print(results['keypoints_3d'].shape)  # [N, 133, 3]

Single Person Detection

Detect only the most prominent person in the image:

# Works with both APIs
results = model(image, single_person=True)  # Returns only N=1

Output Format

{
    'keypoints_3d': np.ndarray,  # [N, 133, 3] - (X, Y, Z) in meters
    'keypoints_2d': np.ndarray,  # [N, 133, 2] - (x, y) pixel coordinates
    'scores': np.ndarray,        # [N, 133] - confidence scores [0, 1]
    'bboxes': np.ndarray         # [N, 4] - bounding boxes [x1, y1, x2, y2]
}

Where N is the number of detected persons.

Coordinate Systems

2D Keypoints - Pixel coordinates:

• X: horizontal position [0, image_width]
• Y: vertical position [0, image_height]

3D Keypoints - Camera-relative coordinates in meters (Z-up convention):

• X: horizontal (negative=left, positive=right)
• Y: depth (negative=closer, positive=farther)
• Z: vertical (negative=down, positive=up)

Keypoint Indices

Training Data

The models were trained on the Cocktail14 dataset, which combines 14 public 3D pose datasets:

• Human3.6M
• COCO-WholeBody
• UBody
• And 11 more datasets

Performance

Evaluated on standard 3D pose benchmarks:

• RTMW3D-L: 0.045 MPJPE, real-time inference (~30 FPS on RTX 3090)
• RTMW3D-X: 0.057 MPJPE, slower but higher accuracy

Requirements

• Python >= 3.8
• PyTorch >= 2.0.0
• CUDA-capable GPU (4GB+ VRAM recommended)
• mmcv >= 2.0.0
• MMPose >= 1.0.0
• MMDetection >= 3.0.0

Citation

@misc{rtmpose3d2025,
  title={RTMPose3D: Real-Time Multi-Person 3D Pose Estimation},
  author={Arac, Bahadir},
  year={2025},
  publisher={GitHub},
  url={https://github.com/b-arac/rtmpose3d}
}

License

Apache 2.0

Acknowledgments

Built on MMPose by OpenMMLab. Models trained by the MMPose team on the Cocktail14 dataset.

Links

• GitHub Repository: b-arac/rtmpose3d
• Documentation: See README in the repository
• MMPose: open-mmlab/mmpose