CaiYuanhao/X-Gaussian

Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis

ECCV 2024

5 CT scenes + 3 real scans · X-ray novel view synthesis · Direct 3D CT reconstruction

Paper · Code · Models · Data

Point Cloud Visualization

Training Process Visualization

GIF demonstrations are from the official X-Gaussian repository. The available CT and real-scan scenes are listed below.

Model Description

X-Gaussian represents X-ray attenuation using radiative 3D Gaussians. Introduced in Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis (ECCV 2024), it supports efficient X-ray novel view synthesis. These checkpoints use the current joint 2D/3D implementation, which also directly reconstructs a CT volume from the same Gaussian field while retaining the original Gaussian point attributes.

This repository provides eight scene-specific trained models: Chest, Abdomen, Foot, Head, Pancreas, and the real FIPS scans Pine, Seashell, and Walnut, paired with the X-Gaussian dataset. Each model represents its corresponding scene; reconstructing another scene requires training from that scene's projections and scanner geometry.

Available Checkpoints

All weights are the final iteration 31,000 checkpoints from the eight runs reported in the code repository.

Each scene folder preserves the layout expected by the project's render.py:

<synthetic|real>/<scene>/
├── config.json
├── metrics.json
└── point_cloud/
    └── iteration_31000/
        └── point_cloud.ply

config.json contains the training settings, decoder configuration, and original voxel geometry. Its data path is relative to the project root (data/synthetic/<scene>_50.pickle or data/real/<scene>). Outputs follow the same layout: output/synthetic/<scene>/ or output/real/<scene>/. The PLY includes global decoder and geometry metadata and retains the original 62 per-point columns. It can be used independently for 3D volume export; 2D rendering also needs the corresponding camera geometry. manifest.json records file sizes and SHA-256 checksums.

Download checkpoints

The root config.json lists this repository's checkpoints, supporting files, and implementation. Include it when downloading selected weights so those downloads can be counted by the Hugging Face Hub. Use the linked implementation and its matching runtime configuration to load the checkpoints.

pip install -U huggingface_hub
hf download CaiYuanhao/X-Gaussian --include "config.json" "synthetic/**" "real/**" "manifest.json" --local-dir pretrained

Training and Evaluation

Each model was trained from 50 X-ray projections at the released dataset resolution, using training-only FDK initialization, 30,000 base training steps, and 1,000 volume-ray refinement steps. Training jointly uses projection loss, 3D total variation, and volume-ray consistency. Reference CT volumes were used only for evaluation.

Performance

X-ray Novel View Synthesis

Results on the original five CT scenes, averaged over 50 held-out views per scene:

Each predicted and reference projection is normalized by its own maximum.

Original paper comparisons

The figures reproduce the original paper results; the table above evaluates the hosted joint 2D/3D checkpoints.

Direct 3D CT Reconstruction

Results measured on the five released models after 30,000 training steps and 1,000 refinement steps, without downsampling:

3D metrics use a fixed range of 1 on the original voxel grid, without clipping or intensity correction. SSIM averages nonempty reference slices along each axis, then across axes. CT ground truth is used only for evaluation.

Real Data

Real FIPS scans use 50 training and 100 held-out views, 560 × 560 detector images, and a 256³ reconstruction grid, without additional downsampling.

2D metrics average 100 held-out measured projections per scene, each normalized by its own maximum. The 3D reference is full-view FDK, not measured CT ground truth. 3D metrics use a fixed range of 1 without clipping or intensity correction. Each model uses 30,000 training and 1,000 refinement steps; the reference volume and test views are not used for training or initialization.

Left to right: measured X-ray, rendered X-ray, full-view FDK pseudo-reference, and X-Gaussian CT. CT previews use the same fixed 0–1 intensity range.

See the real-data sources and CC BY 4.0 attribution.

Usage

Please refer to the X-Gaussian repository for environment setup and model usage, including 2D projection rendering and direct 3D CT reconstruction.

Reconstruction Previews

Central orthogonal slices from the hosted models. In each image, the top row shows the reference CT, the middle row the reconstruction, and the bottom row the absolute error. Displayed CT intensities use a fixed range of 0–1 and errors use 0–0.1; metrics above use the unmodified arrays.

Chest Abdomen Foot Head Pancreas Pine (real; full-view FDK reference) Seashell (real; full-view FDK reference) Walnut (real; full-view FDK reference)

Citation

If you use these models, please cite X-Gaussian and the SAX-NeRF dataset source:

@inproceedings{x_gaussian,
  title={Radiative gaussian splatting for efficient x-ray novel view synthesis},
  author={Yuanhao Cai and Yixun Liang and Jiahao Wang and Angtian Wang and Yulun Zhang and Xiaokang Yang and Zongwei Zhou and Alan Yuille},
  booktitle={ECCV},
  year={2024}
}

@inproceedings{sax_nerf,
  title={Structure-Aware Sparse-View X-ray 3D Reconstruction},
  author={Yuanhao Cai and Jiahao Wang and Alan Yuille and Zongwei Zhou and Angtian Wang},
  booktitle={CVPR},
  year={2024}
}