TotalSynth: whole-body synthetic CT from MRI and CBCT

Pretrained synthetic CT (sCT) models for MRI and CBCT inputs, from the head to the pelvis, built with KonfAI. This repository holds the TotalSynth models described in TotalSynth: Robust Whole-Body Synthetic CT from MRI and CBCT. They run from the command line, from Python, and inside 3D Slicer through the ImpactSynth and KonfAI extensions, with evaluation and uncertainty estimation included.

🧩 Models

All four apps share one network: a 2.5D U-Net++ with a ResNet-34 encoder (26.1 M parameters). Five adjacent axial slices go in, the CT of the central slice comes out. Each app ships five cross-validation checkpoints (CV_0.pt to CV_4.pt) that can be ensembled at inference.

Training pairs come from the public SynthRAD challenges. The planning CT was registered to each MRI or CBCT with IMPACT-Reg, every deformation was reviewed, and 1450 of the 1800 public pairs were kept. The deformation fields are published, so the supervision can be inspected: synthrad2023-impact-registration and synthrad2025-impact-registration. The split files (CrossValidation_0..4.txt, Validation.txt) and the per-fold metrics are in MR_MODEL_INFO.md and CBCT_MODEL_INFO.md.

📊 Performance

Held-out cases, metrics computed inside the body mask against the registered CT (mean ± SD).

MAE by region, in HU:

The evaluation workflow of each app also reports the Dice of TotalSegmentator structures segmented on the sCT and on the CT, a perceptual distance in SAM 2.1 feature space, an IMPACT-based registration term between the sCT and its own input, and the ensemble uncertainty. The BIC-MAC rows show what to expect under domain shift: whole-body Dixon MRI is far from the radiotherapy training data, and fine-tuning on 45 local cases brings the error back to the in-distribution range. Validate on your own data before any clinical use, and fine-tune when the images differ from the training domain.

🚀 Usage

3D Slicer. Install the ImpactSynth extension (or KonfAI for the generic interface) from the Extensions Manager. Load an MRI or a CBCT, choose Synthesis: MR, Synthesis: CBCT or Synthesis: MR/CBCT, pick the checkpoints to ensemble, and click Run. The sCT is loaded over the input. The Evaluation section compares it with a reference CT (MAE map, PSNR, SSIM, Dice) or estimates uncertainty from the inference stack without any reference. Walkthrough: SlicerImpactSynth.

Command line.

pip install impact_synth_konfai
impact-synth-konfai synthesize MR -i input_mr.nii.gz -o output/ --gpu 0 --ensemble 5 --tta 2
impact-synth-konfai eval MR -i input_mr.nii.gz --gt reference_ct.nii.gz -o output/
impact-synth-konfai pipeline CBCT -i cbct.mha --gt ct.mha -o output/ --gpu 0 --ensemble 5 --tta 2 -uncertainty

Generic runner. konfai-apps infer VBoussot/ImpactSynth:MR -i input_mr.nii.gz -o output/ --gpu 0 --ensemble 5 --tta 2. Add --host and --port to run on a konfai-apps-server.

Fine-tune on your own pairs. Each app carries its training configuration (Config.yml), so the released weights can start a local adaptation:

konfai-apps fine-tune VBoussot/ImpactSynth:MR MyCenter_MR -d /data/pairs --models CV_0 --epochs 50 --gpu 0

The dataset directory follows the KonfAI layout of the app (one folder per case with the input and the registered CT). The result is a new local app that Slicer and the CLI run like any other.

⚙️ What an app does

1. A body mask is predicted with ImpactSeg and dilated by five voxels.
2. The input is resampled to 1 × 1 × 3 mm, clipped between its minimum and its 99.5th percentile, and scaled to [-1, 1].
3. Every selected checkpoint synthesises 512 × 512 axial patches, with flipped test-time augmentation (--tta 2 by default).
4. The predictions are averaged, scaled back to HU, masked (air outside the body) and written in the input geometry. With -uncertainty, every sampled prediction is kept in an InferenceStack for the uncertainty workflow.

Evaluation.yml compares an sCT with a reference CT: MAE with a voxel map, PSNR, SSIM, SAM perceptual distance, and the Dice of TotalSegmentator (total-3mm) segmentations of both images. Uncertainty.yml turns an inference stack into a voxel-wise variance map, a segmentation disagreement map (conformity) and their means.

⚡ Performance & VRAM

Benchmarked on a single NVIDIA RTX PRO 5000 (24 GB) with a real whole-body MR (295 × 259 × 219, 2 mm). The batch size is measured on your GPU (a forward of one patch, then of two, then the batch the free VRAM holds).

sCT generation keeps system RAM ~2 GB. A full 5-model ensemble runs in ~82 s on the same card. Override with --patch-size / --batch-size.

⚖️ License and training data

The model weights in this repository and the code that runs them (KonfAI, konfai-apps and the ImpactSynth CLI and 3D Slicer extension) are released under Apache-2.0. The data they were trained on keep their own terms, listed below.

🔗 Links & Citation

• 🧠 KonfAI: github.com/fideus-labs/KonfAI
• 🖥️ 3D Slicer: SlicerImpactSynth, SlicerKonfAI
• 📦 PyPI: impact_synth_konfai
• 📄 Paper: arXiv:2609.13838

@article{boussot2026totalsynth,
  title   = {TotalSynth: Robust Whole-Body Synthetic CT from MRI and CBCT},
  author  = {Boussot, Valentin and H{\'e}mon, C{\'e}dric and Barateau, Ana{\"i}s and Lafond, Caroline and Nunes, Jean-Claude and Dillenseger, Jean-Louis},
  journal = {arXiv preprint arXiv:2609.13838},
  year    = {2026}
}

Training methodology: Boussot V. et al., Why Registration Quality Matters: Enhancing sCT Synthesis with IMPACT-Based Registration, arXiv:2510.21358. Framework: Boussot V., Dillenseger J.-L., KonfAI: A Modular and Fully Configurable Framework for Deep Learning in Medical Imaging, arXiv:2508.09823.

The models are released for research. Every clinical use (dose calculation, contour propagation, attenuation correction, adaptive decisions) needs its own acceptance criteria.