UniMate uniml3d_60frames_graph_adaln — independently trained checkpoint (v2)

A from-scratch training run of the main UniMate model ("UniMate: One Unified Model to Animate Diverse Skeletons", Mou et al., SIGGRAPH Asia 2026, arXiv 2609.05415) on a single GPU, using the authors' released code and the shipped configs/uniml3d_60frames_graph_adaln.json config unchanged.

This repository holds two runs. The files at the root are v2, trained on an audited copy of the dataset from which 13 defective clips were removed and 18 repaired (see Why v2). The earlier v1 run on the unaudited data is kept under v1/ for comparison. Use v2.

This is not the authors' checkpoint. It was trained separately, by a third party, on one workstation GPU. The paper's example launch uses 8 GPUs, which at the same per-GPU batch size would see 8× more samples over the same 120k steps. Expect this model to differ from the official release and from any other independent run: different hardware, precision, sampling order and random seed all change the final weights. Numbers reported in the paper do not apply to this checkpoint unless you evaluate it yourself.

Which file to use

Running inference with the UniMate code

UniMate's sampler expects an experiment directory laid out the way training writes it, and this repository's root mirrors that layout. Download it, then point --exp_dir at it. Exclude v1/ and audit/ to keep the download at 1.7 GB:

git clone https://github.com/Friedrich-M/UniMate && cd UniMate
# set up the environment per the UniMate README, and build dataset/features/ with stage 4 of
# its data pipeline (the sampler reads the target skeleton's T-pose and topology from there)
hf download tarn59/UniMate-Weights --local-dir outputs/uniml3d_60frames_graph_adaln --exclude "v1/*" --exclude "audit/*"

python -m unimate.inference.sample \
    --exp_dir outputs/uniml3d_60frames_graph_adaln \
    --test_cases_json test_cases.json \
    --num_repetitions 3

The sampler picks the latest file in checkpoints/ (here the single step-120000 v2 checkpoint) and swaps in its EMA weights automatically, so nothing else needs to be passed. To run v1 instead, download with --include "v1/*" and point --exp_dir at the v1/ directory.

To reproduce v2's training data, apply the skip list in audit/applied_skip_list_objaverse.txt and the export-level repairs recorded in audit/patch_log.json (script: audit/patch_poison.py) before running UniMate's stage-4 feature extraction. The 18 repaired clips are edits to the stage-1 export NPZs, not to the published UniML3D data.

Loading the safetensors directly

The .pt checkpoint is the format UniMate's scripts read; the safetensors files are provided as a smaller, framework-neutral copy of the weights. To load one into the model class:

import torch
from safetensors.torch import load_model
from unimate.configs.schema import MainConfig
from unimate.models.factory import create_model

cfg = MainConfig.from_json("outputs/uniml3d_60frames_graph_adaln/config.json")
model = create_model(cfg.dataset, cfg.model)
load_model(model, "outputs/uniml3d_60frames_graph_adaln/model_ema.safetensors", strict=True)
model.eval()

Use safetensors.torch.load_model, not load_file + load_state_dict: the spectral-RoPE encoder is one module shared by all ten transformer blocks, so state_dict() lists it under eleven names. The safetensors files store it once, under rope_j.*, and load_model re-ties the aliases.

Why v2

The v1 run trained cleanly except for four isolated single-step loss spikes, at steps 43,600 (loss 91, pre-clip gradient norm 588), 78,100 (loss 29), 106,900 and 112,900. Each recovered by the next logged step, gradient clipping bounded the damage and the EMA averaged it out, so v1 is usable. But every spike was a wasted, clipped update in a bad direction, and the UniMate README warns that exactly this pattern comes from defective Objaverse rigs and clips.

To find them, every one of the 10,204 training clips was scored with the frozen v1 EMA model under three draws of noise, timestep and crop, giving a per-clip loss, alongside a model-free score: the largest normalized feature value in the clip. The distribution is a clean bell that ends by a loss of 2, and then three clips sit alone at 7.7, 13.8 and 153 (1,140 × the median). Diagnosis of the top 24 found one defect behind almost all of them: the root joint travelling many body lengths per frame while the limbs animate normally. In canonical units (body diameter 2.0) a normal clip's root moves 0.007 per frame; the worst moved 6. Captions such as "walks in place" on a root that crossed 5,000 rig units confirmed the root tracks were exported wrong.

One repaired clip (videoClass) was subsequently rejected by stage 4's own discontinuity filter, which found nine jump frames once the root drift was gone. Net: 10,204 → 10,191 clips, a 0.13 % change. Re-scoring the repaired clips against v1 put every one inside the normal feature range.

Full details, per-clip scores for all 10,204 clips, and the scripts are in audit/.

v2 vs v1

Same config, seed, precision and hardware; only the data differs.

The spikes are gone and the baseline is unchanged to within noise, which is the expected effect of a 0.13 % data change. v2 is not measurably better on training loss than v1; it is cleaner in provenance and free of the clipped bad updates. No quantitative evaluation has been run on either.

Model

74.1 M parameters. A factored spatio-temporal diffusion transformer: 10 blocks, 512-wide latent, 8 heads, SwiGLU MLP (2048 nominal), RMSNorm with QK-norm, adaLN-Zero conditioning. Spatial attention runs within each frame across joints with Graphormer-style graph-distance and edge-type biases and a sign-invariant spectral RoPE over 8 Laplacian eigenvectors; temporal attention runs within each joint across frames with 1-D RoPE. The caption (google/flan-t5-base, mean-pooled) and the pooled T-pose enter through the adaLN vector. Trained with flow matching (linear interpolant, velocity prediction) under a masked L2 loss plus a geodesic rotation loss (weight 0.5), with 10 % caption dropout for classifier-free guidance. See the UniMate repository for the full description.

Training run (v2)

Loss (v2)

No excluded steps: these are the raw windows. The trainer's non-finite-loss guard never fired.

Example samples (v2)

Four of the visualizations the trainer rendered at the final checkpoint, sampled from the EMA weights with classifier-free guidance 3.0 and the dopri5 ODE solver. Stick-figure renders via forward kinematics, prompt in the title; the rigs are training rigs with their training captions. They are the first four of the six rigs the visualization step drew at random, not a curated set.

v1's examples are under v1/examples/.

License and data provenance

The weights, configs, logs, audit files and example renders in this repository are released under the MIT License, matching the license of the UniMate code they were produced with. The training data keeps its sources' terms: Mixamo assets under Adobe's Mixamo terms, Objaverse-XL assets under each object's own license, and the Truebones ZOO animal motions under Truebones' commercial terms, which permit royalty-free use including research but forbid redistributing the motion files. No motion data is included in this repository; only weights derived from it, the per-channel normalization statistics, the run configs, and per-clip loss scores. Review the source licenses before using this model commercially.

Citation

If you use this checkpoint, please cite the original work:

@article{mou2026unimate,
  title   = {UniMate: One Unified Model to Animate Diverse Skeletons},
  author  = {Mou, Linzhan and Lei, Jiahui and Dou, Zhiyang and Cai, Chenyue and Song, Chaoyue and Finkelstein, Adam and Rusinkiewicz, Szymon},
  journal = {arXiv preprint arXiv:2609.05415},
  year    = {2026}
}