kairos-person-track
The four graphs of EdgeTAM's promptable video pipeline, in a precision mix for the browser, plus the two learned tensors the JavaScript memory bank needs. It runs in the kaairos video editor on onnxruntime-web 1.30 (WebGPU), where it follows one person the editor picks through a clip so the picture can blur them or keep only them.
Files
| file | precision | bytes |
|---|---|---|
vision_encoder.onnx |
fp32 | 19 826 101 |
mask_decoder.onnx |
fp16, fp32 I/O | 9 005 990 |
memory_attention.onnx |
fp16, fp32 I/O | 10 536 679 |
memory_encoder.onnx |
fp16, fp32 I/O | 3 428 754 |
bank_params.json |
fp32 | 14 743 |
bank_params.json holds no_memory_embedding (1,1,256) and memory_temporal_positional_encoding (7,1,1,64), read out
of yonigozlan/EdgeTAM-hf — no graph carries them, because the memory
bank is assembled outside the graphs.
How it was made
- Graphs: jax-image-tools/edgetam-video-onnx (fp32,
gated against PyTorch
propagate_in_video_iteratorby its authors). - Precision: the decoder, memory attention and memory encoder converted with
onnxruntime.transformers.float16(I/O kept fp32, the Resize roi/scales constants kept fp32). The vision encoder stays fp32: its fp16 conversion returns an empty mask on every frame, and it was no faster. - Scripts and hashes:
training/person-track/in the kaairos repository.
Measured (Intel Iris Xe, Chrome, WebGPU, 4 clips × 80 frames at 10 fps)
| fp32 | this mix | |
|---|---|---|
| ms per tracked frame (wall clock) | ~1015 | ~565 |
Against the fp32 graphs, per-frame mask IoU: median 1.000 on three clips, 0.996 on the fourth, where after an occlusion the mix re-acquires the person about a second later. A single positive click tends to select a part of a person (a shirt, a badge); a box prompt (labels 2 and 3) selects the whole person.
Licence
Apache-2.0, as EdgeTAM.