Sign up

JonathanColetti/LiveWan

🤗 On Hugging Facetext-to-videoapache-2.037 GBother✓ Checksums witnessedupdated today
Magnet

LiveWan: streaming text 2 video, 3000 steps

Unofficial community project. Not affiliated with, endorsed by, or produced
by Alibaba Group or the Wan-Video team. Built on their Apache-2.0
Wan2.1 release.

A 1.3 B student distilled from a Wan2.1-14B teacher that generates video

continuously rather than as a fixed clip: 750 ms of 640x368 video per block,

generated in 428 ms on one A100 80GB, extended block by block for as long as you

let it run. The text conditioning can be changed mid-stream without clearing the

K/V cache, so the scene continues instead of cutting.

Code, setup and documentation: JonathanColetti/LiveWan. This repository holds

weights and data only. It is not usable on its own.

Not audio

The model is text-to-video and was never trained to articulate speech. There is no

audio, no lip sync and no speech anywhere in this project. Text steers the scene.

A face that appears to talk is generating plausible mouth motion, not saying your

sentence.

Files

To run the model — 6.0 GB. This is what setup.sh fetches.

| path | size | what |

|---|---|---|

| checkpoints/t14b_b64/milestones/step002250_noema.pt | 5.3 GB | the model — 825 tensors, bf16, no EMA copy |

| checkpoints/t14b_b64/history.json | 100 KB | the full training history of the run |

| data/prompts.pt | 388 MB | umt5-xxl embeddings for 96 prompts, indices 0–95 |

| out/world_p{0,44,60,82}.pt | 4 × 59 MB | the four cached evaluation worlds — skip base-model generation entirely |

| samples/ | 16 MB | reference clips and analysis filmstrips from those worlds |

To continue the run Not needed for inference. do not

download these unless you intend to train.

| path | size | what |

|---|---|---|

| checkpoints/t14b_b64/latest.pt | 10.6 GB | step 3000, carries its EMA copy. The comparison arm, and the weights a resume starts from |

| checkpoints/t14b_b64/opt_rank00{0..7}.pt | 8 × 2.2 GB | FSDP-sharded AdamW state at step 3000, one shard per rank |

Nothing here duplicates a public download. The Wan2.1 base model, its VAE and the

umt5-xxl encoder are not mirrored in this repository. setup.sh pulls

Wan-AI/Wan2.1-T2V-1.3B (17 GB)

from its own repo, and the 14B teacher is only needed if you are distilling.

The layout mirrors the code repository, so setup.sh downloads straight into place.

To take only part of it:

hf download JonathanColetti/LiveWan --local-dir . \
  --exclude "checkpoints/t14b_b64/latest.pt" "checkpoints/t14b_b64/opt_rank*"

Use

git clone https://github.com/JonathanColetti/LiveWan && cd LiveWan
pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128
./setup.sh                       # pulls this repo, checksum-verified

python scripts/demo.py \
  --weights checkpoints/t14b_b64/milestones/step002250_noema.pt \
  --world-cache out/world_p60.pt --prompt-idx 60 \
  --latent-norm 1.0 --block 3 --steps 2 --window 6 --units 60 --fps 16 \
  --out out/demo_w60

Needs ~10 GiB of VRAM. scripts/live_demo.py serves the same model as a steerable

browser stream.

--prompt-idx must match the world: world_p60.pt goes with --prompt-idx 60.

Indices run 0–95; anything above fails.

Training

3000 iterations of SF-DMD distillation from a Wan2.1-T2V-14B teacher into a

Wan2.1-T2V-1.3B student, effective batch 64 (8×H200, accum 8, FSDP), 41.6 hours

at 66.5 s/it, zero interventions. Losses do not decrease in this trainer and should

not (the critic is retrained every step, so the generator holds position against a

strengthening opponent).

Continuing the run

The optimizer shards make extending the run a resume rather than a restart.

Put latest.pt, all eight opt_rank*.pt and history.json in the same run

directory and:

ITERS=4000 ./run_b64_resume.sh

Four things decide whether this works:

  • Exactly 8 ranks. These are per-rank FSDP shards, not a consolidated state

dict. World size 8 is what wrote them and world size 8 is what can read them;

fewer ranks means larger shards, not smaller, so there is no repacking

shortcut. run_b64_resume.sh checks for all eight up front and refuses rather

than half-starting.

  • You resume from step 3000, not from the released step 2250. Optimizer state

and the EMA copy exist only for step 3000 — 2250 was archived _noema and its

EMA is gone. The released checkpoint is the better model; latest.pt is the

only resumable one.

  • --resume overwrites the checkpoint it resumed from. It reads

OUT/latest.pt and writes that same path at the first save.

run_b64_resume.sh preserves it as step003000.pt first — if you drive the

trainer directly instead, copy it aside yourself.

  • ~30 GiB must be free transiently at each save, because the shards are

written as .tmp and renamed, so old and new coexist. Prune before the save,

not after; a post-save janitor cannot help, since the save is what runs out of

space.

The resume path is verified rather than assumed: killing a run at step 12 brought

AdamW's internal counters back as 16/18 rather than reset, and the real 750→3000

resume held loss_gen, loss_critic, dmd_grad_norm and gn_gen within one

standard deviation of the 60-point pre-crash reference. A subtly broken restore

shows up as a shifted mean or changed variance; neither appeared.

Verified

  • all 825 tensors load, zero non-finite values, weight norms 0.018–115.4
  • three correctness gates pass with their deliberately broken controls failing, both

before and after training

  • 24 streaming cells across every arm and world met the real-time budget
  • same box, same seed → byte-identical output

Limits

  • Quality holds for roughly a minute. By 160 s sharpness has fallen to ~73% of

the world's.

  • Maximum 1024 latent frames (4.3 minutes at block 3), set by WanModel.freqs.
  • The pixel check has not been done. Checkpoint selection rests on proxy metrics,

and in this project nine automatic measurements have pointed the wrong way — every

one caught by looking at pixels at 1:1.

  • World 82 is unexplained: every trained checkpoint drifts on it while the

untrained baseline sits at −0.001.

  • The batch-64 premise is unsettled — confounded with having simply trained

longer.

Licence

Apache-2.0. Derived from Wan2.1

(Apache-2.0, Alibaba Group).