加载中…
加载中…
PackedAvatar is a self-contained talking-head generation runtime that bundles the SadTalker-based avatar pipeline into a single .pt artifact.
It supports generating animated talking avatars from:
All core runtime assets are packaged inside PackedAvatar.pt.
Core model assets are bundled, but a few auxiliary helper weights may still be downloaded on the first run if they are not already cached locally.
PackedAvatar.pt contains:
This is a runtime artifact, not a training checkpoint.
PackedAvatar.pt — full bundled runtimePackedAvatar.py — loader + inference enginerequirements.txt — dependenciesREADME.md — usage guide.pt) for the main runtimerequirements.txtGPU is recommended; CPU is supported.
pip install -r requirements.txt
PackedAvatar.pt
from PackedAvatar import PackedAvatar
model = PackedAvatar("PackedAvatar.pt")
video = model.generate(
source_image="person.jpg",
driven_audio="speech.wav"
)
print(video)
Generate directly from a prebuilt identity:
video = model.generate(
avatar_id="Rebecca",
driven_audio="speech.wav"
)
No source image is required for this path.
If avatar_id is omitted, the runtime selects a default avatar from the packed bank.
The following avatars are prepacked in the bank.
| Style | Names |
|---|---|
| anime | Alison, Amber, Andrea, Angela, Christine, Cynthia, Heidi, Jennifer, Karla, Kristen, Laura, Nancy, Patricia, Rebecca, Sandra, Tara |
| cyber | Amanda, Brenda, Christina, Janet, Jill, Julie, Lisa, Mallory, Mandy, Martha, Melissa, Michelle, Regina |
| drawn | Alyssa, Danielle, Joan, Kaitlyn, Kimberly, Marie, Samantha, Veronica |
| paint | Alejandra, Barbara, Briana, Brittany, Emily, Jacqueline, Jodi, Mary, Rhonda, Savannah, Tammy, Victoria, Yolanda |
| real | Amy, Ann, Ashley, Colleen, Heather, Holly, Jordan, Kristin, Kristine, Mariah, Pamela, Sara, Sharon |
| Style | Names |
|---|---|
| anime | Brad, Brian, David, Gregory, John, Jose, Lawrence, Robert |
| cyber | Daniel, Hayden, James, Jeremy, Paul, Ryan, Sean |
| drawn | Bobby, George, Gregg, Kevin, Matthew, Ricky, Thomas |
| paint | Jacob, Justin, Michael, Nicholas, Steven, William, Zachary |
| real | Aaron, Andrew, Benjamin, Christopher, Derek, Frank, Jesse, Joseph |
There are 100 avatars total in the bank.
If no avatar is explicitly selected, the runtime resolves a default in this order:
defaults.default_avatar from the manifest, if present and validvideo = model.generate(
source_image="portrait.png",
driven_audio="speech.wav"
)
Pipeline:
image → face detection → crop → 3DMM extraction → animation
video = model.generate(
source_image="portrait.png",
driven_audio="speech.wav",
remove_background=True
)
Pipeline:
image → Bria RMBG → foreground → SadTalker → video
avatar_condition may be:
dict.pt / .pth file.mat fileWhen avatar_condition is provided, it supersedes source_image-driven conditioning.
video = model.generate(
avatar_condition="my_avatar_condition.pt",
driven_audio="speech.wav"
)
A valid avatar bundle can include fields such as:
coeff_3dmmmotion_3dmmfull_3dmmcrop_previewcrop_infovideo = model.generate(
source_image="portrait.png",
driven_audio="speech.wav",
motion_condition="motion.pt"
)
Supported motion inputs include:
motion_3dmmcoeff_3dmmfull_3dmm_seqfull_3dmmvideo = model.generate(
source_image="portrait.png",
driven_audio="speech.wav",
use_ref_video=True,
ref_video="reference.mp4",
ref_info="pose"
)
Supported ref_info values:
poseblinkpose+blinkallWhen ref_info="all", the runtime uses the reference video coefficients directly.
video = model.generate(
source_image="portrait.png",
driven_audio="speech.wav",
use_wav2lip=True,
wav2lip_repo="/path/to/Wav2Lip"
)
Post-processes the SadTalker output for improved lip sync.
The bundled checkpoint checkpoints/wav2lip_gan.pth is used automatically.
If no runnable Wav2Lip inference code is found, the runtime falls back to the SadTalker video instead of crashing.
Generate with silent audio instead of an input file:
video = model.generate(
avatar_id="Aaron",
use_idle_mode=True,
length_of_audio=4
)
Reduces head movement:
still_mode=True
exp_scale=1.2
facerender="facevid2vid"
Automatically chooses:
Override:
PackedAvatar(device="cuda")
from PackedAvatar import PackedAvatar
model = PackedAvatar(
packed_pt_path="PackedAvatar.pt",
device="cuda",
cache_dir="./cache"
)
video = model.generate(
source_image="speaker.png",
driven_audio="speech.wav",
remove_background=True,
use_wav2lip=True,
size=512,
exp_scale=1.2,
pose_style=1,
still_mode=False
)
print(video)
The runtime exposes an embedding extraction helper for image or video conditioning:
bundle = model.extract_embeddings(
input_path="test_image.png",
crop_or_resize="crop",
pic_size=256
)
Camel-case alias:
bundle = model.ExtractEmbeddings("test_image.png")
The returned bundle can be saved and reused as avatar_condition or motion_condition.
python PackedAvatar.py \
--source-image person.jpg \
--driven-audio speech.wav
python PackedAvatar.py \
--avatar-id Rebecca \
--driven-audio speech.wav
python PackedAvatar.py \
--source-image portrait.png \
--driven-audio speech.wav \
--remove-background
python PackedAvatar.py \
--source-image portrait.png \
--driven-audio speech.wav \
--use-wav2lip \
--wav2lip-repo /path/to/Wav2Lip
python PackedAvatar.py \
--source-image portrait.png \
--driven-audio speech.wav \
--use-ref-video \
--ref-video reference.mp4 \
--ref-info pose+blink
python PackedAvatar.py \
--avatar-id Aaron \
--use-idle-mode \
--length-of-audio 5
python PackedAvatar.py \
--avatar-condition avatar_condition.pt \
--driven-audio speech.wav
python PackedAvatar.py \
--motion-condition motion_condition.pt \
--driven-audio speech.wav
PackedAvatar runs a full multimodal pipeline.
.ptPriority:
avatar_condition
→ source_image-driven SadTalker path
→ avatar_id / default AvatarBank resolution
If avatar_condition is provided, it supersedes source_image conditioning.
PackedAvatar exposes the packed AvatarBank directly, allowing you to browse, search, inspect, and manage avatars at runtime.
The AvatarBank is loaded automatically when the model is initialized.
from PackedAvatar import PackedAvatar
model = PackedAvatar("PackedAvatar.pt")
avatars = model.list_avatars()
print(avatars)
Returns:
[
"Aaron",
"Rebecca",
"Amy",
...
]
Perform a fuzzy search against avatar IDs.
matches = model.fuzzy_search_avatar("rebeca")
print(matches)
Example:
["Rebecca"]
Search the bank using metadata filters.
female_anime = model.query_avatars(
gender="female",
style="anime"
)
print(female_anime)
Example:
[
"Alison",
"Amber",
"Andrea",
...
]
Fuzzy matching is supported:
model.query_avatars(
gender="femal",
style="anim"
)
metadata = model.get_avatar_metadata("Rebecca")
print(metadata)
Example:
{
"gender": "female",
"style": "anime"
}
Access the full conditioning bundle used internally by PackedAvatar.
avatar = model.get_avatar("Rebecca")
Returned fields may include:
{
"avatar_id": "Rebecca",
"gender": "female",
"style": "anime",
"coeff_3dmm": ...,
"motion_3dmm": ...,
"full_3dmm": ...,
"crop_info": ...,
"crop_preview": ...
}
This bundle can be passed directly as an avatar_condition.
video = model.generate(
avatar_condition=avatar,
driven_audio="speech.wav"
)
Preview images stored in the AvatarBank can be accessed directly.
preview = model.get_avatar_preview("Rebecca")
preview.show()
Returns a PIL image.
Remove an avatar from the in-memory runtime.
model.delete_avatar("Rebecca")
This only affects the current runtime session.
To persist changes, save the bank.
model.save_avatar_bank("ModifiedAvatarBank.pt")
This writes the current AvatarBank state to disk.
exists = model.avatar_bank.exists("Rebecca")
print(exists)
Returns:
True
The underlying AvatarBank runtime is also exposed.
bank = model.avatar_bank
Available methods include:
bank.available_ids()
bank.list_avatars()
bank.exists(...)
bank.query(...)
bank.fuzzy_search_id(...)
bank.get_avatar(...)
bank.get_metadata(...)
bank.get_preview(...)
bank.delete_avatar(...)
bank.save(...)
This provides full programmatic access to the packed avatar database.
Compared to a standard SadTalker setup:
.pt deployment artifactBuilt on top of: