Cosplay character classifier

Tells which character a cosplayer is dressed as, from a photo: 396 characters from anime, games, comics and films, some with their costume variants (2B (Christmas), Marin Kitagawa (Rizu Kyun)).

It recognises the costume, not the person: every class is a character, never a cosplayer.

Use it

import timm, torch
from PIL import Image, ImageOps

model = timm.create_model("hf-hub:GwenRB/cosplay-characters", pretrained=True).eval()
transform = timm.data.create_transform(**timm.data.resolve_data_config(model=model))
names = model.pretrained_cfg["label_names"]

with Image.open("photo.jpg") as im:
    x = transform(ImageOps.exif_transpose(im).convert("RGB"))[None]
with torch.no_grad():
    probs = model(x).softmax(1)[0]
for p, i in zip(*probs.topk(3)):
    print(f"{names[i]:<30} {p:.1%}")

The transform squashes the whole photo into 224×224 instead of cropping its centre, as in training. A centre crop also runs, but is less accurate.

It always answers with one of its 396 characters, even for a photo of someone out of costume or dressed as a character it doesn't know. Treat answers under about 50% as "not sure".

Training

• Base: vit_base_patch16_siglip_224.v2_webli (SigLIP 2 ViT-B/16), fully fine-tuned.
• Data: photos of cosplayers in costume, from their public posts, sorted by character by hand and with the help of earlier versions of this model. A character needs a few photos on each side of the split to be included.
• Validation: top-1 98.1%, macro-F1 97.0%.

The validation photos are split from the same collection by file, so the same cosplayers, and often the same photo shoots, are on both sides. Expect lower accuracy on cosplayers the model has never seen.

Characters

2B, 2B (Christmas), A2, Abella, Ada Wong, Ade, Aerith Gainsborough, Ahri, Ahsoka Tano, Ai Hoshino, Aira Shiratori, Airi Akizuki, Akali, Albedo, Alice (Nikke), Alice Madness, Alya, Amber, Android 18, Anis, Ankha, Ann Takamaki, April ONeil, Aqua, Arlecchino, Asa Mitaka, Asari, Ashe, Ashley, Ashley Graham, Asuka Langley, Asuma Toki, Atom Eve, Azula, Bayonetta, Bela Dimitrescu, Bene Gesserit, Black Cat, Black Widow, Blair, Blanc, Blonde Blazer, Boa Hancock, Bomb Devil, Booette, Bowsette, Bulma, Burnice, C.C, Caitlyn, Camellya, Camie Utsushimi, Cammy, Castorice, Catwoman, Chel, Cheshire, Chie Sayama, Chii, Chocola, Chucky, Chun-Li, Cindy Aurum, Cipher, Ciri, Citlali, Columbina, Cruella, Cyrene, D.Va, Daenerys Targaryen, Daki, Dante, Daphne, Dark Magician (Yu Gi Oh), Dehya, Demogorgon, Diane, Dodomeki, Elastigirl, Elizabeth Comstock, Elizabeth Liones, Ellen Joe, Elsa, Emilia, Emma Frost, Enid Sinclair, Ereshkigal, Esdeath, Esil Radiru, Eula, Eve (Stellar Blade), Evelynn (League of Legends), Evernight, Falin, Fem, Fern, Fiona, Fire keeper, Firecracker (The Boys), Firefly, Fischl, Fluorite, Formidable, Frieren, Frieren (Goth), Fubuki, Furina, Futaba Sakura, Ganyu, Gawr Gura, Gemma, Gold Ship, Grace Ashcroft, Gwen Stacy, Gwenpool, Harley Quinn, Hatsune Miku, Hela, Hermione Granger, Herta, Hestia, Hikari Tachibana, Hilda, Himeno, Hinata Hyuga, Hindenburg, Hitagi Senjougahara, Hitori Gotou, Holo, Homelander, Hornet, Ichinose Asuna, Illidan, Imperial Commissar, Inori, Inosuke, Inquisitor, Inugami Korone, Invisigal, Ishtar, Izutsumi, Jane Doe, Jasmine, Jason Voorhees, Jeanette Voerman, Jennifer Check, Jessica Rabbit, Jessie (Pokemon), Jill Valentine, Jinx, Jolyne Cujoh, Ju Fufu, Jubilee, Judy Hopps, Juri, Kafka, Kaine, Kangel, Kaoruko Waguri, Kashino, Katara, Katarina, Katherine (Kingdom Come), Keqing, Kida Nedakh, Kikuri Hiroi, Killjoy, Kim Possible, Kinako, Kirara Hoshi, Kiriko, Kitana, Kizuna Ai, Kobeni Higashiyama, Kokomi Sangonomiya, Komi, Kratos, Krul Tepes, Kurisu Makise, Kurumi Tokisaki, La Signora, Lady (Devil May Cry), Lady Dimitrescu, Lara Croft, Lauma, Le Malin, Leeloo, Leia (Slave), Lisa, Little Mermaid, Lola Bunny, Lucy, Lucy Maclean, Luigi, Lumine, Luna Snow, Lune, Lynette, Maelle, Magia Baiser, Magik, Mahito, Mai Sakurajima, Mai Shiranui, Maki Zenin, Makima, Malenia Blade of Miquella, Malevola, Mantis Mandy, Mao Mao, Marciana, Marcille, Maria (Silent Hill), Marie Rose, Marika, Marin Kitagawa, Marin Kitagawa (Rizu Kyun), Marin Kitagawa (Shizuku Kuroe), Mary Jane, Mash Kyrielight, Mavis, Mavuika, McDonald, Megumin, Mei Mei, Meiko Shiraki, Melina, Melinoe, Mera, Mihara, Mikasa Ackerman, Mileena, Milena, Misa Amane, Misato Katsuragi, Miss Fortune, Misty, Mita, Mitsuri Kanroji, Miyabi, Mizuki, Modeus, Momo Ayase, Mona, Morrigan (Darkstalkers), Morticia Addams, Motoko Kusanagi, Nagatoro, Nami, Naruto, Navia, Nazuna Nanakusa, Neon, Nero, Nezuko, Nico Robin, Nicole Demara, Nicole Reeyn, Nidalee, Nightsister Merrin, Nilou, Nintendo Switch, Nobara, Nurse (Silent Hill), Ochaco Uraraka, Orihime, Oshino Shinobu, Owari, Panty, Peach, Perlica, Petra Parker, Poison Ivy, Pomni, Powder, Power, Princess Peach, Priscilla, Psylocke, Purah, Queen Marika, Ragatha, Raiden Shogun, Ram, Ramiris, Ramona Flowers, Rangiku Matsumoto, Ranni, Raphtalia, Rapi, Raven, Rebecca (Cyberpunk), Red Sonja, Rem, Reze, Rias Gremory, Rin Tohsaka, Ririsa Amano, Rogue, Rosa (Pokemon), Rosalina, Rosaria, Roxanne, Roxy Migurdia, Rukkhadevata, Ryuko Matoi, Saber, Sakura Haruno, Sameko, Samus, Saskia, Satsuki, Scarlet Witch, Scathach, Seiko Ayase, Seraphine, Serie, Shadowheart, Shani, Shenhe, Shimakaze, Shuten-Douji, Silence Suzuka, Silver Wolf, Snow White, Soda, Sombra, Sonico, Sora Kasugano, Sparkle, Sparxie, Spiderman (Spidergirl), Squirrel girl, St Louis, Stocking, Sucrose, Sue Storm, Sukuna, Sunna, Supergirl, Tae Takemi, Taiga Aisaka, Takao, Tatsumaki, Tifa, Tifa (Nurse), Tinkerbell, Toga Himiko, Topaz, Toph, Toujou Koneko, Trigger, Triss Merigold, Tsunade, Tsuyu Asui (Froppy), Twilek, Ubel, Umbreon, Vampirella, Vanilla, Vaporeon, Varesa, Velma, Vermeil, Vex, Vi, Viper (Nikke), Vladilena Milizé, Wednesday Addams, White Fox, Widowmaker, Wonder Woman, Woody, Xayah, Xena, Yae Miko, Yamada Tayama, Yamato, Yaniko, Ye Shunguang, Yelan, Yennefer, Yoko Littner, Yor Forger, Yu Takeyama, Yuki Tsukumo, Yumeko Jabami, Yuno Gasai, Yuzuriha, Yvonne, Zani, Zelda, Zero Two, Zhu Yuan, darth talon