Retail-TW-Qwen3-4B-Base
Retail-TW-Qwen3-4B-Base is a continually pre-trained (CPT) checkpoint of Qwen/Qwen3-4B-Base, adapted to Taiwan (Traditional Chinese) retail and e-commerce knowledge.
Note: This is a base checkpoint. It is neither instruction-tuned nor safety-aligned, and it is intended as a foundation for downstream supervised fine-tuning (SFT).
Model Details
| Developed by | President Information Corp. (統一資訊, PIC) and Uni-President Information Philippines Corp. (UNI-PIPC) |
| Model type | Decoder-only causal language model (dense Transformer) |
| Base model | Qwen/Qwen3-4B-Base |
| Adaptation method | Continual pre-training (CPT) |
| Domain | Taiwan retail and e-commerce |
| Languages | Traditional Chinese (zh-TW, primary), Simplified Chinese (zh-CN), English (en) |
| Training framework | ms-swift |
| Training hardware | 4× NVIDIA H100 96 GB |
| License | Apache 2.0 |
Key Features
- 🎯 Retail-Domain Adaptation: Continual pre-training on 2.56B tokens, including 7.30M proprietary Taiwan product listings across 15 retail categories.
- 🇹🇼 Taiwan Retail Knowledge: Synthetic brand and contributor wikis, together with name variants (aliases, foreign-language names, Zhuyin and Pinyin spellings, and typographical errors), generated with NVIDIA NeMo Data Designer.
- 🌐 General-Language Retention: A replay mix of general-domain text (Wikipedia in zh-TW, zh-CN, and en, plus en↔zh translation pairs) preserves general and multilingual language modeling ability.
Training Data
Data Mixture
Token counts are computed with the base model tokenizer. A 0.1% split of each dataset is held out for validation.
Token Mix at a Glance
Share of CPT Tokens Product Listings53.7% · 1.38B Taiwan Retail Wiki26.3% · 674.4M Replay Data20.0% · 512.9M Replay Composition (% of Replay Tokens) zh-TW Wikipedia40.0% en↔zh Translation Pairs30.0% en Wikipedia20.0% zh-CN Wikipedia10.0% Product Listings1.38Btokens7,300,108 samples · 53.7% Taiwan Retail Wiki674.4Mtokens2,762,847 samples · 26.3% Replay Data512.9Mtokens2,774,330 samples · 20.0% Total2.56Btokens12,837,285 samples · 100% Taiwan retail wiki (synthetic, NVIDIA NeMo Data Designer): brand and contributor wikis, plus name variants (aliases, foreign-language names, Simplified/Traditional script, Zhuyin and Pinyin spellings, typos). Replay: general-domain text mixed into CPT to mitigate catastrophic forgetting. Sources: wikimedia/wikipedia (zh-TW, zh-CN, en) and en↔zh translation pairs from wmt/wmt19. zh-TW = Chinese converted to Traditional Chinese with OpenCC (translation pairs: Chinese side converted to zh-TW).Product Listings
The product listings are proprietary Taiwan retail data comprising title, brand, category, price, and description. The data was cleaned with NVIDIA NeMo Curator, which removed SEO filler text, emojis, contact details, hyperlinks, and policy terms, followed by exact and fuzzy deduplication.
Category distribution (15 categories, share of listings)
GeneralMerchandise14.67%Apparel, Shoes &Accessories12.72%Food & Beverage11.98%Beauty & PersonalCare10.6%Toys, VideoGames &Collectibles9.88%Sports,Outdoors &Fitness7.14%3C,Computers &Electronics6.36%Home, Kitchen& DailySupplies5.6%Books,Media &Stationery4.2%HomeAppliances4.19%Health &Medical4.16%PetSupplies4.09%Maternity &Baby3.93%Automotive&Motorcycle0.45%Tickets · 0.03%Citation
@misc{retailtwqwen34bbase,
title = {Retail-TW-Qwen3-4B-Base},
author = {President Information Corp. (PIC) and Uni-President Information Philippines Corp. (UNI-PIPC)},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/UNI-PIC/Retail-TW-Qwen3-4B-Base}
}
Contact
- Taiwan: pic.net.tw/contactUs
- Philippines: pic.net.ph/about/contactUs