Loading…
Loading…
To investigate the divergence characteristics of small models while minimizing risks associated with black-box distillation corpora—such as tendencies toward moralizing, refusal, or accusation—this study employs a large volume of native Chinese web text for pre-training. By utilizing shorter context windows, we avoid unexpected model outputs often associated with long contexts and significantly reduce training costs and time; this approach also allows us to explore methods for enhancing model performance—such as adjusting dimensionality and layer depth—within a short-context framework.
This model is capable of tasks such as short-text prediction, text completion, brainstorming, and creative copywriting. You need not worry about it suddenly lecturing or refusing you; as a pre-trained model—rather than a conversational one—it relies on data that has undergone extensive cleaning to minimize any tendency toward preachiness or refusal.
| parameters | hidden_size | hidden_layers | attention_heads | max_seq_len | Parameter volume |
|---|---|---|---|---|---|
| ChatBox-unrestricted-normal | 768 | 8 | 8 | 128 | 63.91M |
Input:什么是
Output:外贸独立站？
Input:什么是
Output:虚拟货币钱包？
Input:什么是
Output:软文推广？
Input:如何学习计算机
Output:编程？
Input:如何学习计算机
Output:专业文理学？
Input:我是一名
Output:教练，我非常担心球队有太多的空白和精彩。
Input:我是一名
Output:中超球队球员，我期望这场比赛能够让球队在最优秀的赛季里，有球迷期待。
Input:记者决定
Output:走进公安干警中心后，调查发现，原告人李某某，在本案中，案件未发生法律效力。
Input:记者决定
Output:走到马里乌干那，看到她的衣服也是非常不错的。
Input:厨师决定
Output:给你一个答案：“我们都在做菜，这就是我们常说的“一分钱一分货”。
Input:如何学习
Output:编程语言？
Input:如何学习
Output:如何学好《外语专业课》？
Input:你好，
Output:我是老师，我们学校的学生有没有可以参加大学的教科书呢？
Input:你好，
Output:我是做seo优化，不觉得这样，我就用了！
Based on the output, the model demonstrates a certain degree of divergent thinking and logic.
The dataset underwent two rounds of in-depth cleaning, ensuring its quality.
Pre-training:
https://huggingface.co/datasets/CASIA-LM/ChineseWebText2.0
If you are looking for a small-scale, portable conversational model without usage limits, you might want to try the ChatBox series.
ChatBox series:
First Generation:
https://huggingface.co/Zhaoming213/ChatBox-unrestricted
Second Generation:
https://huggingface.co/Zhaoming213/ChatBox2-unrestricted
Third Generation:
https://huggingface.co/Zhaoming213/ChatBox3-unrestricted(This model is currently experimental.)
Download all model files and place them in the model folder; then, run Predit.py to start using the model.
The consequences of using this model shall be borne by the user.