XiaomiMiMo/MiMo-7B-MTPs

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Unlocking the Reasoning Potential of Language Model
From Pretraining to Posttraining
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

| 🤗 HuggingFace | 🤖️ ModelScope | 📔 Technical Report |

This model repository is licensed under the MIT License.

I. Pretrained MTPs of MiMo-7B

This model repository contains the pretrained MTP weights of MiMo-7B (model.mtp_layers.1 and model.mtp_layers.2)

Currently, MiMo-7B model each has 1 MTP layer (model.mtp_layers.0). Users may load the weights of pretrained MTPs for potential rollout speedup (please refer to Power Up Speculative Decoding In Reinforcement Learning).

[!IMPORTANT] We tuned 1 MTP layer in SFT and freeze it in RL, and we HAVE NOT test the performance of posttrained models with 2 more pretrained MTP layers.

II. Contact

Please contact us at mimo@xiaomi.com or open an issue if you have any questions.

Use Restrictions

You are solely responsible for your use of XiaomiMiMo/MiMo-7B-MTPs and any outputs, actions, or consequences arising therefrom, and you agree not to use the XiaomiMiMo/MiMo-7B-MTPs or any derivatives thereof:

For any unlawful, fraudulent, or malicious purpose, or in any manner that violates any applicable laws or regulations；
To infringe upon the intellectual property rights, privacy rights, publicity rights, or other lawful rights or interests of any third party;
To exploit, harm, harass, defame, unlawfully discriminate against, or otherwise adversely affect any individual or group, including minors or vulnerable persons;
For any military purpose or application;
To engage in or facilitate malicious cyber activities, including but not limited to malware, ransomware, phishing, credential theft, or unauthorized access, or any activity intended to compromise or unlawfully control computer systems, networks, or encryption systems;
To access, collect, process, disclose, or disseminate personal data or other protected information without appropriate authorization or lawful basis; or to access, interact with, scrape, or automate any third-party platform or service in violation of any applicable laws, policies, or authorization requirements;
To use the XiaomiMiMo/MiMo-7B-MTPs in a manner that autonomously executes high-risk actions without appropriate human oversight or authorization.

This notice is an informational reminder only and does not amend the applicable model licence or impose additional licence conditions. The permissions granted under that licence remain unaffected. Nothing in this notice restricts uses permitted by applicable law, including applicable exceptions and limitations.

Usage enquiries and complaints

mimo-legal@xiaomi.com. For information on submitting a complaint, please visit the https://mimo.xiaomi.com.

加载中…

I. Pretrained MTPs of MiMo-7B

This model repository contains the pretrained MTP weights of MiMo-7B (model.mtp_layers.1 and model.mtp_layers.2)

[!IMPORTANT] We tuned 1 MTP layer in SFT and freeze it in RL, and we HAVE NOT test the performance of posttrained models with 2 more pretrained MTP layers.

Use Restrictions

For any unlawful, fraudulent, or malicious purpose, or in any manner that violates any applicable laws or regulations；

To infringe upon the intellectual property rights, privacy rights, publicity rights, or other lawful rights or interests of any third party;

To exploit, harm, harass, defame, unlawfully discriminate against, or otherwise adversely affect any individual or group, including minors or vulnerable persons;

For any military purpose or application;

To engage in or facilitate malicious cyber activities, including but not limited to malware, ransomware, phishing, credential theft, or unauthorized access, or any activity intended to compromise or unlawfully control computer systems, networks, or encryption systems;

To access, collect, process, disclose, or disseminate personal data or other protected information without appropriate authorization or lawful basis; or to access, interact with, scrape, or automate any third-party platform or service in violation of any applicable laws, policies, or authorization requirements;

To use the XiaomiMiMo/MiMo-7B-MTPs in a manner that autonomously executes high-risk actions without appropriate human oversight or authorization.

Usage enquiries and complaints