Luna 3.1 Model Card

Model Information

• Model creator: AobanZ

Description

Luna 3.1 is our next generation text model designed for heavy usage and lengthy conversations. These models have added Llama.cpp support as well as faster response time. Luna has an 8k context window, which supports long chats and roleplays. Luna 3.1 retains the Lightning architecture, with the only changes being upgraded RMSNorm and RoPE. Luna 3.1 generates at a relatively fast speed of 130 tokens per second.

• Developed by: AobanZ
• Model type: Transformer
• Language(s) (NLP): English
• License: MIT

[!IMPORTANT] If you try installing llama.cpp traditionally, the program will exit with unknown model architecture: 'lightning'. A patch of llama.cpp has been released called Lightning.cpp that supports the Luna architecture. Lightning.cpp

Model Sources

• Repository: https://huggingface.co/Aobangaming/luna-2-70m

Highlights

• Faster Response Time With Llama.cpp, Luna generates tokens faster and more efficiently.
• Increased Intelligence Luna was sized up from 100m to 230m, which leads to around 100M transformer parameters.
• Better Coherency Luna was trained on the capybara dataset, which contains patterns built for chat models.

Usage

Luna can be used directly for text generation, experimentation, and conversational interactions. Users can provide text prompts and generate responses using the model's built-in language modeling capabilities. Direct use is primarily intended for casual conversation, research and experimentation. Outputs may be incomplete, inaccurate, repetitive, or unrelated to the input, and should be evaluated before being used for other purposes.

Recommendations

We recommend users of Luna to fine-tune the model on new text, and add necessary guardrails and precautions to prevent misuse.

Starter

```python
from transformers import pipeline

question = "Replace with your question"
generator = pipeline("text-generation", model="None", device_map="auto")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Framework versions

• TRL: 1.17.1
• Transformers: 5.22.0
• Pytorch: 2.9.1+cu126
• Datasets: 5.1.0
• Tokenizers: 0.23.2
• Python: 3.14.0

Training Details

Training Data

Luna was trained on a subset of the OASST dataset.

Training Procedure

Luna 3 was trained on an RTX 3050 GPU, using FlashAttention/SDPA and MHA. The model was trained on Trl's capybara dataset.

Training Hyperparameters

Hardware & Infrastructure

• Hardware Type: RTX 3050 6GB
• Hours used: 0.5
• Cloud Provider: AobanLabs
• Compute Region: Asia

Technical Specifications

Model Architecture and Objective

Luna uses a 12-layer causal Transformer with 768-dimensional hidden states and 12 attention heads, with RMSNorm and RoPE. Luna uses a 8126 context window and Multi-Head attention.

Citations

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}