zzzekryo/Oprius-3.0-1.1B

Oprius 3.0 · 1.1B

A conversational language model based on TinyLlama, fine-tuned to balance concise replies, detailed explanations, and general assistance.

Oprius 3.0 is a fine-tuned version of TinyLlama 1.1B Chat. It aims to provide a natural conversational experience while adapting the length of its responses to the user's needs.

This release incorporates the trained LoRA adapter into the base model weights, allowing Oprius to be loaded as a standalone Transformers model without attaching the adapter separately.

Features

• Balanced response lengths: Short answers for everyday conversations, medium-length answers for general help, and more elaborate responses for complex concepts that benefit from detail.
• Conversational personality: Designed to communicate casually and naturally instead of sounding unnecessarily formal or robotic.
• General assistance: Fine-tuned examples cover explanations, everyday questions, and helpful responses.
• Web search behavior examples: Training data includes examples involving web search requests and search-related responses, teaching the model patterns for handling these interactions.
• Standalone model: LoRA updates have been merged into the model weights for straightforward loading and deployment.

Response style

Oprius aims to adapt its response length to the context rather than answering everything with the same amount of detail.

The goal is to keep simple conversations simple while providing more depth when a topic calls for it.

Web search behavior

Oprius 3.1 includes training examples related to web search interactions.

These examples help the model learn how search requests and search-oriented responses may be structured. However, the model does not gain live internet access from fine-tuning alone.

To retrieve current information, Oprius must be connected to an external search tool or a compatible application that executes searches and provides the results to the model.

Without such an integration, its responses about current events and online information may be outdated or inaccurate.

Model details

• Base model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
• Model family: TinyLlama
• Trained examples 893
• Parameter count: Approximately 1.1 billion
• Fine-tuning method: LoRA fine-tuning, followed by weight merging
• Model format: Hugging Face Transformers
• Primary language: English
• Intended use: Conversational assistance, general questions, and explanations

Usage

Install the required libraries:

pip install torch transformers

Load Oprius with Transformers:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "zzzekryo/Oprius-3.1-1.1B"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=(
        torch.float16 if torch.cuda.is_available()
        else torch.float32
    ),
    device_map="auto",
)

messages = [
    {"role": "user", "content": "yo, what is a GPU?"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        inputs,
        max_new_tokens=150,
        do_sample=True,
        temperature=0.7,
        top_p=0.95,
        pad_token_id=tokenizer.eos_token_id,
    )

response = outputs[0][inputs.shape[-1]:]

print(tokenizer.decode(response, skip_special_tokens=True))

For reproducible outputs, use deterministic generation with do_sample=False. Sampling settings can affect response length, wording, and consistency.

Limitations

• Oprius is a relatively small 1.1B-parameter model and may struggle with complex reasoning or specialized tasks.
• It can generate incorrect information, so important claims should be verified.
• Fine-tuned web search examples do not replace an actual search integration.
• Response length and conversational behavior are learned tendencies, not guaranteed rules.
• Performance depends on the hardware, Transformers version, and inference configuration.

License & Notice

This project is licensed under the Apache License 2.0.

In accordance with the Apache 2.0 license requirements, please note:

• Modifications: Oprius 3.0 is a modified derivative work of the original TinyLlama model. It incorporates custom fine-tuning via LoRA adapters which have been permanently merged into the base model weights.
• Distribution: Any distribution of this model or its derivatives must retain the original copyright notices, the Apache 2.0 license text, and prominent notice of these modifications.

Acknowledgements

Oprius 3.1 is built upon the TinyLlama project and its open-source ecosystem.

• Base Model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
• Original Authors: Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu.

We express our deepest gratitude to the creators and contributors of TinyLlama for their outstanding work and dedication to democratizing open-source language models.

Test Notes

During Model's testing, There has been issues that can possibly happen during usage:

• On web searches, the model has been to noticeably repeating the system prompt in messages, when it shouldn't.

• The model can generate short responses even for topics where it has been trained to elaborate.