Roshan3-VL-4B-Thinking 🌟

A Pashto-focused fine-tuned language model based on Qwen3-VL-4B-Thinking.

Roshan3-VL-4B-Thinking (روښان) is an independently developed fine-tuned model focused on understanding and generating natural Pashto text.

The project aims to contribute to the development of Pashto-language artificial intelligence by improving Pashto comprehension, conversational ability, reasoning, and language generation.

The model is developed and maintained by nassimjp.

Model Information

Project Goals

Roshan is designed to advance Pashto-language AI capabilities.

The primary objectives include:

• Understanding written Pashto.
• Generating natural and meaningful Pashto responses.
• Improving Pashto conversational interactions.
• Understanding Pashto questions and instructions.
• Supporting Pashto reasoning and analytical tasks.
• Improving Pashto vocabulary, grammar, and sentence construction.
• Supporting multilingual communication.

Current development focus: Pashto text understanding and generation.

Although the underlying architecture is a vision-language model, the current fine-tuning project has focused on text data. Image-understanding performance has not yet been independently evaluated.

Training Information

Roshan3-VL-4B-Thinking was fine-tuned using QLoRA, with Unsloth and Hugging Face training tools.

Training Configuration

Note: The training run stopped at 42% of its planned steps. The exported model represents the available checkpoint, not a fully completed training run.

Model Export

The trained checkpoint was successfully exported and uploaded to Hugging Face Hub.

Original Fine-tuning Checkpoint

nassimjp/Qwen3-VL-4B-Thinking-Pashto-Zi_1793282777

Base Model

nassimjp/Qwen3-VL-4B-Thinking-Pashto-Zi

Installation

Install the required libraries:

pip install -U transformers accelerate torch

For the latest Qwen3-VL architecture support, use a compatible Transformers release.

Usage

The following is a basic Transformers loading example for the Qwen3-VL architecture:

import torch
from transformers import (
    Qwen3VLForConditionalGeneration,
    AutoProcessor
)

model_id = "nassimjp/Roshan3-VL-4B-Thinking"

model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

processor = AutoProcessor.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "سلام روښانه! ته څوک یې؟"
            }
        ]
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True
).to(model.device)

generated_ids = model.generate(
    **inputs,
    max_new_tokens=256
)

generated_ids_trimmed = [
    output_ids[len(input_ids):]
    for input_ids, output_ids in zip(
        inputs.input_ids,
        generated_ids
    )
]

response = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True
)[0]

print(response)

Note: This example assumes the exported repository includes the required model configuration, tokenizer, and processor files.

Model Limitations

• Pashto capabilities require further independent evaluation.
• Training was stopped before reaching the planned 65,207 steps.
• The reported training loss is not an independent measure of model quality.
• The current fine-tuning dataset focused on text rather than image-language training.
• Responses may contain factual errors, grammatical mistakes, or reasoning limitations.
• Image understanding and multilingual performance should be evaluated separately.

Acknowledgements

Special thanks to the developers and communities behind:

• Qwen
• Unsloth
• Hugging Face
• Hugging Face TRL
• PyTorch

This project builds upon the open-source Qwen3-VL architecture and its associated tooling.

About Roshan

روښان (Roshan) means light or illumination.

The purpose of this project is to help bring Pashto language, knowledge, and communication into the world of modern artificial intelligence.

Roshan is an independent fine-tuning project developed by nassimjp.

Built for Pashto. Developed with AI.

🌟 Roshan3-VL-4B-Thinking