Ara-JEV

Ara-JEV is a compact Arabic decision model fine-tuned from Qwen3.5-0.8B. Given a short state, a question, and optional candidate choices, it selects the best answer label.

This repository contains a standalone merged checkpoint: the trained LoRA weights have been merged into the base model. You do not need to load a separate adapter.

Model details

Evaluation

The held-out test set contains 10,419 examples. The results below use constrained candidate-label scoring: for each example, the evaluator scores the allowed labels and selects the highest-scoring one. This measures decision accuracy when the candidate labels are known; it is not free-form generation accuracy.

The primary evaluation prompt does not include dialect metadata.

Accuracy by question type

The yes/no subset is the main weakness. Consider this when choosing the model for arithmetic-heavy or binary decisions.

vLLM latency sample

A separate 100-example sample was benchmarked using vLLM 0.32.0 on an NVIDIA A40. Requests were sequential, used bfloat16 and constrained candidate labels, and ran locally after five warm-up requests.

These are local, single-request measurements; latency and throughput vary with hardware, prompt length, concurrency, and serving configuration. The sample's 93/100 accuracy is not the full held-out test score above.

Quick start

Transformers

Install the runtime dependencies (use Transformers 5.17 or newer for Qwen3.5):

pip install "transformers>=5.17" torch

Load Ara-JEV directly with standard Transformers. The predict function keeps prompt construction and constrained option scoring together:

import torch
from transformers import AutoModelForImageTextToText

model_id = "AhmedReda25300/Ara-JEV"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
).eval()

result = model.predict(
    state="الاجتماع يبدأ الساعة العاشرة صباحًا. دلوقتي الساعة تسعة.",
    question="هل بدأ الاجتماع؟",
    options=["نعم", "لا"],
)
print(result)  # لا

score = model.predict(
    state=(
        "بلاغ الدعم بيقول إن بيانات شخصية ظهرت لمستخدم مش صاحبها. "
        "التواصل كان عن طريق تذكرة من لوحة التحكم، والعميل طلب تحديث لما يظهر جديد."
    ),
    question="قيّم أولوية التعامل مع البلاغ.",
    options=[
        "منخفضة: يتابع في المسار المعتاد",
        "متوسطة: محتاجة متابعة قريبة",
        "عالية: تتصعّد بسرعة بسبب الأثر",
    ],
)
print(score)  # 2 (integer index of the selected option)

The prediction helper is built into the Transformers model class, so prompt construction and option scoring are handled internally. No dialect metadata is required. Arabic Yes/No options return the chosen option text; other option lists return the selected option's zero-based integer index. trust_remote_code is required to load Ara-JEV's custom predict method from the model repository.

vLLM

vllm serve AhmedReda25300/Ara-JEV \
  --dtype bfloat16 \
  --max-model-len 8192

The checkpoint was tested with vLLM 0.32.0. Use the chat-completions API and constrain the response to the labels allowed for each task when supported by your serving client.

Intended use and limitations

• Intended for Arabic decision and routing tasks similar to the evaluation data.
• Performance on dialects, domains, and tasks not represented in evaluation has not been established.
• For the reported evaluation, candidate labels were known and predictions were selected from those labels. Do not interpret that score as open-ended reasoning or general Arabic understanding.
• Test accuracy is an aggregate; results vary by question type, as shown above. Validate on representative data before deployment.
• The model is not a substitute for expert review in high-impact decisions.