Darja-GPT-50M is an open-source, 54.8M parameter causal language model trained from scratch exclusively on Algerian Darija (الدارجة الجزائرية / Dziri).
Unlike previous efforts that retrofitted bidirectional masked encoders (e.g. DziriBERT) with sequence-to-sequence decoders, Darja-GPT is built natively as a modern autoregressive decoder with dedicated per-domain Low-Rank Adapters (DomainLoRALinear) and support for Classifier-Free Guidance (CFG) at inference time. This architecture solves the register-collapse and domain-leakage issues common in low-resource dialect modeling, enabling reliable steering across culinary, sports, technical, podcast, narrative, and colloquial registers.
In earlier iterations (v3 & v4), domain steering was attempted via FiLM (Feature-wise Linear Modulation) — projecting domain embeddings to scale and shift intermediate activations. However, architectural diagnostics revealed that FiLM failed to learn distinct domain representations:
DomainLoRALinear) on attention output projections (o_proj) and MLP down-projections (down_proj) produced weight delta distances of 118%–173% of the mean. Each domain maintains independent low-rank updates (rank r = 16, α = 16.0), eliminating representation compromise.To our knowledge, this is the first deployment of Classifier-Free Guidance for dialectal autoregressive language generation:
<NULL>, d = 6) with probability p = 0.10.z_cond and unconditional logits z_NULL are extrapolated:z_guided = z_NULL + s · (z_cond - z_NULL)
s ∈ [1.5, 2.5] completely suppresses out-of-domain leakages and sharpens domain-specific terminology without degrading grammatical flow.Trained from scratch on conversational Algerian Darja transcripts, our 24,000-vocabulary BPE tokenizer achieves 1.431 tokens/word on raw held-out text, delivering a 39.2% fertility reduction (95% CI [38.4%, 40.0%]) compared to foundation models like Qwen2.5 / Qwen3 (2.354 tokens/word). It matches regional encoders (DarijaBERT 1.427, DziriBERT 1.429) while using 2x to 3.3x fewer vocabulary parameters.
The entire project operates across four synchronized stages: data curation, morphological tokenization, distributed pre-training with dual-dropout regularization, and dual-stream CFG inference.
flowchart TD
subgraph Data["1. Data Curation & Preprocessing"]
C["Algerian Darja Corpus (11,151 documents)"] --> M["Domain Consolidation (6 Domains + NULL)"]
M --> CH["Document-Level Chunking (1,024-token blocks)"]
CH --> REG["Dual-Dropout Regularization (Tag: 20% | CFG: 10%)"]
end
subgraph Tokenizer["2. Tokenizer Pipeline"]
T_CORPUS["Raw Conversational Text"] --> BPE["Byte-Level BPE (24,000 Vocab)"]
BPE --> FER["Fertility: 1.431 Tokens/Word (-39.2% vs Qwen2.5)"]
end
subgraph Training["3. Pre-Training (Single T4 GPU)"]
PADDED["52,038 Packed Examples"] --> TRAIN["AdamW + Cosine Schedule (Peak LR 3e-4)"]
TRAIN --> CKPT["Best Checkpoint (Step 3,000 / Eval Loss 5.978)"]
end
subgraph Inference["4. CFG Inference Engine"]
PROMPT["Prompt with Domain Tag d"] --> DUAL["Dual Stream: z_cond(d) and z_uncond(NULL)"]
DUAL --> CFG["CFG Logit Extrapolation (Scale s in [1.5, 2.5])"]
CFG --> GEN["Nucleus Sampling (p=0.9) + Repetition Penalty (1.3)"]
end
FER --> CH
REG --> PADDED
CKPT --> PROMPT
<pad> (token ID 0). Padding tokens were assigned target labels of -100.p = 0.20): In 20% of training samples, the text tag (e.g. <cooking> ) was omitted from the prompt string while keeping domain_ids = 1. This forced the model's adapter weights to model the lexical and syntactic domain distribution directly rather than simply learning a shallow prompt prefix association.p = 0.10): In 10% of training steps, domain_ids was dynamically reassigned to the unconditional virtual index 6: <NULL>, jointly training conditional and unconditional distributions without needing additional data.Darja-GPT-50M employs a modern autoregressive transformer architecture with SwiGLU non-linearities, Rotary Position Embeddings (RoPE), pre-layer RMSNorm with Head-level RMSNorm for queries and keys, tied word embeddings, and native Domain LoRA Adapters.
flowchart TD
subgraph Inputs["Model Inputs"]
TOK["input_ids [B, T]"]
DOM["domain_ids [B] (0..6)"]
MASK["attention_mask [B, T]"]
end
subgraph EmbeddingLayer["Embedding & Weight Tying"]
EMB["Token Embedding (24,000 × 512)"]
end
subgraph DecoderStack["12 × DecoderBlock Stack"]
direction TB
NORM1["RMSNorm 1 (512)"]
subgraph AttnBlock["Causal Self-Attention"]
QKV["q_proj, k_proj, v_proj (512 → 512)"]
QKNORM["HeadRMSNorm on Q and K (d_head=64)"]
ROPE["Rotary Position Embedding (RoPE, θ=10000)"]
SDPA["Scaled Dot-Product Attention + Causal Mask"]
O_BASE["Base o_proj Linear (512 → 512)"]
O_LORA["DomainLoRA Adapters (7 × r=16 × 1024)"]
O_COMB["Base + α/r · LoRA_B @ LoRA_A(x)"]
end
ADD1["Residual Connection (+)"]
NORM2["RMSNorm 2 (512)"]
subgraph MLPBlock["SwiGLU Feed-Forward Network"]
GATE["gate_proj Linear (512 → 1408)"]
UP["up_proj Linear (512 → 1408)"]
SILU["SiLU(gate) ⊗ up"]
DOWN_BASE["Base down_proj Linear (1408 → 512)"]
DOWN_LORA["DomainLoRA Adapters (7 × r=16 × 1920)"]
DOWN_COMB["Base + α/r · LoRA_B @ LoRA_A(x)"]
end
ADD2["Residual Connection (+)"]
end
subgraph OutputLayer["Prediction Head"]
FINAL_NORM["Final RMSNorm (512)"]
LM_HEAD["Tied LM Head Linear (512 → 24,000)"]
LOGITS["logits [B, T, 24,000]"]
end
TOK --> EMB
EMB --> NORM1
EMB -.-> ADD1
NORM1 --> QKV
QKV --> QKNORM
QKNORM --> ROPE
ROPE --> SDPA
MASK --> SDPA
SDPA --> O_BASE
SDPA --> O_LORA
DOM -.-> O_LORA
O_BASE --> O_COMB
O_LORA --> O_COMB
O_COMB --> ADD1
ADD1 --> NORM2
ADD1 -.-> ADD2
NORM2 --> GATE
NORM2 --> UP
GATE --> SILU
UP --> SILU
SILU --> DOWN_BASE
SILU --> DOWN_LORA
DOM -.-> DOWN_LORA
DOWN_BASE --> DOWN_COMB
DOWN_LORA --> DOWN_COMB
DOWN_COMB --> ADD2
ADD2 --> FINAL_NORM
FINAL_NORM --> LM_HEAD
LM_HEAD --> LOGITS
EMB --- LM_HEAD
RMSNorm(x) = (x / sqrt(mean(x²) + ε)) ⊙ w
where ε = 1e-5. Queries and keys undergo per-head normalization HeadRMSNorm(q_h) with head dimension d_head = 64 prior to rotational embedding.
RoPE(q_m, m) = q_m · cos(m · θ) + rotate_half(q_m) · sin(m · θ)
with base frequency θ = 10,000.0 and inverse frequencies θ_i = 10000^(-2(i-1)/d_head).
Attention(Q, K, V) = softmax((Q · K^T / sqrt(d_head)) + M_causal + M_key_pad) · V
SwiGLU(x) = (SiLU(x · W_gate) ⊙ (x · W_up)) · W_down
where intermediate projection dimension d_ffn = 1,408 (aligned to 64).
DomainLoRALinear):y = x · W_0 + (α / r) · (x · A_d) · B_d
where W_0 is the frozen/shared base projection, A_d ∈ R^(d_in × r), B_d ∈ R^(r × d_out), rank r = 16, and scaling α = 16.0 (scaling = 1.0). B_d is initialized to zeros, ensuring training starts as an exact identity operation.
The model consists of 50,837,504 base parameters and 3,956,736 domain adapter parameters, totaling 54,794,240 parameters (54.8M):
| Module | Component | Tensor Dimensions | Per-Layer Base | Per-Layer Adapters (r=16, D=7) | 12-Layer Total |
|---|---|---|---|---|---|
| Embeddings | model.embed_tokens |
[24000, 512] |
— | — | 12,288,000 |
| Attention (×12) | q_proj, k_proj, v_proj |
3 × [512, 512] |
786,432 | — | 9,437,184 |
q_norm, k_norm |
2 × [64] |
128 | — | 1,536 | |
o_proj.base |
[512, 512] |
262,144 | — | 3,145,728 | |
o_proj.adapters |
A: [7, 16, 512], B: [7, 512, 16] |
— | 114,688 | 1,376,256 | |
| MLP (×12) | gate_proj, up_proj |
2 × [1408, 512] |
1,441,792 | — | 17,301,504 |
down_proj.base |
[512, 1408] |
720,896 | — | 8,650,752 | |
down_proj.adapters |
A: [7, 16, 1408], B: [7, 512, 16] |
— | 215,040 | 2,580,480 | |
| Norms (×12) | input_norm, post_attn_norm |
2 × [512] |
1,024 | — | 12,288 |
| Final Norm | model.final_norm |
[512] |
— | — | 512 |
| Output Head | lm_head |
[24000, 512] |
(Tied with Embeddings) | — | (0) |
| Subtotals | 3,212,416 / layer | 329,728 / layer | |||
| Total Base | 50,837,504 | ||||
| Total Adapters | 3,956,736 | ||||
| Grand Total | 54,794,240 (54.8M) |
flowchart TD
P["User Prompt (e.g. Cooking Recipe)"] --> TOK["Tokenize with Custom Darja BPE"]
TOK --> B_COND["Conditional Stream: domain_ids = [d] (e.g. 1: cooking)"]
TOK --> B_UNCOND["Unconditional Stream: domain_ids = [6] (NULL)"]
B_COND --> M_COND["Model Forward with Domain LoRA"]
B_UNCOND --> M_UNCOND["Model Forward with NULL LoRA"]
M_COND --> LOG_COND["Conditional Logits z_cond"]
M_UNCOND --> LOG_UNCOND["Unconditional Logits z_NULL"]
LOG_COND --> CFG_MATH["CFG Logit Extrapolation: z_guided = z_NULL + s · (z_cond - z_NULL)"]
LOG_UNCOND --> CFG_MATH
CFG_MATH --> REP["Repetition Penalty (1.25 - 1.3)"]
REP --> TEMP["Temperature Scaling (T=0.8)"]
TEMP --> TOPP["Top-p Nucleus Filtering (p=0.9)"]
TOPP --> SAMPLE["Multinomial Sampling"]
SAMPLE --> EMIT["Emitted Next Token"]
EMIT -.->|"Append to Sequence"| TOK
The architecture provides 7 independent low-rank adapter slots:
| Domain ID | Tag | Corpus Sources | Distinctive Lexical & Register Markers |
|---|---|---|---|
0 |
<general> |
Vlogs, comedy channels (DZjoker, Anes Tina, Mourad Oudia) | Everyday colloquial slang, jokes, Arabizi code-switching, greetings |
1 |
<cooking> |
Traditional & modern culinary channels (Amina Cuisine, Chef Moha) | Instructional imperatives, culinary terms (طاجين, بطاطا, مرقة, كوزينة, توابل) |
2 |
<tech> |
Tech reviews, hardware, electronics discussions | Telephony, hardware specifications, prices, English/French loanwords |
3 |
<sports> |
Football match reactions, CAN, national team analysis (Dzair Foot) | Sports passion register, clubs (MCA, USMA, CRB), national team, Mahrez |
4 |
<podcast> |
Business, marketing, startup, and self-help podcasts | Entrepreneurship terms, freelancing, mindset, formal conversational Arabic |
5 |
<story> |
Narrative folklore, storytelling, social chronicles | Narrative past tense, family relationships, traditional formulas (كان يا ما كان) |
6 |
<NULL> |
Unconditional baseline (trained with 10% dropout) | Neutral background register; serves as reference baseline for CFG extrapolation |
We conducted an extensive, controlled benchmark comparing our custom tokenizer against 30 open-weights tokenizers using 200 strictly held-out test documents (341,424 words / 1,855,532 characters across all 6 domains) from the Algerian Darja Corpus. All evaluations used raw document text without domain tags or special tokens.
| # | Tokenizer | Architecture / Family | Vocab Size | Fertility ↓ | Bytes / Tok ↑ | UNK % | Ours Needs Fewer Tokens By (95% CI) |
|---|---|---|---|---|---|---|---|
| 1 | ★ Custom Darja Tokenizer (Ours) | Byte-Level BPE | 24,000 | 1.431 | 6.80 | 0.00% | Baseline (Ours) |
| 2 | SI2M-Lab/DarijaBERT |
WordPiece (Darija) | 80,000 | 1.427 | 6.81 | 0.00% | -0.3% [-1.1%, +0.6%] |
| 3 | alger-ia/dziribert |
WordPiece (Dziri) | 50,000 | 1.429 | 6.80 | 0.00% | -0.1% [-1.1%, +0.8%] |
| 4 | UBC-NLP/MARBERT |
WordPiece (Arabic Dialects) | 100,000 | 1.440 | 6.75 | 0.06% | +0.6% [-0.7%, +1.8%] |
| 5 | inceptionai/jais-family-590m |
BPE (Arabic/English LLM) | 84,992 | 1.639 | 5.93 | 0.00% | +12.7% [+11.3%, +14.0%] |
| 6 | ALLaM-AI/ALLaM-7B-Instruct |
BPE (Arabic LLM) | 64,000 | 1.651 | 5.89 | 0.00% | +13.3% [+11.9%, +14.6%] |
| 7 | CAMeL-Lab/bert-base-arabic-camelbert |
WordPiece (Arabic Mix) | 30,000 | 1.658 | 5.87 | 0.31%* | +13.7% [+12.4%, +14.8%] |
| 8 | aubmindlab/bert-base-arabertv2 |
WordPiece (Arabic) | 64,000 | 1.756 | 5.54 | 0.54%* | +18.5% [+17.7%, +19.4%] |
| 9 | aubmindlab/aragpt2-base |
BPE (Arabic Causal LM) | 64,000 | 1.774 | 5.48 | 0.00% | +19.4% [+17.2%, +21.4%] |
| 10 | bigscience/bloom-560m |
Byte-Level BPE (Multilingual) | 250,680 | 1.799 | 5.41 | 0.00% | +20.4% [+19.3%, +21.5%] |
| 11 | FacebookAI/xlm-roberta-base |
SentencePiece (Multilingual) | 250,002 | 1.961 | 4.96 | 0.00% | +27.0% [+26.1%, +28.0%] |
| 12 | facebook/mbart-large-50 |
SentencePiece (Multilingual) | 250,054 | 1.961 | 4.96 | 0.00% | +27.0% [+26.1%, +28.0%] |
| 13 | mistralai/Mistral-Nemo-Base-2407 |
Tekken (Multilingual LLM) | 131,072 | 1.990 | 4.89 | 0.00% | +28.1% [+27.0%, +29.1%] |
| 14 | openai/gpt-oss-20b |
Byte-Level BPE | 200,019 | 2.040 | 4.77 | 0.00% | +29.8% [+28.9%, +30.7%] |
| 15 | facebook/nllb-200-distilled-600M |
SentencePiece (200 Langs) | 256,204 | 2.061 | 4.72 | 0.09% | +30.6% [+29.8%, +31.3%] |
| 16 | MBZUAI-Paris/Atlas-Chat-2B |
BPE (Arabic-Centric LLM) | 256,000 | 2.128 | 4.57 | 0.00% | +32.8% [+31.9%, +33.6%] |
| 17 | QCRI/Fanar-1-9B |
BPE (Arabic/English LLM) | 128,256 | 2.132 | 4.56 | 0.00% | +32.9% [+32.0%, +33.7%] |
| 18 | google/mt5-small |
SentencePiece (Multilingual) | 250,100 | 2.221 | 4.38 | 0.00% | +35.6% [+34.8%, +36.3%] |
| 19 | Qwen/Qwen2.5-0.5B |
BPE (Multilingual LLM) | 151,665 | 2.354 | 4.13 | 0.00% | +39.2% [+38.4%, +40.0%] |
| 20 | Qwen/Qwen3-0.6B |
BPE (Multilingual LLM) | 151,669 | 2.354 | 4.13 | 0.00% | +39.2% [+38.4%, +40.0%] |
| 21 | google-bert/bert-base-multilingual |
WordPiece (Multilingual) | 119,547 | 2.376 | 4.09 | 0.08% | +39.8% [+39.0%, +40.5%] |
| 22 | EleutherAI/gpt-neox-20b |
BPE | 50,277 | 3.878 | 2.51 | 0.00% | +63.1% [+62.4%, +63.7%] |
| 23 | microsoft/phi-4 |
BPE | 100,352 | 3.900 | 2.49 | 0.00% | +63.3% [+62.6%, +63.9%] |
| 24 | allenai/OLMo-2-1124-7B |
BPE | 100,278 | 3.900 | 2.49 | 0.00% | +63.3% [+62.6%, +63.9%] |
| 25 | mistralai/Mistral-7B-v0.3 |
Llama-style BPE | 32,768 | 5.005 | 1.94 | 0.00% | +71.4% [+70.8%, +71.9%] |
| 26 | microsoft/Phi-3-mini-4k-instruct |
BPE | 32,011 | 5.094 | 1.91 | 0.00% | +71.9% [+71.3%, +72.4%] |
| 27 | HuggingFaceTB/SmolLM2-360M |
Byte-Level BPE | 49,152 | 5.247 | 1.85 | 0.00% | +72.7% [+72.3%, +73.1%] |
| 28 | openai-community/gpt2 |
Byte-Level BPE | 50,257 | 5.595 | 1.74 | 0.00% | +74.4% [+74.0%, +74.8%] |
| 29 | tiiuae/falcon-7b |
BPE | 65,024 | 5.683 | 1.71 | 0.00% | +74.8% [+74.4%, +75.2%] |
*Note: High UNK rates (>0.1%) artificially flatter the apparent fertility of WordPiece models by replacing unsegmented characters with [UNK].
touati-kamel/algerian-darja-corpus (conditioned subset)<eos> boundaries and packed into exact 1,024-token chunks, yielding 52,038 dense training sequences (~4.8× data yield improvement over naive truncation).beta1 = 0.9, beta2 = 0.95, weight_decay = 0.1)eps = 0.1, gradient clipping at 1.03e-4, 500 warmup steps)domain_ids = 6 (<NULL>) per batchload_best_model_at_end = TrueEvaluated on 335 strictly held-out test documents (never seen during training, split with fixed seed):
Direct inspection of weight-delta Frobenius norms ||ΔW||_F across all 12 layers:
| Layer Type | Comparison Pair | FiLM Distance (v4) | v5 Adapter Distance (LoRA) | Finding |
|---|---|---|---|---|
Attention o_proj |
Cooking vs. Tech | 0.922 | 1.412 (+53%) | Distinct lexical heads |
Attention o_proj |
Sports vs. Cooking | 1.124 | 1.731 (+54%) | Strong topical separation |
MLP down_proj |
Cooking vs. NULL | 0.763 | 1.184 (+55%) | Strong divergence from neutral |
MLP down_proj |
Sports vs. NULL | 0.682 | 1.365 (+100%) | Powerful specialized sport projection |
Each domain was evaluated across 5 controlled experimental regimes:
notag_null: Prompt with no text tag, evaluated on unconditional base (<NULL>).tag_null: Prompt with text tag, evaluated on unconditional base.tag_correct: Prompt with text tag + matching domain adapter.tag_mismatch: Causal test — Prompt text tag matches Domain A, but adapter is set to Domain B.tag_correct+CFG: Prompt with text tag + matching adapter guided via CFG (s = 2.5).<cooking> prompt text into the <tech> adapter immediately injected electronics pricing, battery life, and Bluetooth mentions into cooking sentences, proving the adapter causally controls generation semantics rather than relying purely on text tag autoregression.<story>): Produced sustained multi-sentence folklore and personal narratives in 10/10 samples with no phrase-level degeneracy.Because Darja-GPT-50M-v5 uses custom rotary and domain-adapter layers, generation requires registering the configuration or using the standalone model file.
You can either install the repository as an editable Python package or execute scripts directly:
# 1. Clone repository
git clone https://github.com/touati-kamel/darja-gpt.git
cd darja-gpt
# 2. Install dependencies & core package
pip install -e .
Run instant inference using the self-bootstrapping CLI or interactive mode:
# Interactive conversation session
python generate.py --interactive
# Direct CLI steering with CFG
python generate.py --domain cooking --cfg 2.2 --prompt "اليوم غادي نديرو وصفة بسيطة وسهلة، خاصنا"
python generate.py --domain sports --cfg 2.5 --prompt "البارح كانت مباراة كبيرة وقوية بزاف"
from darja_gpt import load_darja_gpt, generate_cfg
# Load model and tokenizer
tokenizer, model, device = load_darja_gpt(
model_id="touati-kamel/darja-gpt-50m",
tokenizer_id="touati-kamel/darja-tokenizer-50m"
)
# Generate with Classifier-Free Guidance
output = generate_cfg(
model=model,
tokenizer=tokenizer,
prompt="اليوم غادي نديرو وصفة بسيطة وسهلة، خاصنا",
domain_name="cooking",
guidance_scale=2.2,
max_new_tokens=150,
device=device
)
print(output)
import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModelForCausalLM
# 1. Load Tokenizer & Model
MODEL_ID = "touati-kamel/darja-gpt-50m"
TOKENIZER_ID = "touati-kamel/darja-tokenizer-50m"
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(TOKENIZER_ID)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID, trust_remote_code=True).to(device)
model.eval()
# Domain Mapping
DOMAIN_TO_IDX = {
"<general>": 0,
"<cooking>": 1,
"<tech>": 2,
"<sports>": 3,
"<podcast>": 4,
"<story>": 5,
"<NULL>": 6
}
NULL_DOMAIN_IDX = 6
@torch.no_grad()
def generate_darja_cfg(
prompt: str,
domain_tag: str = "<cooking>",
guidance_scale: float = 2.0,
max_new_tokens: int = 120,
temperature: float = 0.8,
top_p: float = 0.9,
repetition_penalty: float = 1.25
):
full_prompt = f"{domain_tag} {prompt}"
input_ids = tokenizer.encode(full_prompt, return_tensors="pt").to(device)
cond_domain = torch.tensor([DOMAIN_TO_IDX[domain_tag]], dtype=torch.long, device=device)
uncond_domain = torch.tensor([NULL_DOMAIN_IDX], dtype=torch.long, device=device)
generated = input_ids
for _ in range(max_new_tokens):
if generated.size(1) >= model.config.max_position_embeddings:
break
# Parallel forward passes for CFG
logits_cond = model(input_ids=generated, domain_ids=cond_domain).logits[:, -1, :]
logits_uncond = model(input_ids=generated, domain_ids=uncond_domain).logits[:, -1, :]
# Classifier-Free Guidance extrapolation
logits = logits_uncond + guidance_scale * (logits_cond - logits_uncond)
# Repetition penalty
for token_id in set(generated[0].tolist()):
if logits[0, token_id] > 0:
logits[0, token_id] /= repetition_penalty
else:
logits[0, token_id] *= repetition_penalty
# Temperature & Top-p (Nucleus) sampling
logits = logits / max(temperature, 1e-5)
probs = F.softmax(logits, dim=-1)
sorted_probs, sorted_indices = torch.sort(probs, descending=True)
cumulative_probs = torch.cumsum(sorted_probs, dim=-1)
cutoff = cumulative_probs > top_p
cutoff[..., 1:] = cutoff[..., :-1].clone()
cutoff[..., 0] = False
sorted_probs[cutoff] = 0.0
sorted_probs = sorted_probs / sorted_probs.sum(dim=-1, keepdim=True)
next_token = sorted_indices.gather(-1, torch.multinomial(sorted_probs, num_samples=1))
generated = torch.cat([generated, next_token], dim=1)
if next_token.item() == tokenizer.eos_token_id:
break
return tokenizer.decode(generated[0], skip_special_tokens=True)
# Example 1: Culinary Recipe in Algerian Darija
cooking_output = generate_darja_cfg(
prompt="اليوم غادي نديرو وصفة بسيطة وسهلة، خاصنا",
domain_tag="<cooking>",
guidance_scale=2.2
)
print("--- Cooking Output ---")
print(cooking_output)
# Example 2: Sports Commentary
sports_output = generate_darja_cfg(
prompt="البارح كانت مباراة كبيرة وقوية بزاف",
domain_tag="<sports>",
guidance_scale=2.5
)
print("\n--- Sports Output ---")
print(sports_output)
If you use Darja-GPT-50M or the Algerian Darja Corpus in your research or project, please cite:
@misc{touati2026darjagpt50m,
author = {Kamel Touati},
title = {Darja-GPT-50M: Domain-Conditioned Causal Language Model for Algerian Darija with Classifier-Free Guidance},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/touati-kamel/darja-gpt-50m}}
}
@misc{touati2026darjacorpus,
author = {Kamel Touati},
title = {Algerian Darja Corpus: Multi-Domain Conversational Transcripts in Algerian Arabic},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/datasets/touati-kamel/algerian-darja-corpus}}
}
Developer: Kamel Touati (Hugging Face Profile)
License: Apache 2.0
last-checkpoint/optimizer.pt 439 MBlast-checkpoint/model.safetensors 219 MBmodel.safetensors 219 MBlast-checkpoint/rng_state.pth 14.6 kBlast-checkpoint/training_args.bin 5.2 kBtraining_args.bin 5.2 kBlast-checkpoint/scheduler.pt 1.5 kBlast-checkpoint/scaler.pt 1.4 kB