🚀 GenZ Translator

A compact, 29.65M-parameter language model designed to translate Gen Z slang into clear, natural English.

GenZ Translator is a lightweight language model built for a focused task: understanding informal internet language and rewriting it in standard conversational English while preserving the original meaning, emotion, attitude, and context.

Version 2 continues training from the original GenZ Translator checkpoint, with additional cleaned translation data and curated examples targeting abbreviations, slang, and meaning-related questions.

• ⚡ Compact model for local inference
• 🧠 Specialized slang and abbreviation translation
• 🔤 Custom tokenizer
• 🤗 Compatible with the Transformers library
• 🪶 Approximately 30 million parameters

---

✨ What changed in v2?

Version 2 introduces a continued fine-tuning stage designed to improve translation quality, especially for common abbreviations and contextual slang.

Improvements observed in the current evaluation include:

• Better handling of common abbreviations such as WYD, tbh, omw, ty, and np
• Improved responses to prompts asking what slang expressions mean
• Better performance on the handcrafted evaluation benchmark
• Reduced loss on the curated validation examples

The model still has limitations involving unusual spellings, repeated characters, mixed-language text, URLs, and preservation of every detail in the input.

Important: Improvements vary by evaluation set. The current handcrafted benchmark shows a clear gain, but the held-out dataset benchmark does not show a corresponding improvement in exact-match accuracy.

---

📊 Model overview

The architecture uses separate input embeddings and output-head weights (tie_word_embeddings=False).

---

🎯 Intended use

Suitable for

• Translating common internet abbreviations
• Rewriting informal messages into standard English
• Interpreting common Gen Z expressions
• Processing short social media captions and messages
• Lightweight experimentation with compact language models

Not designed for

• General-purpose conversation
• Coding or mathematical reasoning
• Knowledge retrieval
• Long-context conversations
• Reliable translation of every regional or rapidly evolving slang expression

This model is a specialized translator, not a general-purpose assistant.

---

🧪 Evaluation

Handcrafted benchmark

The handcrafted benchmark measures whether outputs satisfy category-specific evaluation rules. Its results should not be interpreted as a universal translation accuracy percentage.

Held-out dataset benchmark

Exact-match evaluation is strict: a valid translation can be marked incorrect if its wording differs from the reference. Token F1 offers a measure of lexical overlap but does not fully measure semantic correctness.

Validation loss during the v2 fine-tuning run

The combined loss is the arithmetic mean of general and curated validation losses. The curated validation set contained 82 examples, so its result should be interpreted cautiously.

These loss figures come from the current fine-tuning run. They should not be compared directly with validation losses from a different dataset split or evaluation pipeline.

Interpretation

The results suggest that v2 has learned useful mappings for common abbreviations and some meaning-related prompts. However, generalization remains a work in progress.

Known failure cases include:

• Losing negation or other important details
• Mistranslating phrase-level expressions such as pull up and stan
• Rewriting ordinary English that does not need translation
• Producing poor results on repeated characters and adversarially malformed inputs
• Mishandling URLs, usernames, and some mixed-language messages

Further evaluation on an independently held-out test set is needed before making stronger claims about general translation quality.

---

🏗️ Training approach

Original model

The original GenZ Translator was trained from scratch using a custom tokenizer and a specialized Gen Z-to-English dataset.

It did not start from pretrained Meta Llama, Qwen, or Mistral model weights.

Version 2

Version 2 continues from the original Sankar-2910/genz-translator checkpoint rather than starting from random initialization.

The current run used:

The dataset pipeline loaded a source dataset of approximately one million rows and filtered it before training. The figures above describe the prepared data for this run, not the full raw dataset.

Training examples included curated examples repeated during training. The held-out curated examples were kept separate from those repeated training examples.

Loss was computed over response tokens rather than prompt tokens.

---

🔤 Tokenizer and prompt format

The model uses a custom tokenizer trained for the original translation task.

Use the prompt structure expected by the model:

<s><|instruction|>Translate the following Gen Z slang sentence into clear, standard English.<|input|>YOUR TEXT<|response|>

The model is optimized for single-turn translation. It is not designed to maintain conversational history.

---

⚡ Quick start

Install the required packages:

pip install -U transformers torch safetensors

Load v2 with Transformers:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "Sankar-2910/genz-translator"

tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    tie_word_embeddings=False
).eval()

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)

def translate(text, max_new_tokens=96):
    prompt = (
        "<s><|instruction|>"
        "Translate the following Gen Z slang sentence into clear, standard English."
        "<|input|>" + text +
        "<|response|>"
    )

    inputs = tokenizer(
        prompt,
        add_special_tokens=False,
        return_tensors="pt"
    ).to(device)

    with torch.inference_mode():
        output = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=False,
            eos_token_id=tokenizer.eos_token_id,
            pad_token_id=tokenizer.eos_token_id
        )

    generated = output[0, inputs["input_ids"].shape[1]:]

    return tokenizer.decode(
        generated,
        skip_special_tokens=True
    ).strip()

print(translate("WYD"))
print(translate("tbh, that movie was mid"))
print(translate("omw, traffic is crazy"))

Greedy decoding (do_sample=False) provides repeatable outputs under the same model and inference conditions. It does not guarantee that every output will be accurate.

For longer paragraphs, increase max_new_tokens as needed.

---

⚠️ Limitations

GenZ Translator v2 is a compact, specialized model. It can still:

• Misinterpret ambiguous slang
• Lose negation, names, or contextual details
• Over-rewrite already-natural English
• Produce malformed outputs for adversarial or repetitive inputs
• Struggle with regional slang, multilingual text, and recent expressions
• Generate wording different from the reference despite retaining some meaning

Do not assume a translation is correct solely because it is fluent. For important text, verify the output.

---

📦 Repository files

The Hugging Face repository includes the model configuration, weights, and tokenizer files required for loading the model with Transformers.

The repository is intended to distribute the standard safetensors checkpoint. Quantized or GGUF variants are not included unless published separately.

---

📖 Citation

If you use this model in research, please cite:

@misc{genztranslator2026v2,
  title={GenZ Translator v2},
  author={Sankar Narayanan},
  year={2026},
  note={A 29.65M-parameter decoder-only Transformer for Gen Z slang translation, continued from an earlier custom-trained checkpoint},
  url={https://huggingface.co/Sankar-2910/genz-translator-v2}
}

For the underlying Llama architecture, see:

@article{touvron2023llama,
  title={LLaMA: Open and Efficient Foundation Language Models},
  author={Touvron, Hugo and others},
  journal={arXiv preprint arXiv:2302.13971},
  year={2023}
}

---

❤️ Built by a student

GenZ Translator is an exploration of compact language models, custom tokenization, dataset preparation, and specialized natural-language generation.

Created by Sankara Narayanan.

Project: Sankar-2910/genz-translator-v2