🧢 Qwen3.5-4B-StreetTalk

"You are Tommy, a nice bro." An ultra-concise, hyper-natural and deeply authentic West Coast American English (maybe AAVE) conversational model, aligned with SFT → DPO → GRPO where JEV acts as the reward model.

🔬 Training pipeline

[Base: Qwen/Qwen3.5-4B]
        │
        ▼  Phase 1 — SFT
[SFT: AAVE syntax & idiom, persona "Tommy" in chat template]
        │
        ▼  Phase 2 — DPO (β = 0.1)
[DPO: margin against lecture-style, listy, over-long replies]
        │
        ▼  Phase 3 — GRPO (1,000 steps, G = 8)
[StreetTalk: short, warm, zero-cliché, technically sharp]

🧑‍⚖️ JEV as the reward model

The RL stage does not use a scalar preference classifier. Each group of 8 candidate replies is scored by JEV (TypeSafe Jev), a structured-judgement engine that returns one machine-checkable verdict per dimension in a single pass — so the verifier can be audited and versioned instead of being an opaque black box.

Five orthogonal dimensions are judged per candidate:

Dimension	What it measures
`human_naturalness`	Does it read like a person typing, or like a document?
`brevity_and_punch`	Is it short and dense, or padded?
`west_coast_vibe`	Authentic register, not caricature
`is_ai_cliche`	"As an AI…", "It's important to note…", signposting, hedging
`has_error_or_hallucination`	Factual / arithmetic / technical errors

The verdicts are composed into the group reward used by GRPO's advantage normalisation. Because the judge emits structured fields rather than free text, drift in the reward is visible in the logs (see the curve below) instead of hiding inside an average.

📊 Measured alignment results (this run)

Metric	Start (SFT/DPO)	After 1,000 GRPO steps
Mean response length	350+ tokens	115 tokens
Truncation rate	—	0.00%
JEV composite reward	~+0.5	peak ~+4.1 (rolling mean ~+3.9 near step 690)

💬 Demo

🚀 Quickstart

Python — transformers

Qwen3.5 uses the multimodal AutoProcessor path (transformers ≥ 5.x). Text-only messages work exactly the same way:

from transformers import AutoProcessor, AutoModelForMultimodalLM

model_id = "wesjos/Qwen3.5-4B-StreetTalk"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto")

messages = [
    {"role": "user", "content": [{"type": "text", "text": "How do I fix CUDA out of memory errors?"}]},
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

out = model.generate(**inputs, max_new_tokens=256, temperature=0.7, top_p=0.9)
print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Sampling that matches the alignment: temperature=0.7, top_p=0.9, max_new_tokens 256 is plenty (mean output is ~115 tokens). Pushing temperature above ~0.9 drifts into exaggerated slang.

llama.cpp / GGUF

Pre-quantised weights (Q4_K_M, Q8_0) plus the F16 vision projector:

👉 wesjos/Qwen3.5-4B-StreetTalk-GGUF

📜 Chat template

Standard ChatML with an embedded persona fallback — if you pass no system message, you get Tommy:

{%- if messages[0]['role'] == 'system' -%}
    {%- set system_message = messages[0]['content'] -%}
    {%- set loop_messages = messages[1:] -%}
{%- else -%}
    {%- set system_message = "you are Tommy, a nice bro." -%}
    {%- set loop_messages = messages -%}
{%- endif -%}
{{- '<|im_start|>system\n' + system_message + '<|im_end|>\n' -}}
{%- for message in loop_messages -%}
    {{- '<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>\n' -}}
{%- endfor -%}
{%- if add_generation_prompt -%}
    {{- '<|im_start|>assistant\n' -}}
{%- endif -%}

📄 License

Apache License 2.0. Persona name and style are a fictional character; the underlying model is Qwen3.5-4B (Apache 2.0).

wesjos/Qwen3.5-4B-StreetTalk