gemma-4-E4B-it-text-only-non-thinking

A tokenizer-only variant of tuandunghcmut/gemma-4-E4B-it-text-only with a modified chat template that permanently forces non-thinking mode (enable_thinking = false), regardless of any argument passed by the caller.

⚠️ This repo contains no model weights. It is designed to be used as a --tokenizer override with vLLM (or any framework that supports separate model/tokenizer repos). The weights live in the source repo.

---

What changed

A single two-line override is prepended to the top of chat_template.jinja (and embedded in tokenizer_config.json):

{#- Force non-thinking mode: override any caller-supplied enable_thinking to false -#}
{%- set enable_thinking = false -%}

This ensures the <|think|> token is never injected into the system prompt, so the model always responds directly without a reasoning/thinking preamble.

---

Deploying with vLLM ✅

No weight download needed for this repo. Load the model weights from the source and the tokenizer (with non-thinking template) from this repo:

vllm serve tuandunghcmut/gemma-4-E4B-it-text-only \
  --served-model-name gemma-4-E4B-it-text-only-non-thinking \
  --tokenizer tuandunghcmut/gemma-4-E4B-it-text-only-non-thinking \
  --max-model-len 32768 \
  --dtype bfloat16

Then query via the OpenAI-compatible API:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")

response = client.chat.completions.create(
    model="gemma-4-E4B-it-text-only-non-thinking",
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ],
)
print(response.choices[0].message.content)
# ➜ Direct answer, no thinking preamble

---

Deploying with 🤗 Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load weights from source, tokenizer (with non-thinking template) from this repo
model = AutoModelForCausalLM.from_pretrained(
    "tuandunghcmut/gemma-4-E4B-it-text-only",
    dtype=torch.bfloat16,
    device_map="auto",
)
tok = AutoTokenizer.from_pretrained(
    "tuandunghcmut/gemma-4-E4B-it-text-only-non-thinking"
)

messages = [{"role": "user", "content": "What is the capital of France?"}]

# enable_thinking=True is silently ignored — template always uses non-thinking
prompt = tok.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=True
)

inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=200)
print(tok.decode(out[0], skip_special_tokens=True))

---

Why this is useful

---

Behaviour comparison

---

Source model

• Tokenizer + template source: tuandunghcmut/gemma-4-E4B-it-text-only
• Original model: google/gemma-4-E4B-it

License

Inherits the Gemma license from the base model.