Qwen3-1.7B Lipogram Model — LoRA Adapter (Minus the 'E')
This repository contains the fine-tuned PEFT LoRA adapter for Qwen3-1.7B, trained to adhere to a strict character-level constraint: generating fluent, grammatical English without ever using the letter "e" or "E" (a lipogram).
Trained and aligned using a 3-layer architecture:
- Logit Masking: Pre-computed boolean mask blocking 103,639 tokens (68.33% of the Qwen3 vocabulary).
- Supervised Fine-Tuning (SFT): 4-bit QLoRA trained on 180 curated, human-verified e-free instruction pairs.
- Reinforcement Learning (GRPO): Group Relative Policy Optimization optimizing response fluency, length consistency, and lexical diversity.
Benchmark Results (Gadsby Benchmark Suite)
| Model Architecture Stage | Strict Lipogram Pass | Usable Response Rate | N-Gram Diversity |
|---|---|---|---|
| Base Qwen-1.7B + Mask | 88.0% | 28.0%* | 91.5% |
| SFT (180 Rewrites) + Mask | 96.0% | 80.0% | 91.5% |
| RL (GRPO Best) + Mask | 100.0% | 88.0% | 92.5% |
*Note: Base model usable rate reflects pure English ASCII responses; unaligned base models attempt foreign-language token substitutions to evade the constraint.
Usage with PEFT
1. Requirements
pip install torch transformers peft
2. Loading the Adapter with Logit Mask
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# 1. Load base model and tokenizer
model_id = "Qwen/Qwen3-1.7B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
base_model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# 2. Attach fine-tuned lipogram adapter
model = PeftModel.from_pretrained(base_model, "prateekfr/qwen3-1.7b-no-e-lora")
model.eval()
# 3. Build forbidden token mask (blocking 'e' and 'E')
vocab_size = model.config.vocab_size
forbidden_mask = torch.zeros(vocab_size, dtype=torch.bool)
for token_id in range(vocab_size):
token_str = tokenizer.decode([token_id])
if "e" in token_str.lower():
forbidden_mask[token_id] = True
# 4. Generate with mask applied to logits
prompt = "Write a short poem about a frog."
inputs = tokenizer(f"<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n", return_tensors="pt").to("cuda")
with torch.no_grad():
curr_ids = inputs["input_ids"]
for _ in range(60):
outputs = model(input_ids=curr_ids)
scores = outputs.logits[:, -1, :].clone().float()
scores[:, forbidden_mask.to("cuda")] = -float("inf")
next_token = torch.multinomial((scores / 0.7).softmax(-1), num_samples=1)
if next_token.item() == tokenizer.eos_token_id:
break
curr_ids = torch.cat([curr_ids, next_token], dim=-1)
print(tokenizer.decode(curr_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip())
Citation & Attribution
- Author & Post-Training Engineer: Prateek
- Base Architecture: Qwen3-1.7B by the Qwen Team / Alibaba Cloud
- License: Apache 2.0