This repository contains the fine-tuned PEFT LoRA adapter for Qwen3-1.7B, trained to adhere to a strict character-level constraint: generating fluent, grammatical English without ever using the letter "e" or "E" (a lipogram).
Trained and aligned using a 3-layer architecture:
| Model Architecture Stage | Strict Lipogram Pass | Usable Response Rate | N-Gram Diversity |
|---|---|---|---|
| Base Qwen-1.7B + Mask | 88.0% | 28.0%* | 91.5% |
| SFT (180 Rewrites) + Mask | 96.0% | 80.0% | 91.5% |
| RL (GRPO Best) + Mask | 100.0% | 88.0% | 92.5% |
*Note: Base model usable rate reflects pure English ASCII responses; unaligned base models attempt foreign-language token substitutions to evade the constraint.
pip install torch transformers peft
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# 1. Load base model and tokenizer
model_id = "Qwen/Qwen3-1.7B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
base_model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# 2. Attach fine-tuned lipogram adapter
model = PeftModel.from_pretrained(base_model, "prateekfr/qwen3-1.7b-no-e-lora")
model.eval()
# 3. Build forbidden token mask (blocking 'e' and 'E')
vocab_size = model.config.vocab_size
forbidden_mask = torch.zeros(vocab_size, dtype=torch.bool)
for token_id in range(vocab_size):
token_str = tokenizer.decode([token_id])
if "e" in token_str.lower():
forbidden_mask[token_id] = True
# 4. Generate with mask applied to logits
prompt = "Write a short poem about a frog."
inputs = tokenizer(f"<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n", return_tensors="pt").to("cuda")
with torch.no_grad():
curr_ids = inputs["input_ids"]
for _ in range(60):
outputs = model(input_ids=curr_ids)
scores = outputs.logits[:, -1, :].clone().float()
scores[:, forbidden_mask.to("cuda")] = -float("inf")
next_token = torch.multinomial((scores / 0.7).softmax(-1), num_samples=1)
if next_token.item() == tokenizer.eos_token_id:
break
curr_ids = torch.cat([curr_ids, next_token], dim=-1)
print(tokenizer.decode(curr_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip())
Qwen/Qwen3-1.7B