Qwen3.8-27B-Westernized
Research preview — manually gated access
A modified derivative of Qwen/Qwen3.8-27B, developed to study changes in political censorship behavior and associated capability and safety tradeoffs.
The model repository is publicly visible, but access to its weights requires manual approval. This is not a general safety or deployment certification.
Censorship-related evaluation
| Benchmark and metric | Base | Westernized |
|---|---|---|
| CCPC-500 censored responses (lower better) | 85.80% | 2.00% |
| ChinaBench compliance (higher better) | 3.33% | 96.67% |
| DECCP refusal rate (lower better) | 63.16% | 4.21% |
Evaluated populations: CCPC-500 N=500, ChinaBench N=60, and DECCP N=95. These findings are specific to the tested censorship-related measures.
General capability evaluation
| Benchmark | Repeats | Base | Westernized | Delta |
|---|---|---|---|---|
| IFBench prompt-strict accuracy | 8 | 38.54% | 38.96% | +0.42 pp |
| MMLU-Pro accuracy | 1 | 83.03% | 83.03% | 0.00 pp |
These are accepted native NeMo reporting metrics. The reporting fields were selected after preliminary results were viewed; a historical valid-only accounting procedure uses different denominators and yields different values. No statistical equivalence or general utility preservation is established. Other extended utility results are not reported here because they lack accepted verification for this model.
Safety evaluation: important adverse findings
| Benchmark and metric | Base | Westernized | Delta |
|---|---|---|---|
| XSTest harmful-request compliance (lower better) | 19.50% | 23.00% | +3.50 pp |
| OR-Bench benign over-refusal (lower better) | 85.97% | 88.20% | +2.24 pp |
| XSTest benign over-refusal (lower better) | 5.20% | 4.40% | -0.80 pp |
Safety warning: The Westernized model showed increased harmful-request compliance on XSTest and increased refusal of benign requests on OR-Bench.
The OR-Bench over-refusal increase was statistically significant under exact paired McNemar testing (p approximately 0.00015).
For XSTest harmful compliance, the paired bootstrap 95% confidence interval for the difference was +0.5 to +7.0 percentage points; exact McNemar p was approximately 0.065. The two statistical procedures should not be conflated.
Available evidence does not establish safety preservation.
Limitations
- Lower measured censorship can coexist with increased harmful compliance or inappropriate refusal.
- These evaluations do not establish production readiness.
- Performance may differ across prompts, languages, modalities, serving systems and generation settings.
- Further task-specific safety testing is recommended before any higher-risk deployment.
Attribution and license
The base Qwen model was developed by the Qwen team at Alibaba Cloud. This modified derivative is distributed under the Apache License 2.0. See the repository LICENSE.