SmolLM2-135M-PHP

A small PHP code-completion model, fine-tuned from SmolLM2-135M. This is a personal learning project.

Intended use

Learning how code models are fine-tuned, evaluated and published.
Simple PHP completion experiments.

Do not use it in production. It can produce wrong, insecure or non-running code, so review everything it generates.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

name = "chellen021/SmolLM2-135M-PHP"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name)

prompt = "<?php\n\nfunction add($a, $b)\n{\n"
inputs = tok(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=80, do_sample=False, repetition_penalty=1.1)
print(tok.decode(out[0], skip_special_tokens=True))

Training data

Source: The Stack v3 (GitHub snapshot up to 2025-08-07; dataset license ODC-By 1.0).

Filters applied:

language is PHP, license_type is permissive, not vendored
file size between 200 bytes and 200 KB, deduplicated by content
removed files whose first 4,000 characters contain "All rights reserved", "proprietary" or similar wording
split into train and validation by repository


Files used	245,130 train, 10,742 validation
Repositories	10,914
Tokens	285,749,021 train, 16,733,639 validation

Code in the source dataset keeps its original licenses. License detection is automatic and can be wrong. If your code should not be here, please open a discussion on this repository.

Training


Hardware	1 Nvidia T4 (Kaggle)
Steps	5,000 (about 14% of one pass over the training data)
Batch	8, gradient accumulation 2, 512 tokens per sample
Learning rate	5e-5, cosine schedule

Evaluation

Bottom line: it has learned to predict PHP text better, but there is no evidence that it writes more correct code than the base model.

Validation loss, measured on 2,000 blocks from held-out repositories:

	Base model	This model
Validation loss	1.242	1.158
Perplexity	3.46	3.18

Code generation test on 50 fixed prompts. Generation was limited to 400 new tokens with a repetition penalty of 1.1, and each output was cut at the point where all brackets closed. 20 prompts ask for a function that is run against test cases. All 20 reference solutions passed those tests.

Greedy, 1 output per prompt	Base model	This model	95% CI of difference
Brackets closed (50 prompts)	44	46	-6% to +16%
`php -l` passes (50 prompts)	35	39	-4% to +20%
Function tests pass (20 prompts)	4	3	-15% to 0%

Sampled, temperature 0.7, 3 outputs per prompt	Base model	This model	95% CI of difference
Brackets closed	82.7%	82.0%	-8.7% to +6.7%
`php -l` passes	61.3%	63.3%	-8.7% to +12.7%
Function tests pass	8.3%	11.7%	-3.3% to +10.0%

Every confidence interval (bootstrap over prompts) includes zero, so none of these differences is established. The prompts were written for this project and are not a standard benchmark.

Limitations

The training set is small and only covers code with detected permissive licenses, so it is skewed toward certain repositories and licenses.
It knows nothing after August 2025.
The source data may contain malicious code or personal data missed by redaction.
It can repeat training code. Check licensing before reusing output.
It is a completion model, not a chat assistant.
Most function tests fail, for both this model and the base model.

Citation

@misc{lozhkov2026stack-v3,
    author       = {Lozhkov, Anton and Larcher, Hugo and Morlon, Mathieu and Ben Allal, Loubna and von Werra, Leandro},
    title        = {The Stack v3: The Largest Open Code Dataset},
    year         = 2026,
    url          = {https://huggingface.co/datasets/HuggingFaceCode/stack-v3-train},
    publisher    = {Hugging Face}
}

加载中…