NewsLens BERTimbau
NewsLens BERTimbau is a Portuguese text classification model fine-tuned for the binary classification of news articles as false or true.
The model is based on BERTimbau, a BERT model pretrained for Brazilian Portuguese, and was developed as part of the NewsLens academic project.
Model Details
- Model: BERTimbau Base
- Base model:
neuralmind/bert-base-portuguese-cased - Language: Brazilian Portuguese
- Task: Binary text classification
- Framework: Transformers / PyTorch
- Classes:
0→ Falsa1→ Verdadeira
- Maximum sequence length: 512 tokens
NewsLens
NewsLens is an academic project focused on the application of Natural Language Processing and Machine Learning techniques to the analysis and classification of fake news in Portuguese.
The project explores linguistic patterns in news articles and compares different machine learning approaches, including:
- Logistic Regression
- Multinomial Naive Bayes
- Linear SVM with TF-IDF
- BERTimbau
The BERTimbau model presented in this repository was selected as the main classifier after the experimental evaluation.
Project repository:
https://github.com/lunecarvalho/newslens-project
Dataset
The model was fine-tuned using news articles derived from the Fake.Br Corpus, a Portuguese-language fake news dataset.
The processed dataset used in the project contains 7,199 news articles, with an approximately balanced distribution between false and true news.
For the final BERTimbau experiment, the dataset was divided using stratified sampling:
| Split | Proportion | Samples |
|---|---|---|
| Training | 80% | 5,759 |
| Validation | 10% | 720 |
| Test | 10% | 720 |
The final test set contained:
- 360 false news articles
- 360 true news articles
The test set was kept separate from model training and checkpoint selection.
Training
The model was fine-tuned for sequence classification using the following main configuration:
| Parameter | Value |
|---|---|
| Epochs | 3 |
| Learning rate | 2e-5 |
| Train batch size | 8 |
| Evaluation batch size | 8 |
| Weight decay | 0.01 |
| Maximum sequence length | 512 |
| Best model metric | F1-score |
| Precision | BF16 |
The best checkpoint was selected according to the F1-score on the validation set.
Evaluation
The final model was evaluated on the independent test set containing 720 news articles.
Overall Performance
| Metric | Score |
|---|---|
| Accuracy | 99.44% |
| F1-score | 99.44% |
Performance by Class
| Class | Precision | Recall | F1-score |
|---|---|---|---|
| False | 99.17% | 99.72% | 99.45% |
| True | 99.72% | 99.17% | 99.44% |
Confusion Matrix
| Predicted False | Predicted True | |
|---|---|---|
| Actual False | 359 | 1 |
| Actual True | 3 | 357 |
The model correctly classified 716 of the 720 samples in the final test set.
Only one false news article was classified as true.
Comparison with Linear SVM
BERTimbau was also compared with a Linear SVM model using TF-IDF features on the same final test set.
| Model | Accuracy | Precision | Recall | F1-score |
|---|---|---|---|---|
| Linear SVM | 92.22% | 91.30% | 93.33% | 92.31% |
| BERTimbau | 99.44% | 99.72% | 99.17% | 99.44% |
Precision, Recall and F1-score in this comparison use the
Trueclass as the positive class.
The BERTimbau model produced 4 classification errors, compared with 56 errors from the Linear SVM on the same test samples.
Usage
The model can be loaded directly with the Hugging Face Transformers library:
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_name = "lunecarvalho/newslens-bertimbau"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(
model_name
)
text = """
Insira aqui o texto da notícia que deseja analisar.
"""
inputs = tokenizer(
text,
return_tensors="pt",
truncation=True,
max_length=512
)
with torch.no_grad():
outputs = model(**inputs)
prediction = torch.argmax(
outputs.logits,
dim=-1
).item()
labels = {
0: "Falsa",
1: "Verdadeira"
}
print(labels[prediction])
Intended Use
This model was developed primarily for:
- academic research;
- experimentation with NLP techniques;
- study of fake news classification in Brazilian Portuguese;
- development of the NewsLens prototype.
It can be used as a component of systems that analyze linguistic patterns in news articles.
Limitations
The high performance reported for this model refers specifically to the test partition derived from the dataset used in the NewsLens experiment.
It should not be interpreted as a 99.44% accuracy rate for arbitrary news articles from the internet.
During the project, exploratory analysis indicated that the corpus contains lexical, stylistic, temporal, institutional, and source-related patterns that may contribute to the separation between false and true news.
Therefore:
- performance may decrease on news from other sources or time periods;
- the model may learn dataset-specific patterns;
- predictions should not be treated as definitive fact-checking;
- the model does not verify claims against external evidence or trusted sources;
- external validation is recommended before using the model in real-world scenarios.
The model should be considered an NLP classification tool, not a replacement for professional fact-checking.
Future Work
Future development of NewsLens may include:
- evaluation on external fake news datasets;
- testing with recent news articles;
- analysis of model errors and biases;
- model explainability;
- probability calibration;
- integration with the NewsLens web application;
- comparison with additional Transformer architectures.
Author
Developed by Lune Carvalho as part of the NewsLens academic project.
GitHub: https://github.com/lunecarvalho
Hugging Face: https://huggingface.co/lunecarvalho
Acknowledgements
This project uses BERTimbau, a BERT model pretrained for Brazilian Portuguese by NeuralMind.
Base model:
https://huggingface.co/neuralmind/bert-base-portuguese-cased