Model Card for BERT AI Detector

Model Details

Model Description

This model is a fine-tuned BERT designed to classify text as either AI-generated or human-written. The model was trained on data from the Kaggle LLM Detect competition using variable-length text inputs ranging from 5 to 100 words. The fine-tuned model achieves high accuracy in identifying the source of the text, making it a valuable tool for detecting AI-generated content.

• Developed by: Pritam
• Language(s) (NLP): English
• License: Apache 2.0
• Finetuned from model: BERT (base-uncased)

Model Sources

• Repository: Hugging Face Model Card
• Demo: Streamlit Interface

Uses

Direct Use

The model is intended for use in detecting whether text is AI-generated or human-written. Users can input text snippets into the demo or directly integrate the model into their applications for automated content classification.

Downstream Use

Potential downstream uses include:

• Moderating AI-generated content in online platforms.
• Academic and journalistic content verification.
• Detecting plagiarism or misuse of AI writing tools.

Out-of-Scope Use

The model is not suitable for:

• Detecting deeply paraphrased AI-generated text.
• Analysis of languages other than English.
• Scenarios where fairness and bias considerations are critical, as those have not been explicitly addressed.

Bias, Risks, and Limitations

Recommendations

Users should be aware that:

• The model may not perform well on text heavily modified from AI-generated content.
• It may produce false positives or false negatives due to the inherent limitations of the dataset or model architecture.

How to Get Started with the Model

Use the following code snippet to load the model:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("pritamdeb68/BERTAIDetector")
model = AutoModelForSequenceClassification.from_pretrained("pritamdeb68/BERTAIDetector")

text = "Your text here"
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
predictions = outputs.logits.argmax(dim=1).item()
print("AI-generated" if predictions == 1 else "Human-written")

Training Details

Training Data

The training dataset was sourced from the Kaggle LLM Detect competition. The data includes examples of both AI-generated and human-written text, spanning various input lengths (5-100 words).

Training Procedure

Preprocessing

• Text was tokenized using BERT's tokenizer.
• Input lengths ranged between 5 and 100 words, padded or truncated as necessary.

Training Hyperparameters

• Batch Size: 300
• Optimizer: AdamW
• Learning Rate: 1e-5
• Epochs: 1

Speeds, Sizes, Times

• Training Time: 1 hour 10 minutes
• Hardware Used: GPU (Kaggle T4 x 2)
• Loss: 0.12 on train data

Evaluation

Testing Data, Factors & Metrics

Testing Data

Validation data from the Kaggle competition was used for evaluation.

Metrics

• Accuracy: 96.65% on validation data.

Results

The model achieved high accuracy and low validation loss, demonstrating its effectiveness for the task of AI text detection.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator:

• Hardware Type: Kaggle T4 (x2) GPU
• Training Duration: 1 hour 10 minutes
• Compute Region: Not specified

Technical Specifications

Model Architecture and Objective

• Model Architecture: BERT (base-uncased) fine-tuned for text classification.
• Objective: Binary classification of text into AI-generated or human-written categories.

Compute Infrastructure

Hardware

• Type: Kaggle T4(x2) GPU

Software

• Framework: PyTorch with Transformers library

Citation

If you use this model, please cite the repository:

@inproceedings{pritam2024bertaidetector,
  title={BERT AI Detector},
  author={Pritam},
  year={2024},
  url={https://huggingface.co/pritam2014/BERTAIDetector}
}