ConvNeXt-Food-CLF-75

Multi‑label food ingredient classifier (75 classes) based on ConvNeXt‑Tiny with CBAM attention and GeM pooling.
Trained on a heavily cleaned subset of MM‑Food‑100K.
The model serves as a semantic guide for a downstream segmentation pipeline, but can also be used standalone for ingredient recognition.

📊 Key Metrics: Macro F1 = 0.6546 | mAP = 0.7142 | Validation Loss = 0.0489

💡 Full source code, training scripts, and detailed dataset preparation are available in the GitHub repository.
This model card focuses on how to use the model. For deeper technical details, see the links throughout or visit the GitHub repo.

---

Model Description

ConvNeXt-Food-CLF-75 takes an RGB food image and predicts which of 75 ingredients it contains.
Each prediction is a multi‑label probability – a dish can contain multiple ingredients simultaneously (e.g., “rice” and “chicken”).

• Input: RGB image (any size, automatically resized to 640×640 and normalized with ImageNet statistics)
• Output:
  • A list of ingredient names (default threshold 0.5)
  • Raw probability scores for each of the 75 classes
  • Optional: feature embedding from the penultimate layer (for transfer learning)

Supported Classes (75 ingredients)

---

Intended Uses

• Food ingredient classification into 75 categories independently or for downstream systems (calorie estimation, dietary logging, recipe retrieval).
• Feature extraction: the embedding before the final layer can be fed into a segmentation network to condition mask prediction.
• Research on multi‑label classification with long‑tail distributions and noisy training data.

---

Bias, Risks, and Limitations

• Dataset bias: Trained on a cleaned subset of MM‑Food‑100K. It may underrepresent cuisines not well covered in that dataset.
• Class imbalance: The smallest classes contain as few as ~100 training images; performance may be lower for those ingredients.
• Ambiguity: Visually similar dishes (e.g., “apple” vs. “peach”) may be confused.
• Hallucinations: The model can occasionally predict ingredients that are not visible, especially when cues are subtle or missing.

Recommendations

• Always use the provided preprocessing pipeline (resize to 640×640, ImageNet normalization).
• Consider using Test‑Time Augmentation (TTA) for critical applications.
• Validate predictions against your own data distribution if you plan to deploy the model.

Out‑of‑Scope Use

• Not intended for medical diagnosis or dietary prescription.
• Not a zero‑shot classifier – it only covers the 75 classes listed above.

---

Training Procedure

Full training logs, hyperparameters, and stage‑by‑stage breakdown are in the GitHub model documentation.

Model Architecture

Training Configuration

Training Stages

1. Feature Extraction (28 epochs): Only the classification head is trained, backbone frozen.
2. Discriminative Fine‑Tuning (8 epochs): Head + stages 2‑3 of backbone unfrozen, lower LR for backbone.
3. Staged Unfreezing (1 epoch): Head is trained again to adapt to newly unfrozen features.
4. Final Feature Extraction (5 epochs): Only the classification head is trained at a low LR (5e‑5).

Stage‑by‑Stage Results

---

Evaluation Results (Final Model)

Validation set: 2,000 fixed images, split before training.

---

Hardware and Training Time

Despite the limited 6 GB VRAM, the model trains comfortably with a batch size of 10 at 640×640 resolution.

---

Dataset

The model was trained on a cleaned subset of MM‑Food‑100K, a large‑scale food dataset with noisy 4,000+ ingredient labels.
A rigorous multi‑step cleaning pipeline reduced the original chaos to 75 balanced classes:

• Manually curated target class list (103 initial classes).
• Extracted all matching ingredient names from the dataset.
• Merged semantically identical ingredients (e.g., “enoki mushrooms” + “shiitake mushrooms” → “mushroom”).
• Removed irrelevant matches (e.g., “cheesecake” for the “cheese” class).
• Filtered out classes with fewer than 100 images.
• Created multi‑hot label vectors and serialized the final dataset.

Result: ~88k images (86.3k train, 2k validation), 75 classes.

🔗 The complete cleaning pipeline with all scripts and documentation is here:
dataset_preparation/ on GitHub

---

How to Use

👉 Try it live on Spaces - no installation required!

Quick inference using the provided script (recommended)

The repository includes a self‑contained inference.py script. First, clone this HF repo or download the files, then:

Linux / MacOS:

git clone https://huggingface.co/Alas-V/ConvNeXt-Food-CLF-75
cd ConvNeXt-Food-CLF-75
python -m venv venv
source venv/bin/activate
pip install -r requirements-inference.txt
python -m model.inference --image path/to/meal.jpg --checkpoint pytorch_model.bin --mode text

for Windows

git clone https://huggingface.co/Alas-V/ConvNeXt-Food-CLF-75
cd ConvNeXt-Food-CLF-75
python -m venv venv
.\venv\Scripts\Activate
pip install -r requirements-inference.txt
python -m model.inference --image path/to/photo.jpg --checkpoint checkpoint.pth.tar

Qualitative Examples

Correct Predictions

Challenging Cases (Hallucinations / Missing Ingredients)

---

🙏 Acknowledgements

MM‑Food‑100K for the original dataset.

timm for the ConvNeXt backbone.

Albumentations for image augmentations.

---

Citation

If you use this model or the cleaned dataset in your work, please cite it as follows:

@misc{alasv2026convnextfoodclf75,
  author       = {Artem V.},
  title        = {ConvNeXt-Food-CLF-75: A Multi-Label Food Ingredient Classifier},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Alas-V/ConvNeXt-Food-CLF-75}},
  note         = {Model trained on a cleaned subset of MM-Food-100K}
}

---

📄 License

This project is licensed under the Apache 2.0 License. See the LICENSE file for details.