中文Sign up

vidore/ColSmolVLM-Instruct-256M-base

🤗 Hugging Face sourcemit228M params456 MBsafetensors✓ 1 checksumupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo vidore/ColSmolVLM-Instruct-256M-base ./model-folder
Needs a seeder →

ColSmolVLM-256M-Instruct: Visual Retriever based on SmolVLM-256M-Instruct with ColBERT strategy

ColSmolVLM is a model based on a novel model architecture and training strategy based on Vision Language Models (VLMs) to efficiently index documents from their visual features. It is a SmolVLM extension that generates ColBERT- style multi-vector representations of text and images. It was introduced in the paper ColPali: Efficient Document Retrieval with Vision Language Models and first released in this repository

This version is the untrained base version to guarantee deterministic projection layer initialization.

Usage

[!WARNING] This version should not be used: it is solely the base version useful for deterministic LoRA initialization.

License

ColSmol's vision language backbone model (ColSmolVLM) is under apache2.0 license. The adapters attached to the model are under MIT license.

Contact

Citation

If you use any datasets or models from this organization in your research, please cite the original dataset as follows:

@misc{faysse2024colpaliefficientdocumentretrieval,
  title={ColPali: Efficient Document Retrieval with Vision Language Models}, 
  author={Manuel Faysse and Hugues Sibille and Tony Wu and Bilel Omrani and Gautier Viaud and Céline Hudelot and Pierre Colombo},
  year={2024},
  eprint={2407.01449},
  archivePrefix={arXiv},
  primaryClass={cs.IR},
  url={https://arxiv.org/abs/2407.01449}, 
}