InsecureErasure/flan-t5-xxl-encoder-only-GGUF
Quantized GGUF version of Google FLAN T5-XXL (encoder only)
This repo contains additional GGUF quantizations of Google's FLAN-T5 XXL (encoder only) not available in silveroxides/flan-t5-xxl-encoder-only-GGUF, produced using llama.cpp.
These are aimed at users with limited hardware — requiring less memory and storage than higher quantization formats. Other repositories on HF include the decoder layers, resulting in a much bigger file. If you're using this for generating embeddings for Chroma or FLUX models (as I am), you only need the encoder layers.
Big kudos to silveroxides.
| Chroma Flash Q5_K_S | ||
| Chroma Flash Q4_K_S | ||
| flan-t5-xxl-TE-only Q4_K_M | flan-t5-xxl-TE-only Q3_K_M |