Using llama.cpp for GGUF conversion.
Original model: HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1
Run them directly with llama.cpp:
./llama-embedding \
--batch-size 512 \
--ctx-size 512 \
-m KaLM-embedding-multilingual-mini-instruct-v1-GGUF/model.f32.gguf \
--pooling mean \
-p "this is a test sentence for llama cpp"
It is important to note that this model uses the mean pooling method, so the --pooling parameter needs to be specified as mean when invoking it.
Our tests on LM Studio have not yet been successful, and it is unclear whether this is related to the default pooling method used by LM Studio.
If any developers are familiar with how to specify the pooling method for embedding models in LM Studio, we welcome you to contact us for further discussion via the email: yanshek.woo@gmail.com