Hugging face embedding model intfloat/e5-small-v2 (Sentence Transformer) to IR with NNCF optimised

[AmanRavan]

I am trying some hands on OPENVINO.

Not able to convert intfloat/e5-small-v2 from Hugging face to IR with NNCF optimised for faster execution.

I also took reference from Example From OpenVino

or any other way how I can convert my embedding model to IR optimised with NNCF using openvino.

[RitikaxShakya]

Hello @AmanRavan ! Did you try https://github.com/openvinotoolkit/openvino.genai/tree/master/text_generation/causal_lm/cpp this first? Maybe first try this example model "TinyLlama-1.1B-Chat-v1.0" with this reference.

[AmanRavan]

Thanks,

Yup, that's my point.
I am able to use LLM in openvino but not the Embedding model(text-embedding-ada-xxx or some open source).

Meanwhile..Just diving deeper

I was going through this #254llmragopenvino

&&&

there was one line from ov_embedding_model import OVEmbeddings and one more thing what happened to embedding model.

Any idea @RitikaxShakya, suggestions on embedding model running in openvino.