Large files cause freeze when embedding

[rjmacarthy]

Describe the bug
Large files cause freeze when embedding

To Reproduce
Embed a file > 500 lines or so.

Expected behavior
It should work

Screenshots
na

Logging

API Provider
Ollama

Chat or Auto Complete?
Embedding

Model Name
All models

Desktop (please complete the following information):

• OS: Ubuntu

Additional context
Nothing else

[gpinkham]

I was noticing this but I also am noticing that I can't seem to get Ollama to keep the embeddings model in memory. 4 mins into the embedding process the model disappears and the Twinny process hangs there. I thought it was large files but I'm now thinking its the model being unloaded. I have set the env variable to tell it to keep models in memory for 60m but that hasn't impacted it. (guessing I didn't set it properly).