Using a local model for memory formation and passing it to ChatGPT API

[twardoch]

I'd like to chat tooth the ChatGPT API but use a different local model for memory formation (summarization or entity extraction). The idea is: a small local model (or two) can be very good at specialized tasks like summarization or entity extraction, for which I don't need to use the ChatGPT API credits. But ChatGPT is very good at general knowledge. So I imagine it would be useful to use the ChatGPT API to drive the conversation forward, but use a simpler model for the relatively simpler tasks of entity extraction or summarization. Is this a viable approach, and how could I set it up?

[twardoch]

I see that the Spacy Memory example https://github.com/hwchase17/langchain/blob/master/docs/modules/memory/examples/custom_memory.ipynb is following this idea. So I'd like to ask more specifically: are there models you can recommend that are good for custom memory (entity extraction & summarization) which aren't OpenAI but possibly local or usable via the HF API?