Is there a way to keep representative documents after merging models?

[rociortizb]

I've been training a model for a few weeks now, updating it with novel topics as new data comes in (by training on the new data and using the merge_models function). I'd like to do some analysis now but when trying to access the representative documents for the topics I realized that they are empty, and upon investigating a bit I found that they are not saved after merging models "because of privacy reasons".

I understand how this is useful, however for my use case I'd like to actually keep them, which brings me to the question: is there a way to change the default settings such that the representative documents are kept even after merging the models?

[MaartenGr]

That is currently not possible since merge models was also created for federated learning, where privacy is of the utmost concern.

Although that is not possible with public functions, you could use the private ._extract_representative_docs function instead to recalculate the representative documents. It would require you to have access to all documents though.

I believe there are a couple of issues that describe how to use it.

[rociortizb]

Thought this may be the case, but worth asking anyway. Don't really have access to the training documents anymore but will try to reproduce the ._extract_representative_docs function with sufficient new documents to see if I can get good representative documents for each topic. Thank you for your answer! :)