Intended usage questions

[mfcabrera]

• RecList version: 2.1.0
• Python version: 3.10.12
• Operating System: Linux 1014-150306-c0g1jekg-10-146-237-11 5.15.0-1070-aws #76~20.04.1-Ubuntu SMP Mon Sep 2 12:20:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Description

Hi,

First, thanks a lot for putting this package together and sharing with the community. I am relatively new to the RecSys development so I might be asking the wrong questions here.

• From the API or usage examples, you define "test" that actually not tests because they return a value not check for any minimum value. Where should the checks go? (I am making the parallel with uni tests here).

• From the article https://towardsdatascience.com/ndcg-is-not-all-you-need-24eb6d2f1227 :

As a CI/CD check for production: after training a model, you typically run a quantitative benchmark on a test set to ensure some accuracy level before deployment. With RecList, you can supplement your pipeline with behavioral tests as well, and decide which flags to raise / what to do when results are not what is expected.

How do you obtain the values of the "tests" to make those checks? Right not it seems you have to go private propety
self._test_results or create a Logger to write the float values whatever you want (the non float results from the tests are filtered out). How is this intended to be used?

Thanks in advance for the help!

[jacopotagliabue]

Hi @mfcabrera thanks for your message - sorry for being a bit late to the party, but I'm traveling quite a bit these days. I'll be back home in a week or so and I will send over some snippets.

[mfcabrera]

@jacopotagliabue thanks for replying! Looking forward to your answer!