evaluate() needs optional delay to prevent rate limiting by API

[bdytx5]

Describe the Feature
see title
Why is the feature important for you?
avoid rate limiting
Additional context
no

[jjmachan]

we actually have something for that - could you check out https://docs.ragas.io/en/stable/howtos/customizations/_run_config/ and let me know if it is working 🙂

[bdytx5]

ah ok thanks. Are these ""Exception raised in Job[200]: TimeoutError()
Exception raised in Job[198]: TimeoutError()"" self recovering, or is this saying it will skip over the data?

[jjmachan]

it will skip over

check the raise_exceptions flag if you have doubts 🙂

[bdytx5]

I just mean it as a question, do these try the call again, or just skip them?

[jjmachan]

edited

[bdytx5]

Ah ok. By default, do you think it should try again? Skipping over examples due to rate limiting seems to skew the results of the entire evaluation. Do you think it should wait to try again?