New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modify benchmarking tests and set up github workflows to auto-run benchmarks on any PRs #84

Open

danXyu opened this issue Oct 9, 2024 · 0 comments

Contributor

danXyu commented Oct 9, 2024

Requires setting up good testing benchmarks for each of the agents that run as pytests.
Then we can set up a github workflow to automatically run these benchmarks on PRs.
Will ensure that any changes we make, whether that's on the prompts, the LLM model, or anything, don't affect the overall experience / the LLM can still delegate to the appropriate agents.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment