Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Camouflage Mocks to Support gRPC #437

Draft
wants to merge 3 commits into
base: branch-23.11
Choose a base branch
from

Conversation

mdemoret-nv
Copy link
Contributor

This PR adds mocks to Camouflage to support mocking the Triton gRPC endpoint. Currently its a pain to set up testing because we much use MagicMock for mocking Triton with Python and Camouflage when mocking with C++. This would unify them to both use Camouflage.

Some outstanding issues:

  • This requires setting the request id field. Are we sure this wont have any side effects?
  • What is a good way to update these payload files if they change in the future?
  • What organization should we use for the payload files. Its not as clear as the HTTP folder structure
  • Can we use the same payload for both HTTP and gRPC?
  • The request ID solution is a bit hacky for HTTP requiring some weird tricks. Are we ok with that?

@dagardner-nv
Copy link
Contributor

This requires setting the request id field. Are we sure this wont have any side effects?
I think in this case it works as it is something built-in to the Triton client/server, and offers the ability to perform some sanity checks on the responses. This does mean we could make the launch_mock_triton a session scoped fixture and only need to launch it once.

@mdemoret-nv mdemoret-nv changed the base branch from branch-22.11 to branch-23.11 July 14, 2023 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

3 participants