Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make readiness_endpoint liveness_endpoint required to use custom server #1267

Merged

Conversation

tianshuc0731
Copy link
Contributor

@tianshuc0731 tianshuc0731 commented Dec 5, 2024

🚀 What

there will be deployment failures when readiness or liveness are not specified with custom server, I still need to investigate why, this pr is to make those two fields required as a short-term fix

💻 How

🔬 Testing

https://github.com/basetenlabs/truss/actions/runs/12189700386

@tianshuc0731 tianshuc0731 changed the title Make readiness_endpoint liveness_endpoint required Make readiness_endpoint liveness_endpoint required to use custom server Dec 5, 2024
), "docker_server.readiness_endpoint is required to use custom server"
assert (
config.docker_server.liveness_endpoint is not None
), "docker_server.liveness_endpoint is required to use custom server"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should probably be exepctions...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assertions are primarily used as a debugging aid. They are meant to catch programming errors during development by verifying assumptions made by the programmer.

Exceptions are used for handling errors and other "exceptional" conditions that may arise at runtime due to external factors (e.g., invalid user input, file I/O, network errors).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it will change to exceptions in the next pr

@tianshuc0731 tianshuc0731 merged commit 66c3562 into main Dec 5, 2024
12 checks passed
@tianshuc0731 tianshuc0731 deleted the tianshu/custom-server-readiness-liveness-required branch December 5, 2024 23:40
squidarth pushed a commit that referenced this pull request Dec 5, 2024
Co-authored-by: Tianshu Cheng <tianshuc@Tianshus-MBP.localdomain>
joostinyi added a commit that referenced this pull request Dec 10, 2024
* update (#1267)

Co-authored-by: Tianshu Cheng <tianshuc@Tianshus-MBP.localdomain>

* send truss version on patch (#1268)

* Speculative Decoding Interface refactor (#1270)

* spec dec config

* add optional dict of trt llm configs

* fix bad merge

* add extensions support

* fix fixture

* cli push fixes

* constants

* fix ordering

* fix merge

* refactor interface

* add tp validation error

* self review

* use constant

* fix tests

* fix tests

* add request_default_max_tokens

* fix default on trtllm runtime

* update copy

* bump to 54rc0

* add total token limit to toplevel config

* bump briton to 0.3.10

* fix import

* 54rc2

* fix rc3

* rc4

* bump briton server image

* bump rc6 for briton 0.3.12.dev3

* bump rc7

* revert trtllm serialization changes

* bump briton

* interface refactor

* add validation + tests

* 56rc0

* reduce property

* Update trt_llm_config.py (#1274)

* Update trt_llm_config.py -> revision (#1269)

* Better chains error propagation (+various fixes). (#1271)

* Bump briton in truss library (#1273)

* bump briton to briton==0.3.12.dev8

* bump truss to 0.9.56rc1

* Bump version to 0.9.56

---------

Co-authored-by: Tianshu <26018552+tianshuc0731@users.noreply.github.com>
Co-authored-by: Tianshu Cheng <tianshuc@Tianshus-MBP.localdomain>
Co-authored-by: rcano-baseten <raymond.cano@baseten.co>
Co-authored-by: joostinyi <63941848+joostinyi@users.noreply.github.com>
Co-authored-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com>
Co-authored-by: Marius Killinger <155577904+marius-baseten@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants