feat(app): use crc service #1483

olevski · 2023-05-15T23:01:50Z

Uses the compute resource access control service for launching sessions.

RenkuBot · 2023-05-25T14:35:07Z

You can access the deployment of this PR at https://renku-ci-nb-1483.dev.renku.ch

Dockerfile

Panaetius

thank you! nice to see this taking shape 🙂

Dockerfile

Dockerfile.tests

git_services/Dockerfile.init

git_services/Dockerfile.sidecar

Panaetius · 2023-07-04T08:57:10Z

renku_notebooks/api/classes/crc.py

+        # Memory and disk space in CRC are assumed to be in gigabytes whereas
+        # the notebook service assumes that if a plain number is used then it is bytes.


I'm a little confused here. storage comes directly from LaunchNotebookRequestWithoutS3/launch_notebook(), so isn't it in bytes? but above, we compare it directly to max_storage, which should be in gigabytes? Shouldn't there be a conversion somewhere?

So the comment is not right. I updated the schema for the request in launch_notebook to be in gigabytes. But I did not update the comment. Now I added the assumption that disk and memory are in gigabytes in the docstring.

renku_notebooks/api/classes/crc.py

Panaetius · 2023-07-04T09:01:55Z

renku_notebooks/api/classes/crc.py

+        headers = None
+        if user.access_token is not None:
+            headers = {"Authorization": f"bearer {user.access_token}"}
+        res = requests.get(self.crc_url + "/resource_pools", headers=headers)
+        if res.status_code != 200:
+            raise IntermittentError(
+                message="The compute resource access control service sent "
+                "an unexpected response, please try again later",
+            )
+        resource_pools = res.json()


This code is duplicated three times (here and above), maybe it should be its own method?

Panaetius · 2023-07-04T09:08:55Z

renku_notebooks/api/classes/server.py

-                )
-            else:
-                resources["limits"]["cpu"] = cpu_request
+            resources["limits"]["cpu"] = 3 * cpu_request


What's the 3 here?

This is to allow the cpu to spike up momentarily. Because if we set the limit on the cpu it gets throttled and then when you create a new session it takes a really long time for the session to start up. If the limits are set to "lax" (i.e. "relaxed") then the limit is larger than the request. But if the limits are set to strict then the limit is the same as the request and the cpu is throttled.

makes sense. you should give it a descriptive name, random magic numbers in code aren't nice 🙂

renku_notebooks/api/schemas/server_options.py

olevski · 2023-07-04T22:49:06Z

@Panaetius I addressed all the comments. The integration tests are failing because it seems that the oauth application used by the tests has been removed from gitlab. I will fix this later. I ran the integration tests locally and they all passed.

olevski requested a review from a team as a code owner May 15, 2023 23:01

olevski marked this pull request as draft May 15, 2023 23:01

olevski force-pushed the feat-work-with-crac branch 2 times, most recently from 999ea22 to 396d5c5 Compare May 15, 2023 23:08

olevski mentioned this pull request May 15, 2023

chore: 0.31.0 release with crc service SwissDataScienceCenter/renku#3067

Merged

olevski force-pushed the feat-work-with-crac branch from 396d5c5 to fc534a5 Compare May 16, 2023 00:38

olevski added 2 commits May 16, 2023 09:49

feat(app): use crac service

62bd8c6

squashme: switch to debian for faster builds

40a568c

olevski force-pushed the feat-work-with-crac branch from fc534a5 to 40a568c Compare May 16, 2023 13:09

squashme: minor fixes

2e5418f

olevski force-pushed the feat-work-with-crac branch 5 times, most recently from 0560372 to 9fb8044 Compare May 16, 2023 15:34

squashme: fix docker builds

b29012e

olevski force-pushed the feat-work-with-crac branch from 9fb8044 to b29012e Compare May 16, 2023 16:20

squashme: remove curl

20e2ba8

olevski force-pushed the feat-work-with-crac branch from 101928b to 894420a Compare May 16, 2023 17:15

olevski force-pushed the feat-work-with-crac branch 2 times, most recently from 00719ca to 6f99453 Compare May 25, 2023 13:10

olevski had a problem deploying to renku-ci-nb-1483 May 25, 2023 14:26 — with GitHub Actions Error

olevski deployed to renku-ci-nb-1483 May 25, 2023 14:29 — with GitHub Actions Active

olevski force-pushed the feat-work-with-crac branch from 6f99453 to 6f399d3 Compare May 25, 2023 17:35

squashme: fix tests

a13623b

olevski force-pushed the feat-work-with-crac branch from 6f399d3 to a13623b Compare May 25, 2023 17:36

squashme: fix linting

9277024

olevski force-pushed the feat-work-with-crac branch from 9887ed8 to 9277024 Compare May 25, 2023 18:04

squashme: put back server options

65bbfc7

olevski force-pushed the feat-work-with-crac branch from 6766199 to 65bbfc7 Compare June 5, 2023 12:09

olevski added 3 commits June 5, 2023 17:15

fix: properly render crac service url

fb25fcf

squashme: add priority class in amalthea patches

99db2b3

squashme: properly set crac service name

3808ad8

olevski force-pushed the feat-work-with-crac branch from 4da21c8 to 3808ad8 Compare June 6, 2023 09:29

olevski added 2 commits June 6, 2023 15:32

squashme: renku crac to crc

2c9d524

chore: rename crac to crc

74471de

olevski changed the title ~~feat(app): use crac service~~ feat(app): use crc service Jun 6, 2023

olevski added 3 commits June 16, 2023 14:41

chore: surface resource quota errors

dbc9bef

fix: show storage

c065fa3

feat: upgrade amalthea to 0.7.0

7503b2c

rokroskar reviewed Jun 30, 2023

View reviewed changes

Dockerfile Show resolved Hide resolved

Merge branch 'master' into feat-work-with-crac

5b0f017

olevski marked this pull request as ready for review July 4, 2023 08:05

olevski requested a review from a team as a code owner July 4, 2023 08:05

Panaetius requested changes Jul 4, 2023

View reviewed changes

olevski added 2 commits July 4, 2023 17:03

squashme: address comments

26005cf

chore: update renku cli version

05c688d

olevski requested review from rokroskar and Panaetius July 4, 2023 22:48

squashme: address comments

2635dde

Panaetius previously approved these changes Jul 5, 2023

View reviewed changes

olevski dismissed Panaetius’s stale review via cd698a3 July 5, 2023 11:05

squashme: fix tests

b84aa32

olevski force-pushed the feat-work-with-crac branch from cd698a3 to b84aa32 Compare July 5, 2023 11:29

olevski requested a review from Panaetius July 5, 2023 13:58

Panaetius approved these changes Jul 5, 2023

View reviewed changes

olevski merged commit 44ad3dd into master Jul 5, 2023

olevski deleted the feat-work-with-crac branch July 5, 2023 14:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(app): use crc service #1483

feat(app): use crc service #1483

olevski commented May 15, 2023 •

edited

Loading

RenkuBot commented May 25, 2023

Panaetius left a comment

Panaetius Jul 4, 2023

olevski Jul 4, 2023

Panaetius Jul 4, 2023

Panaetius Jul 4, 2023

olevski Jul 4, 2023

Panaetius Jul 5, 2023

olevski commented Jul 4, 2023

		# Memory and disk space in CRC are assumed to be in gigabytes whereas
		# the notebook service assumes that if a plain number is used then it is bytes.

feat(app): use crc service #1483

feat(app): use crc service #1483

Conversation

olevski commented May 15, 2023 • edited Loading

RenkuBot commented May 25, 2023

Panaetius left a comment

Choose a reason for hiding this comment

Panaetius Jul 4, 2023

Choose a reason for hiding this comment

olevski Jul 4, 2023

Choose a reason for hiding this comment

Panaetius Jul 4, 2023

Choose a reason for hiding this comment

Panaetius Jul 4, 2023

Choose a reason for hiding this comment

olevski Jul 4, 2023

Choose a reason for hiding this comment

Panaetius Jul 5, 2023

Choose a reason for hiding this comment

olevski commented Jul 4, 2023

olevski commented May 15, 2023 •

edited

Loading