Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find a way to exclude test data when building docker image. #350

Open
alxmrs opened this issue Jun 24, 2023 · 2 comments
Open

Find a way to exclude test data when building docker image. #350

alxmrs opened this issue Jun 24, 2023 · 2 comments

Comments

@alxmrs
Copy link
Collaborator

alxmrs commented Jun 24, 2023

I just noticed something. When trying to build a docker image with a clean-ish branch (no non-checked in files), I got this log trace from Google Cloud Build:

Creating temporary tarball archive of 159 file(s) totalling 190.3 MiB before compression.

I did an experiment: I deleted all the test data folders and re-rand the command to build our docker image:

Creating temporary tarball archive of 124 file(s) totalling 958.1 KiB before compression.

This is quite a huge improvement! In general, I wonder if there's a way where we can not have checked-in test data.

@alxmrs
Copy link
Collaborator Author

alxmrs commented Jun 24, 2023

Since we git clone weather tools in the docker image, we should also find a way to prevent test data being checked in.

https://github.com/google/weather-tools/blob/main/Dockerfile#L29

I think this could make our image more lightweight!

@alxmrs
Copy link
Collaborator Author

alxmrs commented Jun 25, 2023

Here's an idea! Let's replace hosting our own test data by using examples from the Xarray tutorial: https://docs.xarray.dev/en/stable/generated/xarray.tutorial.open_dataset.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant