-
Notifications
You must be signed in to change notification settings - Fork 33
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Refactor how download_remote_files handles cache directories. (#200)
A few improvements for local workflows: * Allow passing the cache directory via either an environment variable or a CLI flag * Delete preexisting files before writing out a symlink to fix `FileExistsError: [WinError 183] Cannot create a file when that file already exists` on Windows * Pull code into a helper function so it can be shared between Azure file downloading and generic (GCS/Huggingface/etc.) file downloading * Update a few variable names and docstrings
- Loading branch information
Showing
1 changed file
with
93 additions
and
44 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters