Add --test option to specify test to run #368
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
wanchaol
approved these changes
May 29, 2024
test_runner.py
Outdated
| @@ -29,11 +29,12 @@ class OverrideDefinitions: | |||
|
|
|||
| override_args: Sequence[Sequence[str]] = tuple(tuple(" ")) | |||
| test_descr: str = "default" | |||
| test_id: str = "default" | |||
There was a problem hiding this comment.
This feels like a test_name rather than test_id? could we rename to test_name?
| @@ -244,7 +240,7 @@ def run_test(test_flavor: OverrideDefinitions, full_path: str): | |||
|
|
|||
|
|
|||
| def run_tests(args): | |||
| integration_tests_flavors = build_test_list(args) | |||
| integration_tests_flavors = build_test_list() | |||
There was a problem hiding this comment.
why we remove the args here, is it not getting used?
| @@ -255,13 +251,19 @@ def run_tests(args): | |||
| ) | |||
| if is_integration_test: | |||
| for test_flavor in integration_tests_flavors[config_file]: | |||
| run_test(test_flavor, full_path) | |||
| if args.test == "all" or test_flavor.test_id == args.test: | |||
| run_test(test_flavor, full_path, args.output_dir) | |||
There was a problem hiding this comment.
I think we should still assert the output_dir specified, otherwise we should error out
There was a problem hiding this comment.
output_dir is a required argument to the python program:
parser.add_argument("output_dir")
If not provided, the program entry would fail:
usage: test_runner.py [-h] [--config_dir CONFIG_DIR] [--test TEST] output_dir
test_runner.py: error: the following arguments are required: output_dir
tianyu-l
pushed a commit
to tianyu-l/torchtitan_intern24
that referenced
this pull request
Aug 16, 2024
Usage: `--test <test_id>` Acceptable values: `test_id` in `build_test_list` (default: all) Example: ``` rm -rf outputs && python test_runner.py outputs --test pp_gpipe ```
philippguevorguian
pushed a commit
to YerevaNN/YNNtitan
that referenced
this pull request
Aug 17, 2024
Usage: `--test <test_id>` Acceptable values: `test_id` in `build_test_list` (default: all) Example: ``` rm -rf outputs && python test_runner.py outputs --test pp_gpipe ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Usage:
--test <test_id>Acceptable values:
test_idinbuild_test_list(default: all)Example: