Skip to content

Commit

Permalink
✨ Add d command for setting/clearing the test seed (#112)
Browse files Browse the repository at this point in the history
  • Loading branch information
randycoulman authored Sep 19, 2024
1 parent ce32cbb commit aa6a2b5
Show file tree
Hide file tree
Showing 10 changed files with 285 additions and 116 deletions.
104 changes: 68 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,28 @@
# mix test.interactive

[![Build Status](https://github.com/randycoulman/mix_test_interactive/actions/workflows/ci.yml/badge.svg)](https://github.com/randycoulman/mix_test_interactive/actions)
[![Module Version](https://img.shields.io/hexpm/v/mix_test_interactive.svg)](https://hex.pm/packages/mix_test_interactive)
[![Hex Docs](https://img.shields.io/badge/hex-docs-lightgreen.svg)](https://hexdocs.pm/mix_test_interactive/)
[![Build
Status](https://github.com/randycoulman/mix_test_interactive/actions/workflows/ci.yml/badge.svg)](https://github.com/randycoulman/mix_test_interactive/actions)
[![Module
Version](https://img.shields.io/hexpm/v/mix_test_interactive.svg)](https://hex.pm/packages/mix_test_interactive)
[![Hex
Docs](https://img.shields.io/badge/hex-docs-lightgreen.svg)](https://hexdocs.pm/mix_test_interactive/)
[![License](https://img.shields.io/hexpm/l/mix_test_interactive.svg)](https://github.com/randycoulman/mix_test_interactive/blob/master/LICENSE.md)

`mix test.interactive` is an interactive test runner for ExUnit tests.

Based on Louis Pilfold's wonderful [mix-test.watch](https://github.com/lpil/mix-test.watch) and inspired by Jest's interactive watch mode, `mix test.interactive` allows you to dynamically change which tests should be run with a few keystrokes.
Based on Louis Pilfold's wonderful
[mix-test.watch](https://github.com/lpil/mix-test.watch) and inspired by Jest's
interactive watch mode, `mix test.interactive` allows you to dynamically change
which tests should be run with a few keystrokes.

It allows you to easily switch between running all tests, stale tests, or failed tests. Or, you can run only the tests whose filenames contain a substring. Includes an optional "watch mode" which runs tests after every file change.
It allows you to easily switch between running all tests, stale tests, or failed
tests. Or, you can run only the tests whose filenames contain a substring.
Includes an optional "watch mode" which runs tests after every file change.

## Installation

The package can be installed by adding `mix_test_interactive` to your list of dependencies in `mix.exs`:
The package can be installed by adding `mix_test_interactive` to your list of
dependencies in `mix.exs`:

```elixir
def deps do
Expand Down Expand Up @@ -46,9 +55,9 @@ If an option is provided on the command line, it will override the same option
specified in the configuration.

- `--(no-)clear`: Clear the console before each run (default `false`).
- `--command <command> [--arg <arg>]`: Custom command and arguments for
running tests (default: "mix" with no arguments). NOTE: Use `--arg` multiple
times to specify more than one argument.
- `--command <command> [--arg <arg>]`: Custom command and arguments for running
tests (default: "mix" with no arguments). NOTE: Use `--arg` multiple times to
specify more than one argument.
- `--exclude <regex>`: Exclude files/directories from triggering test runs
(default: `["~r/\.#/", "~r{priv/repo/migrations}"`]) NOTE: Use `--exclude`
multiple times to specify more than one regex.
Expand All @@ -61,23 +70,35 @@ specified in the configuration.
(default: `false`).
- `--(no-)watch`: Don't run tests when a file changes (default: `true`).

All of the `<mix test arguments>` are passed through to `mix test` on every
test run.
All of the `<mix test arguments>` are passed through to `mix test` on every test
run.

`mix test.interactive` will detect the `--stale` and `--failed` flags and use those as initial settings in interactive mode. You can then toggle those flags on and off as needed. It will also detect any filename or pattern arguments and use those as initial settings. However, it does not detect any filenames passed with `--include` or `--only`. Note that if you specify a pattern on the command-line, `mix test.interactive` will find all test files matching that pattern and pass those to `mix test` as if you had used the `p` command.
`mix test.interactive` will detect the `--failed`, `--seed`, and `--stale`
options and use those as initial settings in interactive mode. You can then use
the interactive mode commands to adjust those options as needed. It will also
detect any filename or pattern arguments and use those as initial settings.
However, it does not detect any filenames passed with `--include` or `--only`.
Note that if you specify a pattern on the command-line, `mix test.interactive`
will find all test files matching that pattern and pass those to `mix test` as
if you had used the `p` command.

### Patterns and filenames

`mix test.interactive` can take the same filename or filename:line_number
patterns that `mix test` understands. It also allows you to specify one or
more "patterns" - strings that match one or more test files. When you provide
one or more patterns on the command-line, `mix test.interactive` will find all
test files matching those patterns and pass them to `mix test` as if you had
used the `p` command (described below).
patterns that `mix test` understands. It also allows you to specify one or more
"patterns" - strings that match one or more test files. When you provide one or
more patterns on the command-line, `mix test.interactive` will find all test
files matching those patterns and pass them to `mix test` as if you had used the
`p` command (described below).

After the tests run, you can use the interactive mode to change which tests will run.
After the tests run, you can use the interactive mode to change which tests will
run.

Use the `p` command to run only test files that match one or more provided patterns. A pattern is the project-root-relative path to a test file (with or without a line number specification) or a string that matches a portion of full pathname. e.g. `test/my_project/my_test.exs`, `test/my_project/my_test.exs:12:24` or `my`.
Use the `p` command to run only test files that match one or more provided
patterns. A pattern is the project-root-relative path to a test file (with or
without a line number specification) or a string that matches a portion of full
pathname. e.g. `test/my_project/my_test.exs`,
`test/my_project/my_test.exs:12:24` or `my`.

Any patterns that contain a line number specification are passed directly to
`mix test`. Remaining patterns are matched against test filenames as above.
Expand All @@ -86,15 +107,22 @@ Any patterns that contain a line number specification are passed directly to
p pattern1 pattern2
```

Use the `s` command to run only test files that reference modules that have changed since the last run (equivalent to the `--stale` option of `mix test`).
Use the `s` command to run only test files that reference modules that have
changed since the last run (equivalent to the `--stale` option of `mix test`).

Use the `f` command to run only tests that failed on the last run (equivalent to the `--failed` option of `mix test`).
Use the `f` command to run only tests that failed on the last run (equivalent to
the `--failed` option of `mix test`).

Use the `a` command to run all tests.
Use the `a` command to run all tests, turning off the `--failed` and `--stale`
flags as well as clearing any patterns.

Use the `d <seed>` command to run tests with a specific seed, and then use `d`
(with no seed) to remove the seed.

Use the `w` command to turn file-watching mode on or off.

Use the `Enter` key to re-run the current set of tests without requiring a file change.
Use the `Enter` key to re-run the current set of tests without requiring a file
change.

Use the `q` command, or press `Ctrl-D` to exit the program.

Expand Down Expand Up @@ -133,9 +161,9 @@ commands before or after running the tests.

In those cases, you can customize the command that `mix test.interactive` will
use to run your tests. `mix test.interactive` assumes that the custom command
ultimately runs `mix` under the hood (or at least accepts
all of the same command-line arguments as `mix`). The custom command can either be a string or
a `{command, [..args..]}` tuple.
ultimately runs `mix` under the hood (or at least accepts all of the same
command-line arguments as `mix`). The custom command can either be a string or a
`{command, [..args..]}` tuple.

Examples:

Expand Down Expand Up @@ -205,10 +233,10 @@ ignore files with any of these extensions, you can specify an `exclude` regexp
### `runner`: Use a custom runner module

By default `mix test.interactive` uses an internal module named
`MixTestInteractive.PortRunner` to run the tests. If you want to
run the tests in a different way, you can supply your own runner module instead.
Your module must implement a `run/2` function that takes a
`MixTestInteractive.Config` struct and a list of `String.t()` arguments.
`MixTestInteractive.PortRunner` to run the tests. If you want to run the tests
in a different way, you can supply your own runner module instead. Your module
must implement a `run/2` function that takes a `MixTestInteractive.Config`
struct and a list of `String.t()` arguments.

```elixir
# config/config.exs
Expand Down Expand Up @@ -244,8 +272,8 @@ To use a custom command instead, see the `command` option above.

### `timestamp`: Display the current time before running the tests

When `timestamp` is set to true, `mix test.interactive` will display the
current time (UTC) just before running the tests.
When `timestamp` is set to true, `mix test.interactive` will display the current
time (UTC) just before running the tests.

```elixir
# config/config.exs
Expand All @@ -268,13 +296,17 @@ You can enable desktop notifications with

## Acknowledgements

This project started as a clone of the wonderful [mix-test.watch](https://github.com/lpil/mix-test.watch) project, which I've used and loved for years. I've added the interactive mode features to the existing feature set.
This project started as a clone of the wonderful
[mix-test.watch](https://github.com/lpil/mix-test.watch) project, which I've
used and loved for years. I've added the interactive mode features to the
existing feature set.

The idea for having an interactive mode comes from [Jest](https://jestjs.io/) and its incredibly useful interactive watch mode.
The idea for having an interactive mode comes from [Jest](https://jestjs.io/)
and its incredibly useful interactive watch mode.

## Copyright and License

Copyright (c) 2021-2024 Randy Coulman

This work is free. You can redistribute it and/or modify it under the
terms of the MIT License. See the [LICENSE.md](./LICENSE.md) file for more details.
This work is free. You can redistribute it and/or modify it under the terms of
the MIT License. See the [LICENSE.md](./LICENSE.md) file for more details.
21 changes: 12 additions & 9 deletions lib/mix/tasks/test/interactive.ex
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,9 @@ defmodule Mix.Tasks.Test.Interactive do
All of the `<mix test arguments>` are passed through to `mix test` on every
test run.
`mix test.interactive` will detect the `--stale` and `--failed` flags and use
those as initial settings in interactive mode. You can then toggle those flags
on and off as needed.
`mix test.interactive` will detect the `--failed`, `--seed`, and `--stale`
options and use those as initial settings in interactive mode. You can then
use the interactive mode commands to adjust those options as needed.
### Patterns and filenames
Expand All @@ -67,17 +67,20 @@ defmodule Mix.Tasks.Test.Interactive do
After the tests run, you can use the interactive mode to change which tests
will run.
- `a`: Run all tests.
- `a`: Run all tests. Clears the `--failed` and `--stale` options as well as
any patterns.
- `d <seed>`: Run the tests with a specific seed.
- `d`: Clear any previously specified seed.
- `f`: Run only tests that failed on the last run (equivalent to the
`--failed` option of `mix test`).
- `p`: Run only test files that match one or more provided patterns. A pattern
is the project-root-relative path to a test file (with or without a line
number specification) or a string that matches a portion of full pathname.
e.g. `test/my_project/my_test.exs`, `test/my_project/my_test.exs:12:24` or
`my`.
is the project-root-relative path to a test file (with or without a line
number specification) or a string that matches a portion of full pathname.
e.g. `test/my_project/my_test.exs`, `test/my_project/my_test.exs:12:24` or
`my`.
- `q`: Exit the program. (Can also use `Ctrl-D`.)
- `s`: Run only test files that reference modules that have changed since the
last run (equivalent to the `--stale` option of `mix test`).
last run (equivalent to the `--stale` option of `mix test`).
- `w`: Turn file-watching mode on or off.
- `Enter`: Re-run the current set of tests without requiring a file change.
Expand Down
25 changes: 25 additions & 0 deletions lib/mix_test_interactive/command/seed.ex
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
defmodule MixTestInteractive.Command.Seed do
@moduledoc """
Specify or clear the random number seed for test runs.
Runs the tests with the given seed if provided. If not provided, the seed is
cleared and the tests will run with a random seed as usual.
"""
use MixTestInteractive.Command, command: "d", desc: "set or clear the test seed"

alias MixTestInteractive.Command
alias MixTestInteractive.Settings

@impl Command
def name, do: "d [<seed>]"

@impl Command
def run([], %Settings{} = settings) do
{:ok, Settings.clear_seed(settings)}
end

@impl Command
def run([seed], %Settings{} = settings) do
{:ok, Settings.with_seed(settings, seed)}
end
end
2 changes: 2 additions & 0 deletions lib/mix_test_interactive/command_line_parser.ex
Original file line number Diff line number Diff line change
Expand Up @@ -164,13 +164,15 @@ defmodule MixTestInteractive.CommandLineParser do
defp build_settings(mti_opts, mix_test_opts, patterns) do
no_patterns? = Enum.empty?(patterns)
{failed?, mix_test_opts} = Keyword.pop(mix_test_opts, :failed, false)
{seed, mix_test_opts} = Keyword.pop(mix_test_opts, :seed)
{stale?, mix_test_opts} = Keyword.pop(mix_test_opts, :stale, false)
watching? = Keyword.get(mti_opts, :watch, true)

%Settings{
failed?: no_patterns? && failed?,
initial_cli_args: OptionParser.to_argv(mix_test_opts),
patterns: patterns,
seed: seed && to_string(seed),
stale?: no_patterns? && !failed? && stale?,
watching?: watching?
}
Expand Down
14 changes: 8 additions & 6 deletions lib/mix_test_interactive/command_processor.ex
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,23 @@ defmodule MixTestInteractive.CommandProcessor do
alias MixTestInteractive.Command.Pattern
alias MixTestInteractive.Command.Quit
alias MixTestInteractive.Command.RunTests
alias MixTestInteractive.Command.Seed
alias MixTestInteractive.Command.Stale
alias MixTestInteractive.Command.ToggleWatchMode
alias MixTestInteractive.Settings

@type response :: Command.response()

@commands [
Pattern,
Stale,
Failed,
AllTests,
ToggleWatchMode,
RunTests,
Failed,
Help,
Quit
Pattern,
Quit,
RunTests,
Seed,
Stale,
ToggleWatchMode
]

@doc """
Expand Down
Loading

0 comments on commit aa6a2b5

Please sign in to comment.