✨ Add tag filtering (#113)

* ♻️ Split end-to-end tests into separate workflows

* ✅ Add missing test for clearing the seed

* ✨ Add tag filtering settings

* ✨ Extract tag filter options from initial command line

* ✨ Add tag filtering commands

* ✅ Add end-to-end test for tag filtering

* ✨ Include tag filters in run summary

* 🚸 Sort help message by command

* 📝 Add tag filter commands to docs

* ✅ Sort settings on command line

The sort order is different in older Elixir versions, so this adds consistency.

README.md

@@ -16,7 +16,9 @@ interactive watch mode, `mix test.interactive` allows you to dynamically change
 which tests should be run with a few keystrokes.
 
 It allows you to easily switch between running all tests, stale tests, or failed
-tests. Or, you can run only the tests whose filenames contain a substring.
+tests. Or, you can run only the tests whose filenames contain a substring. You
+can also control which tags are included or excluded and easily specify the test
+seed to use.
 Includes an optional "watch mode" which runs tests after every file change.
 
 ## Installation
@@ -73,14 +75,13 @@ specified in the configuration.
 All of the `<mix test arguments>` are passed through to `mix test` on every test
 run.
 
-`mix test.interactive` will detect the `--failed`, `--seed`, and `--stale`
-options and use those as initial settings in interactive mode. You can then use
-the interactive mode commands to adjust those options as needed. It will also
-detect any filename or pattern arguments and use those as initial settings.
-However, it does not detect any filenames passed with `--include` or `--only`.
-Note that if you specify a pattern on the command-line, `mix test.interactive`
-will find all test files matching that pattern and pass those to `mix test` as
-if you had used the `p` command.
+`mix test.interactive` will detect the `--exclude`, `--failed`, `--include`,
+`--only`, `--seed`, and `--stale` options and use those as initial settings in
+interactive mode. You can then use the interactive mode commands to adjust those
+options as needed. It will also detect any filename or pattern arguments and use
+those as initial settings. Note that if you specify a pattern on the
+command-line, `mix test.interactive` will find all test files matching that
+pattern and pass those to `mix test` as if you had used the `p` command.
 
 ### Patterns and filenames
 
@@ -91,40 +92,35 @@ more patterns on the command-line, `mix test.interactive` will find all test
 files matching those patterns and pass them to `mix test` as if you had used the
 `p` command (described below).
 
-After the tests run, you can use the interactive mode to change which tests will
-run.
-
-Use the `p` command to run only test files that match one or more provided
-patterns. A pattern is the project-root-relative path to a test file (with or
-without a line number specification) or a string that matches a portion of full
-pathname. e.g. `test/my_project/my_test.exs`,
-`test/my_project/my_test.exs:12:24` or `my`.
-
-Any patterns that contain a line number specification are passed directly to
-`mix test`. Remaining patterns are matched against test filenames as above.
-
-```
-p pattern1 pattern2
-```
-
-Use the `s` command to run only test files that reference modules that have
-changed since the last run (equivalent to the `--stale` option of `mix test`).
-
-Use the `f` command to run only tests that failed on the last run (equivalent to
-the `--failed` option of `mix test`).
-
-Use the `a` command to run all tests, turning off the `--failed` and `--stale`
-flags as well as clearing any patterns.
-
-Use the `d <seed>` command to run tests with a specific seed, and then use `d`
-(with no seed) to remove the seed.
-
-Use the `w` command to turn file-watching mode on or off.
-
-Use the `Enter` key to re-run the current set of tests without requiring a file
-change.
-
-Use the `q` command, or press `Ctrl-D` to exit the program.
+After the tests run, you can use the interactive commands to change which tests
+will run.
+
+- `a`: Run all tests. Clears the `--failed` and `--stale` options as well as
+  any patterns.
+- `d <seed>`: Run the tests with a specific seed.
+- `d`: Clear any previously specified seed.
+- `f`: Run only tests that failed on the last run (equivalent to the
+  `--failed` option of `mix test`).
+- `i <tags...>`: Include tests tagged with the listed tags (equivalent to the
+  `--include` option of `mix test`).
+- `i`: Clear any included tags.
+- `o <tags...>`: Run only tests tagged with the listed tags (equivalent to the
+  `--only` option of `mix test`).
+- `o`: Clear any "only" tags.
+- `p`: Run only test files that match one or more provided patterns. A pattern
+  is the project-root-relative path to a test file (with or without a line
+  number specification) or a string that matches a portion of full pathname.
+  e.g. `test/my_project/my_test.exs`, `test/my_project/my_test.exs:12:24` or
+  `my`.
+- `q`: Exit the program. (Can also use `Ctrl-D`.)
+- `s`: Run only test files that reference modules that have changed since the
+  last run (equivalent to the `--stale` option of `mix test`).
+- `x <tags...>`: Exclude tests tagged with the listed tags (equivalent to the
+  `--exclude` option of `mix test`).
+- `x`: Clear any excluded tags.
+- `w`: Turn file-watching mode on or off.
+- `Enter`: Re-run the current set of tests without requiring a file change.
+- `?`: Show usage help.
 
 ## Configuration
 

lib/mix/tasks/test/interactive.ex

@@ -5,8 +5,9 @@ defmodule Mix.Tasks.Test.Interactive do
 
   `mix test.interactive` allows you to easily switch between running all tests,
   stale tests, or failed tests. Or, you can run only the tests whose filenames
-  contain a substring. Includes an optional "watch mode" which runs tests after
-  every file change.
+  contain a substring. You can also control which tags are included or excluded
+  and easily specify the test seed to use. Includes an optional "watch mode"
+  which runs tests after every file change.
 
   ## Usage
 
@@ -49,9 +50,13 @@ defmodule Mix.Tasks.Test.Interactive do
   All of the `<mix test arguments>` are passed through to `mix test` on every
   test run.
 
-  `mix test.interactive` will detect the `--failed`, `--seed`, and `--stale`
-  options and use those as initial settings in interactive mode. You can then
-  use the interactive mode commands to adjust those options as needed.
+  `mix test.interactive` will detect the `--exclude`, `--failed`, `--include`,
+  `--only`, `--seed`, and `--stale` options and use those as initial settings in
+  interactive mode. You can then use the interactive mode commands to adjust
+  those options as needed. It will also detect any filename or pattern arguments
+  and use those as initial settings. Note that if you specify a pattern on the
+  command-line, `mix test.interactive` will find all test files matching that
+  pattern and pass those to `mix test` as if you had used the `p` command.
 
   ### Patterns and filenames
 
@@ -64,15 +69,21 @@ defmodule Mix.Tasks.Test.Interactive do
 
   ## Interactive Commands
 
-  After the tests run, you can use the interactive mode to change which tests
-  will run.
+  After the tests run, you can use the interactive commands to change which
+  tests will run.
 
   - `a`: Run all tests. Clears the `--failed` and `--stale` options as well as
     any patterns.
   - `d <seed>`: Run the tests with a specific seed.
   - `d`: Clear any previously specified seed.
   - `f`: Run only tests that failed on the last run (equivalent to the
   `--failed` option of `mix test`).
+  - `i <tags...>`: Include tests tagged with the listed tags (equivalent to the
+    `--include` option of `mix test`).
+  - `i`: Clear any included tags.
+  - `o <tags...>`: Run only tests tagged with the listed tags (equivalent to the
+    `--only` option of `mix test`).
+  - `o`: Clear any "only" tags.
   - `p`: Run only test files that match one or more provided patterns. A pattern
     is the project-root-relative path to a test file (with or without a line
     number specification) or a string that matches a portion of full pathname.
@@ -81,8 +92,12 @@ defmodule Mix.Tasks.Test.Interactive do
   - `q`: Exit the program. (Can also use `Ctrl-D`.)
   - `s`: Run only test files that reference modules that have changed since the
     last run (equivalent to the `--stale` option of `mix test`).
+  - `x <tags...>`: Exclude tests tagged with the listed tags (equivalent to the
+    `--exclude` option of `mix test`).
+  - `x`: Clear any excluded tags.
   - `w`: Turn file-watching mode on or off.
   - `Enter`: Re-run the current set of tests without requiring a file change.
+  - `?`: Show usage help.
 
   ## Configuration
 

lib/mix_test_interactive/command/exclude.ex

@@ -0,0 +1,26 @@
+defmodule MixTestInteractive.Command.Exclude do
+  @moduledoc """
+  Specify or clear tags to exclude.
+
+  Runs the tests excluding the given tags if provided. If not provided, the
+  excludes are cleared and the tests will run with any excludes configured in
+  your `ExUnit.configure/1` call (if any).
+  """
+  use MixTestInteractive.Command, command: "x", desc: "set or clear excluded tags"
+
+  alias MixTestInteractive.Command
+  alias MixTestInteractive.Settings
+
+  @impl Command
+  def name, do: "x [<tags...>]"
+
+  @impl Command
+  def run([], %Settings{} = settings) do
+    {:ok, Settings.clear_excludes(settings)}
+  end
+
+  @impl Command
+  def run(tags, %Settings{} = settings) do
+    {:ok, Settings.with_excludes(settings, tags)}
+  end
+end

lib/mix_test_interactive/command/include.ex

@@ -0,0 +1,26 @@
+defmodule MixTestInteractive.Command.Include do
+  @moduledoc """
+  Specify or clear tags to include.
+
+  Runs the tests excluding the given tags if provided. If not provided, the
+  includes are cleared and the tests will run with any includes configured in
+  your `ExUnit.configure/1` call (if any).
+  """
+  use MixTestInteractive.Command, command: "i", desc: "set or clear included tags"
+
+  alias MixTestInteractive.Command
+  alias MixTestInteractive.Settings
+
+  @impl Command
+  def name, do: "i [<tags...>]"
+
+  @impl Command
+  def run([], %Settings{} = settings) do
+    {:ok, Settings.clear_includes(settings)}
+  end
+
+  @impl Command
+  def run(tags, %Settings{} = settings) do
+    {:ok, Settings.with_includes(settings, tags)}
+  end
+end

lib/mix_test_interactive/command/only.ex

@@ -0,0 +1,26 @@
+defmodule MixTestInteractive.Command.Only do
+  @moduledoc """
+  Specify or clear the only tags to run.
+
+  Runs the tests with only the given tags if provided. If not provided, the list
+  of only tags is cleared and the tests will run with any only configured in
+  your `ExUnit.configure/1` call (if any).
+  """
+  use MixTestInteractive.Command, command: "o", desc: "set or clear only tags"
+
+  alias MixTestInteractive.Command
+  alias MixTestInteractive.Settings
+
+  @impl Command
+  def name, do: "o [<tags...>]"
+
+  @impl Command
+  def run([], %Settings{} = settings) do
+    {:ok, Settings.clear_only(settings)}
+  end
+
+  @impl Command
+  def run(tags, %Settings{} = settings) do
+    {:ok, Settings.with_only(settings, tags)}
+  end
+end

lib/mix_test_interactive/command_line_parser.ex

@@ -163,14 +163,20 @@ defmodule MixTestInteractive.CommandLineParser do
 
   defp build_settings(mti_opts, mix_test_opts, patterns) do
     no_patterns? = Enum.empty?(patterns)
+    {excludes, mix_test_opts} = Keyword.pop_values(mix_test_opts, :exclude)
     {failed?, mix_test_opts} = Keyword.pop(mix_test_opts, :failed, false)
+    {includes, mix_test_opts} = Keyword.pop_values(mix_test_opts, :include)
+    {only, mix_test_opts} = Keyword.pop_values(mix_test_opts, :only)
     {seed, mix_test_opts} = Keyword.pop(mix_test_opts, :seed)
     {stale?, mix_test_opts} = Keyword.pop(mix_test_opts, :stale, false)
     watching? = Keyword.get(mti_opts, :watch, true)
 
     %Settings{
+      excludes: excludes,
       failed?: no_patterns? && failed?,
+      includes: includes,
       initial_cli_args: OptionParser.to_argv(mix_test_opts),
+      only: only,
       patterns: patterns,
       seed: seed && to_string(seed),
       stale?: no_patterns? && !failed? && stale?,
@@ -262,10 +268,10 @@ defmodule MixTestInteractive.CommandLineParser do
 
   defp parse_one_option_value(_name, value), do: {:ok, value}
 
-  defp combine_multiples(mti_opts) do
+  defp combine_multiples(opts) do
     @options
     |> Enum.filter(fn {_name, type} -> type == :keep end)
-    |> Enum.reduce(mti_opts, fn {name, _type}, acc ->
+    |> Enum.reduce(opts, fn {name, _type}, acc ->
       case Keyword.pop_values(acc, name) do
         {[], _new_opts} -> acc
         {values, new_opts} -> Keyword.put(new_opts, name, values)

lib/mix_test_interactive/command_processor.ex

@@ -5,8 +5,11 @@ defmodule MixTestInteractive.CommandProcessor do
 
   alias MixTestInteractive.Command
   alias MixTestInteractive.Command.AllTests
+  alias MixTestInteractive.Command.Exclude
   alias MixTestInteractive.Command.Failed
   alias MixTestInteractive.Command.Help
+  alias MixTestInteractive.Command.Include
+  alias MixTestInteractive.Command.Only
   alias MixTestInteractive.Command.Pattern
   alias MixTestInteractive.Command.Quit
   alias MixTestInteractive.Command.RunTests
@@ -19,8 +22,11 @@ defmodule MixTestInteractive.CommandProcessor do
 
   @commands [
     AllTests,
+    Exclude,
     Failed,
     Help,
+    Include,
+    Only,
     Pattern,
     Quit,
     RunTests,
@@ -52,6 +58,7 @@ defmodule MixTestInteractive.CommandProcessor do
     usage =
       settings
       |> applicable_commands()
+      |> Enum.sort_by(& &1.command())
       |> Enum.flat_map(&usage_line/1)
 
     IO.ANSI.format([:bright, "Usage:\n", :normal] ++ usage)