Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Documentation revamp #1146

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Conversation

telyn
Copy link
Contributor

@telyn telyn commented Dec 8, 2020

Hi Markus and any other mutant contributors,

I've been reworking some of the documentation to try to reduce the feeling of absolute befuddlement any mutant novice experiences. This PR is still very much a to-do, though!

Please ignore the long lines, double-spaces and fancy quotes - they are not my preference. I've been trying out a new editor and it seemed like it was correctly automatically putting new-lines in, but it was all a front! I'll be going back over everything quickly at the end with vim, my one true love.

I'll be continuing to alter this PR over the next week or so - this push is just so you can see the current style and content and give any early feedback.

TODOS (probably not exhaustive)

  • Continue reworking docs/nomenclature
    • TODO under Mutation Operators
    • everything from Insertion until the end of the file.
  • Big rework of docs/mutant-rspec.md
  • More jokes (or fewer jokes - depending on the main authors' preferences)
  • Figure out how to do footnotes more cleanly
  • plaintext style fixes
    • Get rid of the fancy quote marks,
    • Remove trailing whitespace
    • Fix everything to 80 cols

Still to-do:
* Big rework of docs/mutant-rspec.md
* More jokes (or fewer jokes - depending on the main authors' preferences)
* Get rid of the fancy quote marks & trailing whitespace
* Continue reworking docs/nomenclature, starting with the TODO under
Mutation Operators, and then carrying on from Insertion until the end of
the file.
docs/limitations.md Outdated Show resolved Hide resolved
Comment on lines 30 to 34
Other subjects are possible (even project-specific subjects) but aren't implemented in the OSS version. Please get in touch with the authors. Some examples are:
* Constants
* Class bodies for DSLs

The more subjects that mutant can alter in your project, the more mutations it can create, and so the higher your confidence you can have that your tests cover the semantics of your application. Please get in touch if you require subjects beyond those implemented in Mutant already - support may be available in the commercial version.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This advises you to get in touch twice--feels slightly redundant.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I had meant to supplant the first one with the latter 😅


The more subjects that mutant can alter in your project, the more mutations it can create, and so the higher your confidence you can have that your tests cover the semantics of your application. Please get in touch if you require subjects beyond those implemented in Mutant already - support may be available in the commercial version.

## Mutation Operator
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if this was discussed in the discord or elsewhere, but this terminology seems slightly confusing. What if this heading was just something like "Kinds of Mutations".

Copy link
Contributor Author

@telyn telyn Dec 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that it's confusing, and I think it's an unnecessary type of thing to have as its own heading. I am aware of some nomenclature-related discussions occurring in the discord but I've struggled to find any kind of conclusive decision-making.

I used it because it was used in the original document, but I think it may be better to have it as a sort-of "here are some types of mutations that can occur" sub-section in the "Mutations" section. What do you & @mbj think to that?

docs/nomenclature.md Outdated Show resolved Hide resolved
docs/nomenclature.md Outdated Show resolved Hide resolved
We could argue[^1] that this category includes all of the following:

* Removing a particular expression, from the AST entirely
* Replacing an expression with `nil`, or `true`, or `false`, or other simple literals.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't tend to think of those as quite the same as "code removal" but maybe it belongs here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm. That's true, it's more of an extreme semantic reduction, I suppose 😛

I think one thing that's missing from this whole section is a motivation for choosing to categorise things, as well as a motivation for these specific categories. I'm mostly using it as a way to structure the documentation to break it up into smaller more digestible pieces.

In my experience, when I run mutant on a subject and it reduces 2-3 lines of code to nil or self, I'm thinking "ah I just haven't specified those features in my tests", or "perhaps I could restructure that into its own method and test it individually". When I run it and it removes or reduces a smaller amount of code, I've found it's more likely that it's actually removing / reducing something that isn't necessary. This may just be a symptom of the fact my mutation testing has so far been limited to things I have built since setting up mutant.

I realise I am coming to this from a bit of a beginner's standpoint, which is perhaps not helping me to write this document in a universally applicable way. I'm happy to continue a discussion on this and to defer to experts!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reworked this a bit. As well as combining the mutation and mutation operator sections, I've reworked this section a bit to add in some of that motivation. Let me know what you think of it now!

docs/nomenclature.md Outdated Show resolved Hide resolved
## Mutation Testing
The practice of systematically applying small changes one at a time to a codebase then re-running the (relevant) tests for each change. If the tests fail, then they cover the semantics of those changes. If the tests continue to pass with the changes in place, the tests do not cover the complete semantics of the changed parts of the codebase.

Each type of change (for example, “change `a && b` to `a`”) is known as a [Mutation Operator](#mutation-operator), and is applied to the pristine codebase - that is, the changes are not stacked up, but applied in isolation. The places in which the Mutation Operators can make their changes are called [Subjects](#subject). The changed Subject is then referred to as a [Mutation](#mutation).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Each type of change (for example, “change a && b to a”) is known as a ..."

This sounds like the change itself is a "Mutation Operator" but I think that would be a Mutation. You mean to be talking about kinds of changes but I think the example confuses it a bit because it is an instance of a mutation.

docs/reading-reports.md Outdated Show resolved Hide resolved
Comment on lines +8 to +16
* `str` and `sym` are string and symbol literals that do not contain dynamic elements (`"I'm a dstring #{i_am_the_dynamic_element}"`)
* `dst` and `dsym` are the equivalents for strings and symbols which do
* `regexp` is a regular expression without any options (`/like this/`)
* `regopt` is a regular expression with options (`/like this/i`)
* a few come in regular and `-asgn` variants - the latter are the assignment versions. E.g., `||=` is the assignment version (`or_asgn.rb`) of `||` (`or.rb`)
* `gvar`, `cvar`, `ivar` and `lvar` refer to global (`$VERBOSE`), class (`@@debug`), instance (`@amount`) and local (`response`) variables, respectively.
* `csend` is the conditional send operator (`&.`)
* `cbase` is a top-level constant (`::Kernel` or `::Errno::ENOENT`), whereas `const` is a "normal" constant (`Kernel` or `Errno::ENOENT`)
* `casgn` is an assignment to a constant (`VERSION = "0.1.0"`)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This strikes me as very helpful to new contributors :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

, or Things I Had To Learn When I Was Adding Metaclass Support 😁

@dgollahon
Copy link
Collaborator

Lots of improvements here 🎉 Glad to see the docs getting additional attention.

I left a few thoughts I had as I read through.

@telyn
Copy link
Contributor Author

telyn commented Dec 9, 2020

I left a few thoughts I had as I read through.

Thanks for the feedback - I haven't read any of it (yet) as it's midnight but I'll incorporate it into my TODOs up top tomorrow ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants