The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
- Renamed the project to
darglint2
while forking it from the archived terrencepreilly/darglint to akaihola/darglint2.
- There is now a Flake8 flag exposing the ignore regex option. This will allow you to ignore a given file/set of files based on a regex during a CI pipeline, etc. Thanks to @AdamGleave for the pull request!
- Removed logging for some aspects of analysis, and an assertion from the raise visitor. They were a little overly-broad, and it was causing some integration tests to fail when run against new repos.
- We now ignore return, pass, yield, and raise in abstract methods. Thanks to @maltevesper!
- Resolved most of the mypy issues in the repository. This is mostly a code cleanliness issue.
- We now recognize property methods on a class, and there are options to ignore properties for checking. Thanks to @saroad2!
- Added an option to ignore the presence of certain exceptions. This is
useful if your project has a common error which you don't want to have to
document everywhere (for example, a custom assertion error.) This can be
used through the
ignore_raise
configuration option, or command-line option. Thanks to @palt0 for the pull request! - Added
ignore_regex
as a command-line option (this already existed as a configuration option, but I overlooked it as a command-line option.) Thanks to @saroad2 for the pull request!
- Deprecated Python3.5 support, since it breaks the Travis CI build. I've left the dockerfile test for it in, as everything still works.
- Added Python3.9 support.
- Changed config.cfg to setup.cfg in the pre-commit documentation, which was the correct name. Thanks to @nuno-andre for the pull request.
- Load the
enable
option from the configuration file, so that disabled-by-default errors can be shown. Thanks to @palt0 for the pull request!
- Bare returns now do not have to be documented. This aligns better with expectations for docstrings, and matches what darglint did prior to refactoring to handle nested functions.
- Refactored some enumerations into their own files, and moved strictness checks to the base docstring method. Thanks to @skarzi for the pull requests! It makes the code a good deal cleaner.
- Lambdas no longer result in erroneous arguments in the docstring. Lambdas apparently reuse the Arguments ast node, which means that, since they were scoped in the same function, they were adding erroneous arguments. By moving lambdas into their own scope, we resolve that issue. Of course, if a lambda raises an exception, we will no longer catch that in the function description. However, doing that seems far less likely, and not worth the excess logic to handle.
-
Dockerfiles for testing each supported Python version. Also added a tox section to allow for testing these dockerfiles. To build the images, you can first do
make -C docker-build
then you can run
tox -e docker
And it will run the unit tests against Python3.5 to Python3.8.
-
Nested functions/methods. We should now handle nested functions/methods explicitly. Previously, some attributes (such as returns, etc.) would leak through nested functions into their parents. (Since we were just walking the tree and looking for the presence of certain tokens.) This also allowed for a nice refactoring of how we walk through/collect data on the function ast.
-
Exception handlers with named instances now have the correct type reported with bare raises. Previously, if you had something like
def inverse(x): try: return 1 / x except ZeroDivisionError as zde: raise
darglint would incorrectly drop the
ZeroDivisionError
as a possible error source when examining the function. This was due to named exception handlers being handled differently from other exceptions. (The name had to be stored in case a user did something likeraise err
.) -
Remove variables section from the numpy supported sections. Thanks to @skarzi for the pull request!
- Permissions errors from searching for a config would previously crash darglint. Fixed thanks to @pawamoy.
- Some code highlighting issues fixed thanks to @sobolevn.
- Documentation on installing test dependencies updated thanks to @mathieu.
- Finally/else blocks for try statements are now handled when analyzing whether an exception can be raised.
- Added handling of positional-only arguments, and some tests for mixing with positional/keyword arguments and keyword-only arguments.
- Allowed sphinx docstrings to raise an
IndentError
when a line is underindented in an item definition. This makes troubleshooting incorrect indentation easier. (Previously, it would have just failed to parse.) - Configuration for log level. By default, now, all assertions
are logged at
ERROR
, and the default log level isCRITICAL
. If darglint doesn't act as expeced, then, the user could pass the--log-level
flag to rerun and see if an expectation was missed.
- Allowed
AssertionError
for functions which contain anassert
statement. Previously, this would result in an excess raises error. - Allowed qualified names in tuple exception handlers.
- Removed legacy token for quotation mark, which could cause poor parsing in edge cases (due to the token not being used.)
- A colon after an error in Numpy format is now reported as an empty type error. Even though exception types are not a part of the function signature, and even though it's not a part of the numpy standard, this at least gives a hint to the user that a dangling colon should be removed. Incidentally, exceptions could have different types, and it could be somewhat useful to know what those are.
- Typo in readme fixed thanks to cthoyt@.
- Underspecified types previously resulted in confusing error messages because dargint incorrectly matched types up with the arguments they describe. This is now corrected.
- Ignoring style errors wasn't working. Any error captured during the parsing phase was just being added without respecting the "ignore" configuration.
- Make flake8 config use default config as a base configuration.
- Fix style being parsed for strictness in configuration parsing.
- Settings can now be configured from flake8's configuration file, or from the flake8 command. Thanks to @Harrison88 for the PR!
- Handle bare raise statement in multiple exception handlers.
- Handle bare raise statement in catch-all exception handler.
- Handle reraising an error from a handler where the caught error(s) is a tuple.
- Private arguments (arguments with a leading underscore) are no longer required. If present, they will still be subject to other checks. (For example, if the description is missing, an error will be reported.)
- Handled newlines after Google argument types. Newlines were handled in most other situations (inside of types, after an untyped item, etc.) but this one slipped through.
- Handle parentheses inside parenthetical google types. Previously, darglint simply failed to parse those arguments. Now it will raise a ParameterMalformedError.
- Updated README to be more explicit about how Darglint handles types and style.
-
Qualified exceptions in a raise section were not being handled correctly by the
FunctionDescription
class. So, if you had a catch statement like,def finish_me(): try: raise requests.exceptions.ProxyError() except requests.exceptions.ProxyError: raise
It wouldn't have been handled correctly. That is, if you documented
requests.exceptions.ProxyError
in the docstring, darglint would throw an error. This change resolves the situation, at lest somewhat. It expects the documented exception and the caught exception to match exactly. -
Implicitly raised exceptions which are rethrown was also not handled. So, for example, if you had:
def throwing_up(): """Throws up.
Raises: ProxyError: If failed to yarf. """ try: proxy_puke() except ProxyError: raise
Darglint would report an error. It no longer reports an error in this case. See Issue #88 and Issue #68.
- A race condition in the integrity checker. I had moved the function to be an instance variable, rather than a member of the class, but I had forgotten to do so with the docstring representation. So, if the next docstring was fast enough to parse, it could override the previous docstring. This didn't happen often because parsing is what takes the longest, and there is a high variability between the time it takes to parse each docstring.
- Handle newline after exception in raises section. Rather than failing to parse the raises section, it now reports an indentation error.
- Made ordering of error messages more strict. Error messages are grouped by function, then sorted alphabetically, then sorted in place by their line numbers within the function groups.
-
Handle the else branch of try-except statements when searching for raise statements during analysis.
-
Handle empty type in Google-style arguments (previously raised an exception, now it's a darglint error with a description.)
-
Handle exceptions raised by the Python module, ast.
-
Handle a bare raise statement within an exception handler. For example, if you have
try: raise SyntaxError('Wrong!') except SyntaxError: raise
Then you know that you really have to document a syntax error.
- Certain versions of Python failed with in analysis, complaining of a missing 'id' attribute.
- Merged in a configuration file for the pre-commit project. Thanks to @mikaelKvalvaer for the PR and @nioncode for pointing out the advantages of this configuration.
- Made error analysis more robust. Previously, errors were
compared simply by looking a the name in the
raises
statement. Now analysis builds contexts around try-except blocks, and accounts for the catch-and-throw pattern. Anything unusual will still raise an error, but it's more robust than previously.
- Fixed the handling of breaks in item descriptions. Thanks to lvermue@ for the pull request and submitting the bug.
- Added initial implementation for the numpy docstring format to darglint. Support for this docstring format is not yet stable, and may have performance problems specific to the format.
- Add error number to exceptions raised when using the flake8 runner. This allows flake8 to report the error without incorrectly unpacking the string.
- Fix sphinx return type annotation. Previously, there was an
inline return type which was accepted, in the format
:return x: Explanation.
wherex
is the type returned. However, this should really only be handled by the:rtype:
section. This incorrect return type was labelled using the same node name as the normal return type, leading to an assertion being raised. The inline return type was removed.
- Allow for two-space indentation.
- Make the asterisk(s) in star arguments optional.
- Added compatibility test for Darglint with flake8-docstrings and flake8-rst-docstrings.
- Added
DAR104
which checks for the presence of types in the docstring.
- Updated the flake8 entry point to reflect the new error codes.
- Updated readme and error description list.
- Move debug- or test-only functions to utils file. Probably this won't help much with load times or memory use, but it will at least reduce the risk of parse errors from that code.
- Removed the google_types.py target, since that is not used directly by any source code. (google_types.bnf is imported by other BNF files, only.)
- Fixed mypy errors in darglint.
- Changed the error code prefixes from "I" and "S" to "DAR". This will prevent collisions with other utilities reported through flake8. See Issue #40.
- UTF-8 encoded source files were not working previously. Rather than
reading the string (and forcing an encoding on the data), we're now
reading bytes and passing that along to the
ast
module. See Issue #46.
- Changed the long description format to text/markdown.
-
A bin/ folder to hold various development utilities for darglint. Currently, it holds
bnf_to_cnf
(a utility to convert BNF grammars to CNF grammars), anddoc_extract
(a utility to extract docstrings from repositories, and annotate them for use in integration tests.) -
A (crappy) integration test framework. Test fixtures are ignored in git, since the integration tests are only relevant for local development (and even then, mostly just release). The integration tests are as follows:
-
goldens.py: Tests against goldens for individual docstrings. This attempts to ensure that parsed docstrings always contain the expected sections after development. Goldens are generated using the
doc_extract
utility in the bin/ folder. -
grammar_size.py: Tests that the grammar size doesn't increase significantly. Larger grammars will result in longer parse times, and it could be relatively easy to accidentally introduce a much larger grammar.
-
performance.py: Tests performance of the parser against individual docstrings to make sure we don't introduce a performance regression. Also tests performance for individual files in some repositories.
-
TODO: We still need to add some tests against multiple configurations, and against entire repositories.
-
- Changed the recursive descent parser to a CYK parser. This was a significant change. It makes darglint much more flexible and extensible. However, it also introduces a significant performance regression. For that reason, this will be released first as an alpha, while I figure out how to handle the performance regression (or I determine whether or not it's even all that important of a problem.)
-
Incorrect configuration for flake8. See Issue #35.
-
Incorrect check for strictness options long and full. See Issue #37.
Thanks to sobolevn for these fixes!
-
Minimum strictness configuration option. You can now specify a minimum amount of strictness to have when checking docstrings. Strictness does not affect whether the docstring will be checked or not; it only changes the amount of checking which is done. For example, if your config file looks like
[darglint] strictness=short
Then the following would pass with no errors:
def double(x): """Returns the number, multiplied by two.""" return x * 2
The following levels of strictness are available:
-
short: One-line descriptions are acceptable; anything more and the docstring will be fully checked.
-
long: One-line descriptions and descriptions without arguments/returns/yields/etc. sections will be allowed. Anything more, and the docstring will be fully checked.
-
full: (Default) Docstrings will be fully checked.
-
- Syntax error when logging about unusual raises description. (See Issue #34).
- Handle async function definitions. Previously they were simply skipped. Thanks to @zeebonk!
- Erroneous I203 was being raised for return type when one of the type annotations was missing (it should only ever be raised when both type signatures are present.) Thanks to @asford!
-
Try-block handlers were not included when tranversing the function ast. This meant that return or yield statements in the except-block or else block would not register, and an exception would be raised.
-
Allow indents in types. This lets us handle line continuation for type signatures.
-
Fix appropriate noqas not hiding S001. (Because the docstring is never actually parsed when it's a syntax error.)
- Fixed double reports from flake8. These were occurring due to the two entry points listed in the setup file. Currently, the fix just uses two temporary subclasses that filter the response from running Darglint. Ideally, flake8 would not run the same program twice -- a script may want to report more than one error code. (As is the case here.)
- The ability to read from stdin, which should make it easier to integrate darglint into other tools.
- The restriction that a file must end in
.py
.
-
Simplified the interface for the Docstring base class. This should make modifications easier, and should ensure that the same value is returned consistently between Google and Sphinx styles.
This comes at the cost of a slightly more complicated function signature for the functions which remain. However, the single parameter which is passed to the functions (an enumeration named
Sections
), can (and should) be used everywhere theNodeType
enumeration is currently used (outside of the actual parsing step.) This will effectively create a firewall around the parsing step.
- Fix the parser failing on single-word return description.
- Check for excess variable descriptions in Sphinx docstrings. If the
docstring contains
:var <name>:
, and the name doesn't exist in the actual function, an error will be issued.
- Parser exception being thrown when there is a whitespace-only line with indentation at the same level as an item within a compound section. (See Issue #7.)
- Support for using Darglint as a Flake8 extension. If Flake8 and Darglint are installed in the same environment, Darglint will issue warnings through Flake8.
- Parser for sphinx-style docstrings. This parser handles most
sphinx-style docstrings, but adds certain restrictions. For example,
the fields such as
:returns:
must be the last items in the docstring, must be together, and any multiple lines must be indented with four spaces. - Pipfile for setup with pipenv.
- Previously, darglint always exited with status 0, even if errors were encountered. Now, darglint exists with status 1 if docstring errors were encountered.
- Add description of errors to both the readme and the driver. The errors are each described on a single line to make it easy to search for errors from the command line.
- Fix regression in type hints: Deque was only available in Python3.6.
- Update the line numbers to be closer to the actual source of problems.
- Change the parser to a recursive descent parser. This allows greater precision and flexibility. It will also make it possible to capture and pass along line numbers.
- Line numbers added to
Token
s. The line numbers are added and incremented in thelex()
function.
-
Message templates for error reports based on Pylint's syntax. The message template uses a normal Python format string with named arguments. For example, the default format string for an error message is '{path}:{obj}:{line}: {msg_id}: {msg}'.
This is passed to the linter by a command-line argument. For example, to get only the path, line number, and error code, we could pass something like the following:
darglint -m "{path}:{line} -> {msg_id}" *.py
This value can also be specified in the configuration file
as the value, message_template
. For example, the above
template could have been specified in the configuration
as
[darglint]
message_template={path}:{line} -> {msg_id}
-
Added support for Python3.5. Probably earlier versions of Python can also be supported, but they need to be tested, first.
-
Added tox script for running tests against all environments. To run the tests, make sure the test dependencies are installed, and run
tox
-
Allow global noqa statements in docstring. If we have a docstring which we would like to ignore (say, because it has a different format), then we can ignore it either by adding "# noqa" or "# noqa: *" to it.
For example, the following program would raise an exception that there is a missing "Returns" section:
def is_palindrome(word):
"""True if the word is a palindrome.
# noqa
"""
return word == word[::-1]
However, since there is a global noqa statement, all errors are ignored.
- Added a mechanism for raising more specific style errors from the
parser. Unfortunately, the parser does not have access to the
functions whose docstrings are being parsed. (These two halves are
put together in the
IntegrityChecker
.) The parser cannot raise specific error messages, then, since it can't access the function. So, currently, theParserException
takes an optional, more specific error, which can be created by theIntegrityChecker
.
- Fixed broken unit test.
- Made error handling more robust. If darglint is unable to parse, now, it will display an error message for that function. This error message is taken from the assert statements in the parser, or is a general message. This, at least, prevents an ugly stack trace and provides a model for how to handle style errors in the future.
- Fixed a couple of type errors. Typing is either going to be removed or moved to a different format to allow for < Python3.6.
- Previously, returns in inner functions would raise errors for missing returns sections in the docstring, even if (for example), the parent function yielded values. For example:
- Broken tests were fixed. (The
ast
module does not return functions in any particular order, but tests were relying on a specific order.)
def walk(root):
"""Walk the tree, yielding active nodes.
Args:
root: The root of the tree.
Yields:
Active nodes.
"""
def skip(node):
return node.inactive
queue = deque()
queue.append(root)
while len(queue) > 0:
curr = queue.pop()
if skip(curr):
continue
yield curr
queue.extend(curr.children)
This function previously would have raised a missing return error (I201), and would have required a noqa. That is no longer the case.
- Renamed "darglint.py" to a more appropriate name: "function_description.py".
-
ParserException
is thrown if there is no colon after the type annotation or argument/exception in the description.For example, the following function would raise the
ParserException
:
def hello(name):
"""Say hello to the person.
Args:
name: The name of the person to
whom we are saying hello.
"""
print('hello, {}'.format(name))
-
Added optional hint to the function
_expect_type
indarglint/parse.py
. This will be displayed after the default message. We'll have to add an option for hints in the configuration, and show or hide them accordingly. -
Added check to ensure that a description exists after an item in the argument descriptions or exception descriptions (or any multi-section). At some point, this should probably be optional, but it is currently raises a
ParserException
(which is too general to really want to exclude.)
GenericSyntaxError
for anyParserException
s which are raised when parsing. Ideally, we would add the offset for the lines in the docstring so that we could get a more accurate indication of where the actual error occurred. (This would be useful for when we integrate with ALE or Syntastic or something.)get_logger()
function to the config file. This will help us to get a fully configured logger. It'll also be a nice way to request a preconfigured logger (say, "darglint.parser" or something if we want to log from the parser.) Then we could parse the same configuration to every logger.- Added check for bare raises. For example, if we had a function such as:
def do_something_dangerous():
try:
dangerous_action()
except CertainDoom:
raise
This would previously caused an error. Now, no checks are made for it. A "TODO" was left in the code as a reminder that we could handle this better in the future.
-
Added configuration files. Configuration files use the normal Python
configparser
and TOML format. Configuration files for darglint should have one of the following names:.darglint
setup.cfg
tox.ini
In addition, the settings for darglint must be in a section described by
[darglint]
. The configuration takes a single key,ignore
, and a list of error codes to ignore as a value. For example, if we would like to ignore the errors "I101" and "I102", then we would could write the following section in our tox.ini file:
[darglint]
ignore=I101,I102
darglint will look for a config file stating at the current directory and working its way up to the root directory.
- Fixed star arguments not being treated correctly.
-
Simplified error messages.
There are now only two verbosity levels, 1 and 2. I've kept it as an integer argument so that, in the future, we can add other levels if necessary. The default level, now, is 1. At that level, the error message is something like:
::: :
Where message is an abbreviated version. (It uses symbols primarily. For example "- word" denotes a missing parameter in a docstring. "+ word" denotes an extra parameter in the docstring.
Using a level 2 verbosity also prints out the general error message. (This describes what the error is. So, if level 1 is too cryptic, we can switch to level 2.) This will look like:
::: : :
This gets pretty long, so that's why it's not the default.
-
Verifies types specified in the docstring against type hints on the function. Also added
noqa
for these errors.For example, if we have a function with a mismatched type, such as:
def mismatched_type(x: int) -> str: """Return the string representation of the number. Args: x (float): The float to represent. Returns: str: The string representation of the number. """ return str(x)
Then it would raise a
TypeMismatchError
, since the parameter,x
, has a type hint ofint
, while the docstring documents it as being of typefloat
. We could prevent this error by either adding# noqa: I103
or# noqa: I103 x
.
-
Checks for a "Yields" section added. If a function contains either the
yield
oryield from
keywords, darglint will expect to find a "Yields" section in the docstring. -
Checks for a "Raises" section added. If a function raises an exception, then the raises section should document the exact exception or error raised. Will also warn if the raises section documents exceptions not explicitly raised in the function or method. (This feature will be disabled by default.)
It may be possible to check the super classes of a given error, to make more general descriptions in the raises section. That would prevent the interface from leaking implementation details. However, that would entail a good deal more work. This may be implemented at a later date, but for now, if you're doing that, then you should just disable the checks for exception/error raising.
-
A
Docstring
class. This class now handles all of the parsing for docstrings, and stores the attributes found from the docstring. It is the docstring corollaryFunctionDescription
.It's attributes are either strings (
Docstring.returns_description
,Docstring.yields_description
,Docstring.short_description
,Docstring.long_description
), or dictionaries containing arguments/exceptions and their descriptions (Docstring.arguments_descriptions
,Docstring.raises_descriptions
). -
We can now ignore certain errors using
noqa
. The syntax fornoqa
is very similar to that used for pycodestyle and other linters, with the exception thatnoqa
, here, can take an argument.Let us say that we want to ignore a missing return statement in the following docstring:
def we_dont_want_a_returns_section(): """Return the value, 3. # noqa: I201 """ return 3
We put the
noqa
anywhere in the top level of the docstring. However, this won't work if we are missing something more specific, like a parameter. We may not want to ignore all missing parameters, either, just one particular one. For example, we may be writing a function that takes a class instance as self. (Say, in a bound celery task.) Then we would do something like:def a_bound_function(self, arg1): """Do something interesting. Args: arg1: The first argument. # noqa: I101 arg1 """ arg1.execute(self)
So, the argument comes to the right of the error.
We may also want to mark excess documentation as being okay. For example, we may not want to explicitly catch and raise a
ZeroDivisionError
. We could do the following:def always_raises_exception(x): """Raise a zero division error or type error.o Args: x: The argument which could be a number or could not be. Raises: ZeroDivisionError: If x is a number. # noqa: I402 TypeError: If x is not a number. # noqa: I402 """ x / 0
So, in this case, the argument for
noqa
is really all the way to the left. (Or whatever description we are parsing.) We could also have put it on its own line, as# noqa: I402 ZeroDivisionError
.
- Setup script was removing the README.rst file, which could not be referenced in the setup script. So, it wasn't showing up as the long description on pypi. This should fix that.
-
Command line interface. There is currently only a single option, verbosity, and a single multi-value argument, files. Help for the command can be accessed using
darglint -h
Example Usage:
darglint -v 3 darglint/*.py
This runs a documentation check on all of the internal modules for darglint, with a verbosity level of 3 (the highest.)
-
Changelog.
-
Handling for methods. Checks for whether the method is a static method or a class method. Doesn't prompt for documentation for "self", or "cls" when not appropriate.
-
Change format for error messages to be less verbose. The format is now:
<function/method name> - <missing argument> + <extraneous argument>
- Removed dependency on Red Baron. This allows us to use line numbers, and to parse python which contains type hints.
- Handle functions/methods without docstrings by ignoring them.
darglint
should only care about whether a docstring is up to date, not whether it is present or not.
Began project. Added check of function definitions. Buggy and doesn't handle many options.