Lint open source repositories for common issues.

Changes in this fork

• Possibility to create a GitHub Issue per rule violation (#1)
• Ability to dynamically pass a ruleset using base64 encoded string (#7)
• New rule that checks whether a file or directory exists (file-or-directory-existence, #19)
• New rule that checks if content exists within a file (any-file-contents, #18)
• Minor optimization if you have a very large repository (lots of commits) so it doesn't do a full checkout. (#12)
• Add ability to specify which Git branches to search for with the file-contents and any-file-content rules.
• Rules: (any)-file-(not)-content can now specifify if they should be executed in sequential mode or not. The option sequentialOnly should be used for this. Default is false. This feature is useful if you check different branches, which is not possible to do without affecting other rule checks.

We are working on getting (some) of these changes merged back into upstream. Time is a limiting factor here..

Installation

Repolinter requires Node.JS >= v12 to function properly. Once Node.JS is installed, you can install Repolinter using npm:

npm install -g repolinter

Linting a Local Repository

Once installed, run the following to lint a directory:

repolinter lint <directory>

The above command will lint <directory> with the local repolinter.json ruleset or the default ruleset if none is found:

repolinter % repolinter lint .
Target directory: <directory>
Lint:
✔ license-file-exists: Found file (LICENSE)
✔ readme-file-exists: Found file (README.md)
✔ contributing-file-exists: Found file (CONTRIBUTING)
✔ code-of-conduct-file-exists: Found file (CODE-OF-CONDUCT)
✔ changelog-file-exists: Found file (CHANGELOG)
...
repolinter % echo $?
0

Linting a Remote Repository

Repolinter also supports linting a git repository using the --git flag. With this flag enabled, the directory input will be interpreted as a git URL which Repolinter will automatically clone into a temporary directory.

repolinter lint -g https://github.com/todogroup/repolinter.git

Formatting the Output

The Repolinter CLI currently supports three output formatting modes:

• Default (also referred to as result)
• JSON
• Markdown

You can switch formatters using the --format flag. An example of using the JSON formatter:

repolinter % repolinter lint --format json .
{"params":{"targetDir":"/Users/nkoontz/Documents/code/repolinter","filterPaths":[],...

An example of using the Markdown formatter:

repolinter % repolinter lint --format markdown .
# Repolinter Report

This Repolinter run generated the following results:
| ❗  Error | ❌  Fail | ⚠️  Warn | ✅  Pass | Ignored | Total |
|---|---|---|---|---|---|
| 0 | 0 | 0 | 15 | 10 | 25 |
...

Limiting Paths

Repolinter supports an allowed list of paths through the --allowPaths option to prevent the accidental linting of build artifacts. These paths must still be contained in the target directory/repository.

repolinter lint --allowPaths ./a/path --allowPaths /another/path

Disabling Modifications

By default Repolinter will automatically execute fixes as specified by the ruleset. If this is not desired functionality, you can disable this with the --dryRun flag.

Ruleset Configuration

Similar to how eslint uses an eslintrc file to determine what validation processes will occur, Repolinter uses a JSON or YAML configuration file (referred to as a ruleset) to determine what checks should be run against a repository. Inside a ruleset, there are two main behaviors that can be configured:

• Rules - Checks Repolinter should perform against the repository.
• Axioms - External libraries Repolinter should use to conditionally run rules.

These combined capabilities give you fine-grained control over the checks Repolinter runs.

Providing a Ruleset

Repolinter will pull its configuration from the following sources in order of priority:

1. A ruleset specified with --rulesetFile or --rulesetUrl
2. A repolint.json, repolinter.json, repolint.yaml, or repolinter.yaml file at the root of the project being linted
3. The default ruleset

Creating a Ruleset

Any ruleset starts with the following base, shown in both JSON and YAML format:

{
  "$schema": "https://raw.githubusercontent.com/todogroup/repolinter/master/rulesets/schema.json",
  "version": 2,
  "axioms": {},
  "rules": {}
}

version: 2
axioms: {}
rules:

Where:

• $schema- points to the JSON schema for all Repolinter rulesets. This schema both validates the ruleset and makes the ruleset creation process a bit easier.
• version - specifies the ruleset version Repolinter should expect. Currently there are two versions: omitted for legacy config (example) and 2 for all others. Use 2 unless you know what you're doing.
• axiom - The axiom functionality, covered in Axioms.
• rules - The actual ruleset, covered in Rules.

Rules

Rules are objects of the following format:

"<rule-name>": {
  "level": "error" | "warning" | "off",
  "rule": {
    "type": "<rule-type>",
    "options": {
      // <rule-options>
    }
  },
  "where": ["condition=*"],
  "fix": {
    "type": "<fix-type>",
    "options": {
      // <fix-options>
    }
  },
  "policyInfo": "...",
  "policyUrl": "..."
}

<rule-name>:
  level: error | warning | off
  rule:
    type: <rule-type>
    options:
      <rule-options>
  where: [condition=*]
  fix:
    type: <fix-type>
    options:
      <fix-options>
  policyInfo: >-
    ...
  policyUrl: >-
    ...

• rule - The check to perform. Repolinter can perform any check listed under the rules documentation. Unlike eslint, Repolinter checks are designed to be reused and specialized: for example, the file-existence check can be used in a README-file-exists rule and a LICENSE-file-exists rule in the same ruleset. This allows a user to write a very specific ruleset from configuring generic checks.
• level - The error level to notify if the check fails. warning will not change the exit code and off will not run the check.
• where - Conditionally enable or disable this rule based off of axioms. Strings in this array follow the format of <axiom>=<value>, where value is either an axiom output or * to simply test if the axiom is enabled. If this option is present, this rule will only run if all specified axiom outputs are present. The available axioms in Repolinter can be found in the axioms documentation.
• fix (optional) - The action to perform if the check performed by rule fails. Repolinter can perform any action listed under fixes documentation.
• policyInfo, policyUrl (optional) - Information used by the formatter to indicate why the check exists from a policy perspective. Note: policyInfo will automatically have a period appended to it for formatting purposes.

A minimal example of a rule that checks for the existence of a README:

"readme-file-exists" : {
  "level": "error",
  "rule": {
    "type": "file-existence",
    "options": {
      "globsAny": ["README*"]
    }
  }
}

readme-file-exists:
  level: error
  rule:
    type: file-existence
    options:
      globsAny:
      - README*

Checking that the README matches a certain hash, and replacing it if not:

"readme-file-up-to-date" : {
  "level": "error",
  "rule": {
    "type": "file-hash",
    "options": {
      "globsAny": ["README*"],
      "algorithm": "sha256",
      "hash": "..."
    }
  },
  "fix": {
    "type": "file-create",
    "options": {
      "file": "README.md",
      "replace": true,
      "text": { "url": "www.example.com/mytext.txt" }
    }
  },
  "policyInfo": "Gotta keep that readme up to date",
  "policyUrl": "www.example.com/mycompany"
}

readme-file-up-to-date:
  level: error
  rule:
    type: file-hash
    options:
      globsAny:
      - README*
      algorithm: sha256
      hash: "..."
  fix:
    type: file-create
    options:
      file: README.md
      replace: true
      text:
        url: www.example.com/mytext.txt
  policyInfo: Gotta keep that readme up to date
  policyUrl: www.example.com/mycompany

Axioms

"axioms": {
  "<axiom-id>": "<axiom-target>"
}

axioms:
  <axiom-id>: axiom-target

Each axiom is configured as a key value pair in the axioms object, where <axiom-id> specifies the program to run and <axiom-target> specifies the target to be used in the where conditional. The available axiom IDs can be found in the axiom documentation. It should be noted that some axioms require external packages to run.

An example configuration using an axiom to detect the packaging system for a project:

{
  "$schema": "https://raw.githubusercontent.com/todogroup/repolinter/master/rulesets/schema.json",
  "version": 2,
  "axioms": {
    "packagers": "package-type"
  },
  "rules": {
    "this-only-runs-if-npm": {
      "level": "error",
      "where": ["package-type=npm"],
      "rule": { /* ... */ }
    }
  }
}

version: 2
axioms:
  packagers: package-type
rules:
  this-only-runs-if-npm:
    level: error
    where: [package-type=npm]
    rule:
      ...

Some axioms (ex. contributor-count) output numerical values instead of strings. For these axioms, numerical comparisons (<, >, <=, >=) can be also be specified in the where conditional. Note that if a numerical comparison is used for a non-numerical axiom, the comparison will always fail.

{
  "axioms": {
    "contributor-count": "contributors"
  },
  "rules": {
    "my-rule": {
      "where": ["contributors>6", "contributors<200"],
      // ...
    }
  }
}

axioms:
  contributor-count: contributors
rules:
  my-rule:
    where:
    - contributors>6
    - contributors<200
    rule:
      ...

Extending Rulesets

A ruleset can extend another ruleset, in which case the two files will be recursively merged. Extended rulesets can themselves extend additional rulesets up to 20 rulesets deep.

Extend a ruleset by including an "extends" top-level key which identifies a URL or file path:

{
  "extends": "https://raw.githubusercontent.com/todogroup/repolinter/master/rulesets/default.json"
  "rules": {
    # disable CI check
    "integrates-with-ci": {
      "level": "off"
    }
  }
}

extends: https://raw.githubusercontent.com/todogroup/repolinter/master/rulesets/default.json
rules:
  # disable CI check
  integrates-with-ci
    level: off
    ...

Relative paths are resolved relative to the location used to access the extending file. For example, if repolinter is invoked as:

repolinter -u http://example.com/custom-rules.yaml

And that ruleset includes extends: "./default.yaml", the path will be resolved relative to the original URL as http://example.com/default.yaml. If instead repolinter is invoked as:

repolinter -r /etc/repolinter/custom-rules.yaml

And that ruleset includes extends: "./default.yaml", the path will be resolved relative to the original file path as /etc/repolinter/default.yaml.

YAML and JSON rulesets can be extended from either format.

API

Repolinter also includes an extensible JavaScript API:

const repolinter = require('repolinter')
const result = await repolinter.lint('.')

This API allows the developer to have complete control over the configuration and formatting Repolinter should use. Documentation for this library can be found under API Documentation.

Going Further

• Rule Reference
• Fix Reference
• Axiom Reference
• API Reference
• Developer Guide

License

This project is licensed under the Apache 2.0 license.