Incomplete parser model and sample application for parsing Core Rule Set written in the ModSecurity DSL SecRule language. It uses the python library textX for parsing.
-
Install dependencies Dependencies can be installed system-wide, or just for your user (using
--user
).System-wide:
sudo pip install secrules-parsing
User:
pip install --user secrules-parsing
-
Execute
secrules-parser
specifying the location of the files you want to scan using the -f/--files argument. This takes wildcards or individual files.$ secrules-parser -c -f /owasp-crs/rules/*.conf
-
Add flags to accomplish needed tasks:
-
-h, --help:
- Description: show the help message and exit
- Example:
$ secrules-parser -h
-
-r, --regex:
- Description: Extract regular expressions from rules file
- Example:
$ secrules-parser --regex -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf {"/owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf": [{"920100": ["^(?i:(?:[a-z]{3,10}\\s+(?:\\w{3,7}?://[\\w\\-\\./]*(?::\\d+)?)?/[^?#]*(?:\\?[^#\\s]*)?(?:#[\\S]*)?|connect (?:\\d{1,3}\\.){3}\\d{1,3}\\.?(?::\\d+)?|options \\*)\\s+[\\w\\./]+|get /[^?#]*(?:\\?[^#\\s]*)?(?:#[\\S]*)?)$"]}, {"920120": ["(?<!&(?:[aAoOuUyY]uml)|&(?:[aAeEiIoOuU]circ)|&(?:[eEiIoOuUyY]acute)|&(?:[aAeEiIoOuU]grave)|&(?:[cC]cedil)|&(?:[aAnNoO]tilde)|&(?:amp)|&(?:apos));|['\\\"=]"]}, {"920160": ["^\\d+$"]}, {"920170": ["^(?:GET|HEAD)$"]}, {"920171": ["^(?:GET|HEAD)$"]}, {"920180": ["^POST$"]}, {"920190": ["(\\d+)\\-(\\d+)\\,"]}, {"920210": ["\\b(?:keep-alive|close),\\s?(?:keep-alive|close)\\b"]}, {"920220": ["\\%(?:(?!$|\\W)|[0-9a-fA-F]{2}|u[0-9a-fA-F]{4})"]}, {"920240": ["^(?:application\\/x-www-form-urlencoded|text\\/xml)(?:;(?:\\s?charset\\s?=\\s?[\\w\\d\\-]{1,18})?)??$"]}, {"920260": ["\\%u[fF]{2}[0-9a-fA-F]{2}"]}, {"920290": ["^$"]}, {"920310": ["^$"]}, {"920311": ["^$"]}, {"920330": ["^$"]}, {"920340": ["^0$"]}, {"920350": ["^[\\d.:]+$"]}, {"920420": ["^(?:GET|HEAD|PROPFIND|OPTIONS)$"]}, {"920440": ["\\.(.*)$"]}, {"920450": ["^.*$"]}, {"920200": ["^bytes=(?:(?:\\d+)?\\-(?:\\d+)?\\s*,?\\s*){6}"]}, {"920230": ["\\%((?!$|\\W)|[0-9a-fA-F]{2}|u[0-9a-fA-F]{4})"]}, {"920121": ["['\\\";=]"]}, {"920460": ["(?<!\\Q\\\\\\E)\\Q\\\\\\E[cdeghijklmpqwxyz123456789]"]}]}
-
-c, --correctness:
- Description: Check the validity of the syntax
- Example:
$ secrules-parser -c -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf Syntax OK: ../../../rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
-
-v, --verbose
- Description: Print verbose messages
- Example:
$ secrules-parser -c -v -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf ...
-
-o FILE, --output FILE
- Description: Output results to file
- Example:
$ secrules-parser -c -o out.json -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
-
--output-type github | plain
- Description: Desired output format. Useful if running from Github Actions and you want annotated output
- Example:
$ secrules-parser -c --output-type github -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
Takes a list of file path's and returns models
import glob
import os
from secrules_parsing import parser
# Extract all of our pathing
files = glob.glob("../../rules/*.conf")
# Pass absolute paths because of module location
files = [os.path.abspath(path) for path in files]
models = parser.process_rules(files)
import glob
import os
from secrules_parsing import parser
# Extract all of our pathing
files = glob.glob("../../rules/*.conf")
# Pass absolute paths because of module location
files = [os.path.abspath(path) for path in files]
models = parser.process_rules(files)
parser.get_correctness(files, models)
If you want to modify this module, follow these steps:
- Clone this repository:
git clone git@github.com:coreruleset/secrules_parsing.git
- Do not forget to install dependencies using poetry:
poetry install
first! - Edit and change the files you want.
- Write tests! Tests are in the
tests
subdirectory - Create a PR here, and ask for review!
To visualize the syntax tree, use:
textx visualize secrules.tx
dot -Tpng -O secrules.tx.dot
Then review the generated PNG modsec.tx.dot.png!
Please file an issue if you find a bug or you want some feature added.