diff --git a/README.md b/README.md index e95e0035..e89c17f0 100644 --- a/README.md +++ b/README.md @@ -27,6 +27,7 @@ More detailed information and examples about the specific usage of the additiona |:------------|:------------|:--------------| | [navlayers](https://github.com/mitre-attack/mitreattack-python/tree/master/mitreattack/navlayers) | A collection of utilities for working with [ATT&CK Navigator](https://github.com/mitre-attack/attack-navigator) layers. Provides the ability to import, export, and manipulate layers. Layers can be read in from the filesystem or python dictionaries, combined and edited, and then exported to excel or SVG images. | Further documentation can be found [here](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/README.md).| | [attackToExcel](https://github.com/mitre-attack/mitreattack-python/tree/master/mitreattack/attackToExcel) | A collection of utilities for converting [ATT&CK STIX data](https://github.com/mitre/cti) to Excel spreadsheets. It also provides access to [Pandas](https://pandas.pydata.org/) DataFrames representing the dataset for use in data analysis. | Further documentation can be found [here](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/attackToExcel/README.md).| +| [attackToJava](https://github.com/mitre-attack/mitreattack-python/tree/master/mitreattack/attackToJava) | An utility for converting [ATT&CK STIX data](https://github.com/mitre/cti) to Java class hierarchy. It uses the Pandas dataframe from attackToExcel| Further documentation can be found [here](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/attackToJava/README.md).| | [collections](https://github.com/mitre-attack/mitreattack-python/tree/master/mitreattack/collections) | A set of utilities for working with [ATT&CK Collections and Collection Indexes](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md). Provides functionalities for converting and summarizing data in collections and collection indexes, as well as generating a collection from a raw stix bundle input. | Further documentation can be found [here](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/collections/README.md).| | [diffStix](https://github.com/mitre-attack/mitreattack-python/tree/master/mitreattack/diffStix) | Create markdown, HTML, JSON and/or ATT&CK Navigator layers reporting on the changes between two versions of the STIX2 bundles representing the ATT&CK content. Run `diff_stix -h` for full usage instructions. | Further documentation can be found [here](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/diffStix/README.md).| diff --git a/docs/attacktojava.rst b/docs/attacktojava.rst new file mode 100644 index 00000000..d5a04924 --- /dev/null +++ b/docs/attacktojava.rst @@ -0,0 +1,93 @@ +ATT&CK to Java +============================================== + +ATT&CK to Java contains a module for converting `ATT&CK STIX data `_ to Java class hierarchy. + + +Usage: +----- + +Command Line +----- + +Print full usage instructions: + +.. code:: bash + + python3 attackToJava.py -h + + +Example execution: + +.. code:: bash + + python3 attackToJava.py -output /tmp/attack + + +Build a Java files corresponding to a version of ATT&CK: + +.. code:: bash + + python3 attackToJava -output /tmp/attack -version v5.0 + + + +Interfaces: +----- + +attackToJava +----- +attackToJava provides the means by which to convert/extract the ATT&CK STIX data to Java class hierarchy. +A brief overview of the available methods follows. + +.. list-table:: Title + :widths: 33 33 34 + :header-rows: 1 + + * - method name + - arguments + - usage + * - export + - `domain`: the domain of ATT&CK to download
`version`: optional parameter specifying which version of ATT&CK to download
`output_dir`: optional parameter specifying output directory + - `version` : The version of ATT&CK to download, e.g "v8.1". If omitted will build the current version of ATT&CK, by default None + - `output_dir` : The directory to write the Java files to. + - `remote` : The URL of a remote ATT&CK Workbench instance to connect to for stix data. Mutually exclusive with stix_file. + - `stix_file` : Path to a local STIX file containing ATT&CK data for a domain, by default None + - Download ATT&CK data from MITRE/CTI and convert it to Java class hierarchy + +stixToJava +----- + +stixToJava provides various methods to process and manipulate the STIX data in order to create Java + +.. list-table:: Method Documentation + :widths: 33 33 34 + :header-rows: 1 + + * - method name + - arguments + - usage + * - runMaven + - `output_dir`: str + - Run Maven to build the Java classes.
`output_dir`: The directory to run Maven in, by default "." + * - remove_tautology + - `text`: str + - Remove tautology from the text.
`text`: The text to process.
Returns the processed text without tautology. + * - formatTextToLines + - `text`: str
`max_line_length`: int = 80 + - Format text to lines of a specified maximum length.
`text`: The text to format.
`max_line_length`: The maximum line length, by default 80.
Returns the formatted lines. + * - buildOutputDir + - `package_name`: str = None
`output_dir`: str = None + - Build the output directory for the Java classes.
`package_name`: The name of the package to create the directory for.
`output_dir`: The root directory for output.
Returns the path to the output directory. + * - nameToClassName + - `name`: str + - Convert a name to a class name.
`name`: The name to convert.
Returns the class name. + * - writeJinja2Template + - `templateEnv`: jinja2.Environment
`template_name`: str
`output_file`: str
`fields`: dict + - Write a Jinja2 template to a file.
`templateEnv`: The Jinja2 environment.
`template_name`: The template file to use.
`output_file`: The output file to write to.
`fields`: The fields to use in the template. + * - stixToTactics + - `stix_data`: MemoryStore
`package_name`: str
`domain`: str
`verbose_class`: bool = False
`output_dir`: str = "." + - Parse STIX tactics from the given data and write corresponding Java classes.
`stix_data`: MemoryStore or other stix2 DataSource object holding the domain data.
`package_name`: The base package name for the output Java classes.
`domain`: The domain of ATT&CK stix_data corresponds to, e.g., "enterprise-attack".
`verbose_class`: Whether to include verbose class information, by default False.
`output_dir`: The root directory for output, by default ".". + * - stixToTechniques + - `all_data_sources`: dict
`all_defenses_bypassed`: dict
`all_platforms`: dict
`stix_data`: MemoryStore
`package_name`: str
`domain`: str
`verbose_class`: bool = False
`output_dir`: str = "." + - Parse STIX techniques from the given data and write corresponding Java classes.
`all_data_sources`: Dictionary to hold all data sources.
`all_defenses_bypassed`: Dictionary to hold all defenses bypassed.
`all_platforms`: Dictionary to hold all platforms.
`stix_data`: MemoryStore or other stix2 DataSource object holding the domain data.
`package_name`: The base package name for the output Java classes.
`domain`: The domain of ATT&CK stix_data corresponds to, e.g., "enterprise-attack".
`verbose_class`: Whether to include verbose class information, by default False.
`output_dir`: The root directory for output, by default ".". diff --git a/docs/index.rst b/docs/index.rst index a624e96d..adb95c1f 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -30,5 +30,6 @@ other modules in this library under "Additional Modules". additional_modules/navlayers additional_modules/attackToExcel + additional_modules/attackToJava additional_modules/collections additional_modules/diffStix \ No newline at end of file diff --git a/mitreattack/__init__.py b/mitreattack/__init__.py index 3bc2aa08..b6acc5e4 100644 --- a/mitreattack/__init__.py +++ b/mitreattack/__init__.py @@ -1,3 +1,4 @@ from .attackToExcel import * +from .attackToJava import * from .navlayers import * from .collections import * diff --git a/mitreattack/attackToJava/README.md b/mitreattack/attackToJava/README.md new file mode 100644 index 00000000..3fbe5fc4 --- /dev/null +++ b/mitreattack/attackToJava/README.md @@ -0,0 +1,3 @@ +# ATT&CK To Java + + diff --git a/mitreattack/attackToJava/__init__.py b/mitreattack/attackToJava/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/mitreattack/attackToJava/attackToJava.py b/mitreattack/attackToJava/attackToJava.py new file mode 100644 index 00000000..e6eace24 --- /dev/null +++ b/mitreattack/attackToJava/attackToJava.py @@ -0,0 +1,162 @@ +"""Functions to convert ATT&CK STIX data to Java, as well as entrypoint for attackToJava_cli.""" + +import argparse +import os +from typing import Dict, List +from sortedcontainers import SortedDict + +import pandas as pd +import requests +from loguru import logger +from stix2 import MemoryStore +from pprint import pprint + +INVALID_CHARACTERS = ["\\", "/", "*", "[", "]", ":", "?"] +SUB_CHARACTERS = ["\\", "/"] + +from mitreattack.attackToExcel import attackToExcel +from mitreattack.attackToJava import stixToJava +from mitreattack.attackToJava import getJavaImports + + + +def export( + version: str = None, + output_dir: str = None, + remote: str = None, + stix_path: str = None, + package_name: str = None, + verbose_class: bool = False, + ): + """Download ATT&CK data from MITRE/CTI and convert it to Java class hierarchy + + Parameters + ---------- + domain : str, optional + The domain of ATT&CK to download, e.g "enterprise-attack", by default "enterprise-attack" + version : str, optional + The version of ATT&CK to download, e.g "v8.1". + If omitted will build the current version of ATT&CK, by default None + output_dir : str, optional + The directory to write the excel files to. + If omitted writes to a subfolder of the current directory depending on specified domain and version, by default "." + remote : str, optional + The URL of a remote ATT&CK Workbench instance to connect to for stix data. + Mutually exclusive with stix_file. + by default None + stix_file : str, optional + Path to a local STIX file containing ATT&CK data for a domain, by default None + + Raises + ------ + ValueError + Raised if both `remote` and `stix_file` are passed + """ + + if not package_name: + raise ValueError("Package name needs to be specified") + + if not output_dir: + raise ValueError("Output directory needs to be specified") + + if output_dir == ".": + raise ValueError("Output directory cannot be the current directory, as the output directoty will be deleted and recreated. Please specify a valid directory") + + if remote and stix_path: + raise ValueError("remote and stix_file are mutually exclusive. Please only use one or the other") + + #Verify that if stix path is specified it contains JSONs for all three domains + if stix_path: + if os.path.exists(os.path.join(stix_path, "enterprise-attack.json")) and os.path.exists(os.path.join(stix_path, "mobile-attack.json")) and os.path.exists(os.path.join(stix_path, "ics-attack.json")): + pass + else: + raise ValueError("""stix_path must contain JSON files for all three domains: enterprise-attack.json, mobile-attack.json, ics-attack.json. + Use download_attack_stix tool to fetch the files""") + + all_data_sources = SortedDict() + all_defenses_bypassed = SortedDict() + all_platforms = SortedDict() + + stixToJava.buildOutputDir(package_name=package_name, output_dir=output_dir) + + for domain in ["enterprise-attack", "mobile-attack", "ics-attack"]: + + logger.info(f"************ Exporting {domain} to To Java ************") + + if stix_path: + #Use local files if stix path is specified + mem_store = attackToExcel.get_stix_data(domain=domain, version=version, remote=remote, stix_file=os.path.join(stix_path, f"{domain}.json")) + else: + mem_store = attackToExcel.get_stix_data(domain=domain, version=version, remote=remote) + + stixToJava.stixToTactics(stix_data=mem_store, package_name=package_name, domain=domain, verbose_class=verbose_class,output_dir=output_dir) + + stixToJava.stixToTechniques(all_data_sources,all_defenses_bypassed,all_platforms,stix_data=mem_store, package_name=package_name, domain=domain, verbose_class=verbose_class,output_dir=output_dir) + + logger.info(f"************ Generating import statements for easy use ************") + + + logger.info(f"************ Running Maven to format and test ************") + + with open(os.path.join(output_dir, "imports_example.txt"), "w") as f: + for import_line in getJavaImports.getJavaImports(output_dir,package_name): + f.write(f"{import_line}\n") + + stixToJava.runMaven(output_dir=output_dir) + + + + + +def main(): + """Entrypoint for attackToExcel_cli.""" + parser = argparse.ArgumentParser( + description="Download ATT&CK data from MITRE/CTI and convert it to excel spreadsheets" + ) + + parser.add_argument( + "-version", + type=str, + help="which version of ATT&CK to convert. If omitted, builds the latest version", + ) + parser.add_argument( + "-output", + type=str, + required=True, + help="output directory. If omitted writes to a subfolder of the current directory depending on " + "the domain and version", + ) + parser.add_argument( + "-remote", + type=str, + default=None, + help="remote url of an ATT&CK workbench server. If omitted, stix data will be acquired from the" + " official ATT&CK Taxii server (cti-taxii.mitre.org)", + ) + parser.add_argument( + "-stix-path", + type=str, + default=None, + help="Path to a local directory containing downlaoded STIX filse containing ATT&CK data for all supported domains (enterprise,mobile,ICS) by default None", + ) + + parser.add_argument( + "-package", + type=str, + default="org.mitre.attack", + help="Java package name from which to start the class hierarchy. If omitted, will use the org.mitre.attack is used", + ) + + parser.add_argument( + "-verbose", + action="store_true", + help="Populate all fields in Java class, including description and other non-essential. Note this will increase memory usage and file size.", + ) + args = parser.parse_args() + + export(version=args.version, output_dir=args.output, remote=args.remote, stix_path=args.stix_path, package_name=args.package, verbose_class=args.verbose + ) + + +if __name__ == "__main__": + main() diff --git a/mitreattack/attackToJava/getJavaImports.py b/mitreattack/attackToJava/getJavaImports.py new file mode 100644 index 00000000..0fc00f62 --- /dev/null +++ b/mitreattack/attackToJava/getJavaImports.py @@ -0,0 +1,78 @@ +""" +print_java_imports.py + +This Python script generates Java import statements for all packages in a given directory. +It's useful when you want to import all classes from all packages in a directory. + +Usage: + Run the script from the command line with the directory as an argument: + python print_java_imports.py + Replace with the path to the directory for which you want to generate import statements. + +Functions: + createImportStatement(package_path): + This function takes a package path as an argument and returns a string that is a Java import statement + for all classes in that package. + + print_java_imports(directory): + This function walks through the directory structure of the provided directory and prints an import + statement for each subdirectory. + +Error handling: + The script checks if the provided argument is a valid directory. If it's not, it prints an error message + and exits with a status code of 1. + + If the script is run without exactly one argument, it prints a usage message and exits with a status code of 1. +""" +import os +import sys + +def createImportStatement(package_path): + """ + This function takes a package path as an argument and returns a string that is a Java import statement + for all classes in that package. + """ + return f"import {package_path.replace(os.sep, '.')}.*;" + +def getJavaImports(directory, package_name): + """ + This function walks through the directory structure of the provided directory and returns a list of import + statements for each subdirectory. + """ + import_statements = [] + for root, dirs, files in os.walk(directory): + + # Get the relative path from the directory to the current root + relative_path = os.path.relpath(root, directory) + + if package_name not in relative_path.replace(os.sep,"."): + #Skip directories that are not part of the package + continue + + #remove everything before the package name, but keep the package name + relative_path = relative_path[relative_path.index(package_name.replace(".",os.sep)):] + + # Skip the current directory (.) + if relative_path == ".": + continue + + # Create and add the import statement to the list + import_statement = createImportStatement(relative_path) + import_statements.append(import_statement) + + return import_statements + +if __name__ == "__main__": + if len(sys.argv) != 2: + print("Usage: python print_java_imports.py ") + sys.exit(1) + + directory_path = sys.argv[1] + + if not os.path.isdir(directory_path): + print(f"The path {directory_path} is not a valid directory.") + sys.exit(1) + + import_statements = getJavaImports(directory_path) + for statement in import_statements: + print(statement) \ No newline at end of file diff --git a/mitreattack/attackToJava/stixToJava.py b/mitreattack/attackToJava/stixToJava.py new file mode 100644 index 00000000..757afadb --- /dev/null +++ b/mitreattack/attackToJava/stixToJava.py @@ -0,0 +1,428 @@ +from mitreattack.attackToExcel import stixToDf +from stix2 import Filter, MemoryStore + + +from pprint import pprint +from loguru import logger +from sortedcontainers import SortedDict +from sortedcontainers import SortedSet +from sortedcontainers import SortedList + +import os +import sys +import jinja2 +import shutil + +script_dir = os.path.dirname(os.path.realpath(__file__)) +template_dir = os.path.join(script_dir, "templates") + + +def runMaven(output_dir: str): + """Run Maven to build the Java classes + + Parameters + ---------- + output_dir : str required + The directory to run Maven in + """ + logger.info("Running Maven") + os.system(f"mvn -f {output_dir} clean compile package") + +#Function to remove tautology from the text, that is remove parts of the beginning of the text that are repeated +#that is WINDOWS_REGISTRY_WINDOWS_REGISTRY_KEY_CREATION should be reduced to WINDOWS_REGISTRY_KEY_CREATION +def remove_tautology(text: str): + text_parts = text.split("_") + if len(text_parts) < 2: + return text + + #Find the first part that is repeated, going each part alone + for i in range(1, len(text_parts)): + if text_parts[i] == text_parts[i-1]: + return "_".join(text_parts[i:]) + + #Repeat same by combining two parts + if len(text_parts) > 4: + #Combine two parts and check if they are repeated + for i in range(2, len(text_parts)): + if text_parts[i] == text_parts[i-2] and text_parts[i-1] == text_parts[i-3]: + return "_".join(text_parts[i-1:]) + + return text + +def formatTextToLines(text: str, max_line_length: int = 80): + """Format text to lines of max_line_length + + Parameters + ---------- + text : str + The text to format + + max_line_length : int, optional + The maximum line length, by default 80 + + Returns + ------- + list + The formatted lines + """ + + lines = [] + for line in text.split("\n"): + while len(line) > max_line_length: + #Find the last space before 80 characters + last_space = line.rfind(" ", 0, max_line_length) + if last_space == -1: + last_space = max_line_length + lines.append(line[:last_space]) + line = line[last_space+1:] + lines.append(line) + return "\n".join(lines) + + +def buildOutputDir(package_name: str = None, output_dir: str = None): + """ + Build the output directory for the Java classes + + Parameters + ---------- + package_name : str + The name of the package to create the directory for + + Returns + ------- + str + The path to the output directory + """ + + + package_root_dir = os.path.join(output_dir,"src","main","java", package_name.replace(".", os.sep) ) + + #Remove the output directory if it exists and is other than current dir + if output_dir != "." and os.path.exists(output_dir): + if os.path.exists(output_dir): + shutil.rmtree(output_dir) + + os.makedirs(package_root_dir, exist_ok=True) + +def nameToClassName(name: str): + """Convert a name to a class name + + Parameters + ---------- + name : str + The name to convert + + Returns + ------- + str + The class name + """ + #Make sure name field does not have spaces and every word is capitalized + #There can be " " or "-" or "_" in the name so split by all of them + name= name.replace("-", " ") + name= name.replace("_", " ") + name= name.replace("/", " ") + name= name.replace("(", " ") + name= name.replace(")", " ") + name= name.replace("&", " and ") + name_parts = name.split(" ") + + return "".join([part.capitalize() for part in name_parts]) + +def writeJinja2Template(templateEnv,template_name: str, output_file: str, fields: dict): + """Write a Jinja2 template to a file + + Parameters + ---------- + template_file : str + The template file to use + + output_file : str + The output file to write to + + fields : dict + The fields to use in the template + """ + + #make sure output path exists + os.makedirs(os.path.dirname(output_file), exist_ok=True) + + template = templateEnv.get_template(template_name) + outputText = template.render(fields) + + #Jinja2 for loop is producing pythonic empty lines so remove lines with something else than newline for empty lines + #outputText = "\n".join([line for line in outputText.split("\n") if line.strip() != ""]) + + with open(output_file, "w") as f: + logger.info(f"Writing {output_file}") + f.write(outputText) + + +def stixToTactics(stix_data: MemoryStore, package_name: str, domain: str , verbose_class: bool = False, output_dir: str ="."): + + package_root_dir = os.path.join(output_dir,"src","main","java", package_name.replace(".", os.sep) ) + + #Add Tactic to the base package name + root_package_name = package_name + package_name = f"{package_name}.tactic" + + package_dir = os.path.join(package_root_dir, "tactic" ) + os.makedirs(package_dir, exist_ok=True) + + tactics = stix_data.query([Filter("type", "=", "x-mitre-tactic")]) + tactics = stixToDf.remove_revoked_deprecated(tactics) + + tactic_rows = [] + for tactic in tactics: + tactic_rows.append(stixToDf.parseBaseStix(tactic)) + + #Use Jinja2 to load and render the template + templateLoader = jinja2.FileSystemLoader(searchpath=template_dir) + templateEnv = jinja2.Environment(loader=templateLoader) + tactic={} + tactic["package_name"] = package_name + tactic["root_package_name"] = root_package_name + #Write the AbstractTactic.java + writeJinja2Template(templateEnv, "AbstractTactic.jinja2", os.path.join(package_dir,"AbstractTactic.java"), tactic) + + for tactic in tactic_rows: + tactic["domain"]= domain + tactic["package_name"] = package_name + tactic["root_package_name"] = root_package_name + #Make sure name field does not have spaces and every word is capitalized + tactic["class_name"] = nameToClassName(tactic["name"]) + + if "description" in tactic: + tactic["description_field"] = tactic["description"].replace("\\", "\\\\").replace('"', "'").replace("\n", "") + tactic["description_lines"] = formatTextToLines(tactic["description_field"]) + + if "detection" in tactic: + tactic["detection_field"] = tactic["detection"].replace("\\", "\\\\").replace('"', "'").replace("\n", "") + tactic["detection_lines"] = formatTextToLines(tactic["detection_field"]) + + #Write the Tactic as Interface as techniques commonly can be present in multiple tactics + writeJinja2Template(templateEnv, "Tactic.jinja2", os.path.join(package_dir,f"{tactic['class_name']}.java"), tactic) + + #Write the GenericTactic to be used in cases when specific technique is not known + writeJinja2Template(templateEnv, "GenericTactic.jinja2", os.path.join(package_dir,f"Generic{tactic['class_name']}.java"), tactic) + + +def stixToTechniques(all_data_sources:SortedDict, all_defenses_bypassed:SortedDict ,all_platforms:SortedDict ,stix_data: MemoryStore,package_name: str, domain , verbose_class: bool = False, output_dir: str ="."): + """Parse STIX techniques from the given data and write corresponding Java classes + + :param stix_data: MemoryStore or other stix2 DataSource object holding the domain data + :param domain: domain of ATT&CK stix_data corresponds to, e.g "enterprise-attack" + """ + + package_root_dir = os.path.join(output_dir,"src","main","java", package_name.replace(".", os.sep) ) + + domain_bare = domain.replace("-attack", "") + + domain_package_dir = os.path.join(package_root_dir, domain_bare ) + + techniques = stix_data.query([Filter("type", "=", "attack-pattern")]) + techniques =stixToDf.remove_revoked_deprecated(techniques) + technique_rows = [] + + tactics = stix_data.query([Filter("type", "=", "x-mitre-tactic")]) + tactics =stixToDf.remove_revoked_deprecated(tactics) + tactic_names = {} + for tactic in tactics: + x_mitre_shortname = tactic["x_mitre_shortname"] + tactic_names[x_mitre_shortname] = tactic["name"] + + all_sub_techniques = stix_data.query( + [ + Filter("type", "=", "relationship"), + Filter("relationship_type", "=", "subtechnique-of"), + ] + ) + all_sub_techniques = MemoryStore(stix_data=all_sub_techniques) + + for technique in techniques: + # get parent technique if sub-technique + #pprint(technique) + subtechnique = "x_mitre_is_subtechnique" in technique and technique["x_mitre_is_subtechnique"] + if subtechnique: + subtechnique_of = all_sub_techniques.query([Filter("source_ref", "=", technique["id"])])[0] + parent = stix_data.get(subtechnique_of["target_ref"]) + + # base STIX properties + row =stixToDf.parseBaseStix(technique) + + # sub-technique properties + if "kill_chain_phases" not in technique: + attack_id = technique['external_references'][0]['external_id'] + logger.error(f"Skipping {attack_id} [{technique['id']}] because it does't have kill chain phases") + continue + tactic_shortnames = [] + for kcp in technique["kill_chain_phases"]: + tactic_shortnames.append(kcp["phase_name"]) + + technique_tactic_names = [] + implements = [] + for shortname in tactic_shortnames: + tactic_display_name = tactic_names[shortname] + technique_tactic_names.append(tactic_display_name) + implements.append(f"{package_name}.tactic.{nameToClassName(tactic_display_name)}") + row["tactics"] = ", ".join(sorted(technique_tactic_names)) + + #remove the last comma and space, if they are present + row["implements"] = False + if len(implements) > 0: + row["implements"] = ", ".join(sorted(implements)) + + if "x_mitre_detection" in technique: + row["detection"] = technique["x_mitre_detection"] + if "x_mitre_platforms" in technique: + row["platforms"] = ", ".join(sorted(technique["x_mitre_platforms"])) + + # domain specific fields -- ICS + Enterprise + if domain in ["enterprise-attack", "ics-attack"]: + if "x_mitre_data_sources" in technique: + row["data sources"] = ", ".join(sorted(technique["x_mitre_data_sources"])) + + row["class_name"] = nameToClassName(f"{domain_bare} {technique['name']}") + + row["extends"] = f"{package_name}.MitreTTP" + + # domain specific fields -- enterprise + if domain == "enterprise-attack": + row["is_sub-technique"] = subtechnique + + if subtechnique: + parent_name= nameToClassName(f"{domain_bare} {parent['name']}") + parent_name_bare = nameToClassName(f"{parent['name']}") + row["sub-technique of"] = parent["external_references"][0]["external_id"] + row["extends"] = f"{package_name}.{domain_bare}.technique.{parent_name}" + row["parent_name"] = parent_name + row["parent_name_bare"] = parent_name_bare + + if "x_mitre_system_requirements" in technique: + row["system requirements"] = ", ".join(sorted(technique["x_mitre_system_requirements"])) + if "x_mitre_permissions_required" in technique: + row["permissions required"] = ", ".join( + sorted(technique["x_mitre_permissions_required"], key=str.lower) + ) + if "x_mitre_effective_permissions" in technique: + row["effective permissions"] = ", ".join( + sorted(technique["x_mitre_effective_permissions"], key=str.lower) + ) + + if "defense-evasion" in tactic_shortnames and "x_mitre_defense_bypassed" in technique: + row["defenses bypassed"] = ", ".join(sorted(technique["x_mitre_defense_bypassed"])) + if "execution" in tactic_shortnames and "x_mitre_remote_support" in technique: + row["supports remote"] = technique["x_mitre_remote_support"] + if "impact" in tactic_shortnames and "x_mitre_impact_type" in technique: + row["impact type"] = ", ".join(sorted(technique["x_mitre_impact_type"])) + capec_refs = list( + filter( + lambda ref: ref["source_name"] == "capec", + technique["external_references"], + ) + ) + if capec_refs: + row["CAPEC ID"] = ", ".join([x["external_id"] for x in capec_refs]) + + # domain specific fields -- mobile + elif domain == "mobile-attack": + if "x_mitre_tactic_type" in technique: + row["tactic type"] = ", ".join(sorted(technique["x_mitre_tactic_type"])) + mtc_refs = list( + filter( + lambda ref: ref["source_name"] == "NIST Mobile Threat Catalogue", + technique["external_references"], + ) + ) + if mtc_refs: + row["MTC ID"] = mtc_refs[0]["external_id"] + + #modify the row dictionary keys so that they do not have spaces, as those can't be used in Jinja2 templates easily + row = {key.replace(" ", "_"): value for key, value in row.items()} + + #Add all data source entries from row to all_data_sources + data_source_keys = SortedSet() + if "data_sources" in row: + for data_source in row["data_sources"].split(", "): + #convert key value to all uppercase and relace spaces with underscores + data_source_key = data_source.upper().replace(" ", "_").replace(":", "_").replace("/", "_").replace("__", "_") + data_source_key= remove_tautology(data_source_key) + data_source_keys.add(data_source_key) + all_data_sources[data_source_key] = data_source + row["data_source_keys"] = data_source_keys + + defense_bypassed_keys = SortedSet() + if "defenses_bypassed" in row: + for defense_bypassed in row["defenses_bypassed"].split(", "): + #convert key value to all uppercase and relace spaces with underscores + defense_bypassed_key = defense_bypassed.upper().replace(" ", "_").replace(":", "_").replace("/", "_").replace("__", "_").replace("-", "_") + defense_bypassed_key= remove_tautology(defense_bypassed_key) + defense_bypassed_keys.add(defense_bypassed_key) + all_defenses_bypassed[defense_bypassed_key] = defense_bypassed + row["defense_bypassed_keys"] = defense_bypassed_keys + + platform_keys = SortedSet() + if "platforms" in row: + for platform in row["platforms"].split(", "): + #convert key value to all uppercase and relace spaces with underscores + platform_key = platform.upper().replace(" ", "_").replace(":", "_").replace("/", "_").replace("__", "_").replace("-", "_") + platform_key= remove_tautology(platform_key) + platform_keys.add(platform_key) + all_platforms[platform_key] = platform + row["platform_keys"] = platform_keys + + if "description" in row: + row["description_field"] = row["description"].replace("\\", "\\\\").replace('"', "'").replace("\n", "") + row["description_lines"] = formatTextToLines(row["description_field"]) + + if "detection" in row: + row["detection_field"] = row["detection"].replace("\\", "\\\\").replace('"', "'").replace("\n", "") + row["detection_lines"] = formatTextToLines(row["detection_field"]) + + technique_rows.append(row) + + + #Produce data sources enum from the collected data source items + templateLoader = jinja2.FileSystemLoader(searchpath=template_dir) + templateEnv = jinja2.Environment(loader=templateLoader) + + #split the package name into organization and package name f.ex org.mitre.attack -> org.mitre, attack + organization, package_bare = package_name.rsplit(".", 1) + + if domain=="enterprise-attack": + #Write common files for all domains when Enterprise domain is being processed + writeJinja2Template(templateEnv, "pom.jinja2", os.path.join(output_dir,"pom.xml"), {"organization":organization,"package_bare":package_bare}) + writeJinja2Template(templateEnv, "MitreTTP.jinja2", os.path.join(package_root_dir,"MitreTTP.java"), {"package_name":package_name,"verbose_class":verbose_class}) + + #Write or update enums + writeJinja2Template(templateEnv, "MitreAttackDatasource.jinja2", os.path.join(package_root_dir,"MitreAttackDatasource.java"), {"all_data_sources":all_data_sources,"package_name":package_name}) + writeJinja2Template(templateEnv, "MitreAttackPlatform.jinja2", os.path.join(package_root_dir,"MitreAttackPlatform.java"), {"all_platforms":all_platforms,"package_name":package_name}) + writeJinja2Template(templateEnv, "MitreAttackDefensesBypassed.jinja2", os.path.join(package_root_dir,"MitreAttackDefensesBypassed.java"), {"all_defenses_bypassed":all_defenses_bypassed,"package_name":package_name}) + + for technique in technique_rows: + + class_package_name = f"{package_name}.{domain_bare}.technique" + class_package_postfix = "technique" + + if(technique.get("is_sub-technique",False)): + class_package_name = f"{package_name}.{domain_bare}.technique.{technique['parent_name_bare'].lower()}" + class_package_postfix = f"technique.{technique['parent_name_bare'].lower()}" + + + package_dir = os.path.join(domain_package_dir, class_package_postfix.replace(".", os.sep) ) + os.makedirs(package_dir, exist_ok=True) + + technique["domain"]= domain + technique["class_package_name"] = class_package_name + technique["package_name"] = package_name + technique["verbose_class"] = verbose_class + + #Use Jinja2 to load and render the template + + writeJinja2Template(templateEnv, "Technique.jinja2", os.path.join(package_dir,f"{technique['class_name']}.java"), technique) + + + + + + \ No newline at end of file diff --git a/mitreattack/attackToJava/templates/AbstractTactic.jinja2 b/mitreattack/attackToJava/templates/AbstractTactic.jinja2 new file mode 100644 index 00000000..5bccab8a --- /dev/null +++ b/mitreattack/attackToJava/templates/AbstractTactic.jinja2 @@ -0,0 +1,10 @@ +package {{ package_name }}; +/** +* This abstract class is the parent class for all the tactics in the {{ domain }} domain. +* It exists so that methods that accept any tactic can be passed an instance of this class inheriting from this class. +* +*/ +public abstract class AbstractTactic extends {{ root_package_name }}.MitreTTP +{ + +} \ No newline at end of file diff --git a/mitreattack/attackToJava/templates/GenericTactic.jinja2 b/mitreattack/attackToJava/templates/GenericTactic.jinja2 new file mode 100644 index 00000000..d768f18c --- /dev/null +++ b/mitreattack/attackToJava/templates/GenericTactic.jinja2 @@ -0,0 +1,20 @@ +package {{ package_name }}; +/** +* This Generic class represents the {{ class_name }} in attack matrix and can be used in case there is no technique that fits the description of the attack matrix. +* +{{description_lines |safe}} + +*/ +public class Generic{{ class_name }} extends {{ package_name }}.AbstractTactic implements {{ package_name }}.{{ class_name }} +{ + public Generic{{ class_name }}() { + super.name = "{{ name }}"; + super.domain = "{{ domain }}"; + super.id = "{{ ID }}"; + super.url = "{{ url }}"; + super.version = "{{ version }}"; + {% if verbose_class %} + super.description = "{{ description_field }}"; + {% endif %} + } +} \ No newline at end of file diff --git a/mitreattack/attackToJava/templates/MitreAttackDatasource.jinja2 b/mitreattack/attackToJava/templates/MitreAttackDatasource.jinja2 new file mode 100644 index 00000000..1090f705 --- /dev/null +++ b/mitreattack/attackToJava/templates/MitreAttackDatasource.jinja2 @@ -0,0 +1,47 @@ +package {{ package_name }}; + +/** Represents critical assert, source code, or data that attacker is trying to reach */ +public enum MitreAttackDatasource { + + + {% for key, value in all_data_sources.items() %} + {% if not loop.last %} + {{ key }}("{{ value }}"), + {% else %} + {{ key }}("{{ value }}"); + {% endif %} + {% endfor %} + + private final String value; + + MitreAttackDatasource(String value) { + this.value = value; + } + + /** + * Gets the string value of the report confidence. + * + * @return The string value of the report confidence. + */ + public String getValue() { + return value; + } + + /** + * Converts a string value to the corresponding MitreAttackDatasource enum value. If no matching value is + * found a RuntimeException is thrown + * + * @param value The string value to convert. + * @return The corresponding MitreAttackDatasource enum value. + * @throws RuntimeException if the value does not match any MitreAttackDatasource enum value. + */ + public static MitreAttackDatasource fromValue(String value) { + for (MitreAttackDatasource ac : MitreAttackDatasource.values()) { + if (ac.getValue().equals(value)) { + return ac; + } + } + + throw new RuntimeException("Invalid MitreAttackDatasource value: " + value); + } +} diff --git a/mitreattack/attackToJava/templates/MitreAttackDefensesBypassed.jinja2 b/mitreattack/attackToJava/templates/MitreAttackDefensesBypassed.jinja2 new file mode 100644 index 00000000..82de99a5 --- /dev/null +++ b/mitreattack/attackToJava/templates/MitreAttackDefensesBypassed.jinja2 @@ -0,0 +1,46 @@ +package {{ package_name }}; + +/** Represents critical assert, source code, or data that attacker is trying to reach */ +public enum MitreAttackDefensesBypassed { + + {% for key, value in all_defenses_bypassed.items() %} + {% if not loop.last %} + {{ key }}("{{ value }}"), + {% else %} + {{ key }}("{{ value }}"); + {% endif %} + {% endfor %} + + private final String value; + + MitreAttackDefensesBypassed(String value) { + this.value = value; + } + + /** + * Gets the string value of the report confidence. + * + * @return The string value of the report confidence. + */ + public String getValue() { + return value; + } + + /** + * Converts a string value to the corresponding MitreAttackDefensesBypassed enum value. If no matching value is + * found a RuntimeException is thrown + * + * @param value The string value to convert. + * @return The corresponding MitreAttackDefensesBypassed enum value. + * @throws RuntimeException if the value does not match any MitreAttackDefensesBypassed enum value. + */ + public static MitreAttackDefensesBypassed fromValue(String value) { + for (MitreAttackDefensesBypassed ac : MitreAttackDefensesBypassed.values()) { + if (ac.getValue().equals(value)) { + return ac; + } + } + + throw new RuntimeException("Invalid MitreAttackDefensesBypassed value: " + value); + } +} diff --git a/mitreattack/attackToJava/templates/MitreAttackPlatform.jinja2 b/mitreattack/attackToJava/templates/MitreAttackPlatform.jinja2 new file mode 100644 index 00000000..89932d5b --- /dev/null +++ b/mitreattack/attackToJava/templates/MitreAttackPlatform.jinja2 @@ -0,0 +1,47 @@ +package {{ package_name }}; + +/** Represents critical assert, source code, or data that attacker is trying to reach */ +public enum MitreAttackPlatform { + + + {% for key, value in all_platforms.items() %} + {% if not loop.last %} + {{ key }}("{{ value }}"), + {% else %} + {{ key }}("{{ value }}"); + {% endif %} + {% endfor %} + + private final String value; + + MitreAttackPlatform(String value) { + this.value = value; + } + + /** + * Gets the string value of the report confidence. + * + * @return The string value of the report confidence. + */ + public String getValue() { + return value; + } + + /** + * Converts a string value to the corresponding MitreAttackPlatform enum value. If no matching value is + * found a RuntimeException is thrown + * + * @param value The string value to convert. + * @return The corresponding MitreAttackPlatform enum value. + * @throws RuntimeException if the value does not match any MitreAttackPlatform enum value. + */ + public static MitreAttackPlatform fromValue(String value) { + for (MitreAttackPlatform ac : MitreAttackPlatform.values()) { + if (ac.getValue().equals(value)) { + return ac; + } + } + + throw new RuntimeException("Invalid MitreAttackPlatform value: " + value); + } +} diff --git a/mitreattack/attackToJava/templates/MitreTTP.jinja2 b/mitreattack/attackToJava/templates/MitreTTP.jinja2 new file mode 100644 index 00000000..c60234b1 --- /dev/null +++ b/mitreattack/attackToJava/templates/MitreTTP.jinja2 @@ -0,0 +1,94 @@ +package {{ package_name }}; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.Objects; + +public abstract class MitreTTP { + protected String name; + protected String domain; + protected String id; + protected String url; + protected String version; + + + + {% if verbose_class %} + protected final String detection; + + protected ArrayList dataSources = new ArrayList(); + protected ArrayList defencesBypassed = new ArrayList(); + protected ArrayList platforms = new ArrayList(); + + {% endif %} + + // Getters + public String getName() { + return name; + } + + public String getDomain() { + return domain; + } + + public String getId() { + return id; + } + + public String getUrl() { + return url; + } + + public String getVersion() { + return version; + } + + + /** + * Override equals() to use the id as the unique identifier + * This is sufficient as the classes are immiutable, that is no setters, no runtime changes + */ + @Override + public boolean equals(Object obj) { + if (this == obj) { + return true; + } + if (obj == null || getClass() != obj.getClass()) { + return false; + } + + MitreTTP that = (MitreTTP) obj; + return this.id.equals(that.id); + } + + /** + * Override hashCode() to use the id as the hashcode + * This is sufficient as the classes are immiutable, that is no setters, no runtime changes + */ + @Override + public int hashCode() { + return Objects.hash(id); + } + + + + {% if verbose_class %} + + public String getDescription() { + return description; + } + + public ArrayList getDataSources() { + return Collections.unmodifiableList(new ArrayList<>(dataSources)); + } + + public ArrayList getDefencesBypassed() { + return Collections.unmodifiableList(new ArrayList<>(defencesBypassed); + } + + public ArrayList getPlatforms() { + return Collections.unmodifiableList(new ArrayList<>(platforms); + } + + {% endif %} +} \ No newline at end of file diff --git a/mitreattack/attackToJava/templates/Tactic.jinja2 b/mitreattack/attackToJava/templates/Tactic.jinja2 new file mode 100644 index 00000000..4314579c --- /dev/null +++ b/mitreattack/attackToJava/templates/Tactic.jinja2 @@ -0,0 +1,16 @@ +package {{ package_name }}; +/** +* This interface represents the {{ class_name }} tactic in the MITRE ATT&CK framework. +* For each of the tactics, there are techniques that are associated with it which implement an interface for each of the tactics they are associated with. +* Each tactic also has an Abstract class that represents the tactic itself. +*/ +public interface {{ class_name }} +{ + + String getName(); + String getDomain(); + String getId(); + String getUrl(); + String getVersion(); + +} \ No newline at end of file diff --git a/mitreattack/attackToJava/templates/Technique.jinja2 b/mitreattack/attackToJava/templates/Technique.jinja2 new file mode 100644 index 00000000..594f2b2d --- /dev/null +++ b/mitreattack/attackToJava/templates/Technique.jinja2 @@ -0,0 +1,45 @@ +package {{ class_package_name }}; + +import java.util.ArrayList; + +import {{ package_name }}.*; +import {{ package_name }}.tactic.*; + +/** +* This class represents the {{ class_name }} Tehnique or sub technique in attack matrix +* +{{description_lines |safe}} +*/ +public class {{ class_name }} extends {{ extends }} {% if implements %} implements {{ implements }}{% endif %} +{ + + public {{ class_name }}() { + super.name = "{{ name }}"; + super.domain = "{{ domain }}"; + super.id = "{{ ID }}"; + super.url = "{{ url }}"; + super.version = "{{ version }}"; + + {% if verbose_class %} + super.description = "{{ description_field }}"; + {% if detection %} + detection = "{{ detection_field }}"; + {% endif %} + + + {% for item in data_source_keys %} + super.dataSources.add(MitreAttackDatasource.{{ item }}); + {% endfor %} + + {% for item in defense_bypassed_keys %} + super.defencesBypassed.add(MitreAttackDefensesBypassed.{{ item }}); + {% endfor %} + + {% for item in platform_keys %} + super.platforms.add(MitreAttackPlatform.{{ item }}); + {% endfor %} + + {% endif %} + } + +} \ No newline at end of file diff --git a/mitreattack/attackToJava/templates/pom.jinja2 b/mitreattack/attackToJava/templates/pom.jinja2 new file mode 100644 index 00000000..9adc96e7 --- /dev/null +++ b/mitreattack/attackToJava/templates/pom.jinja2 @@ -0,0 +1,46 @@ + + + 4.0.0 + {{ organization }} + {{package_bare}} + 1.0-SNAPSHOT + jar + + + 1.8 + 1.8 + + + + + + + + central + https://repo.maven.apache.org/maven2 + + + plugins.gradle.org + https://plugins.gradle.org/m2/ + + + + + + + com.spotify.fmt + fmt-maven-plugin + 2.23 + + + + format + + + + + + + \ No newline at end of file diff --git a/setup.py b/setup.py index 61e4252a..d41d6f5e 100644 --- a/setup.py +++ b/setup.py @@ -20,6 +20,7 @@ entry_points={ "console_scripts": [ "attackToExcel_cli=mitreattack.attackToExcel.attackToExcel:main", + "attackToJava_cli=mitreattack.attackToJava.attackToJava:main", "layerExporter_cli=mitreattack.navlayers.layerExporter_cli:main", "layerGenerator_cli=mitreattack.navlayers.layerGenerator_cli:main", "indexToMarkdown_cli=mitreattack.collections.index_to_markdown:main", diff --git a/tests/test_to_java.py b/tests/test_to_java.py new file mode 100644 index 00000000..b1881ee2 --- /dev/null +++ b/tests/test_to_java.py @@ -0,0 +1,29 @@ +from pathlib import Path + +from loguru import logger + +from mitreattack.attackToJava import attackToJava + +# tmp_path is a built-in pytest tixture +# https://docs.pytest.org/en/7.1.x/how-to/tmp_path.html + +def check_java_files_exist(output_dir: Path): + """Check that all expected excel files exist""" + + assert (output_dir / "src").exists() + assert (output_dir / "target").exists() + assert (output_dir / "src" / "main" / "java" / "org" / "mitre" / "attack" / "MitreTTP.java").exists() + assert (output_dir / "target" / "attack-1.0-SNAPSHOT.jar").exists() + +def test_latest(tmp_path: Path, stix_file_enterprise_latest: str): + """Test most recent enterprise to excel spreadsheet functionality""" + logger.debug(f"{tmp_path=}") + + #We need all domain stix files, so instead of enterprise file, get the path to the directory + stix_file_path = Path(stix_file_enterprise_latest).parent + + attackToJava.export(output_dir=str(tmp_path), stix_path=stix_file_path, package_name="org.mitre.attack", verbose_class=True) + + check_java_files_exist(output_dir=tmp_path) + +