Skip to content
@threathawkproject

ThreatHawk

An Investigative Threat Intelligence Platform

Banner

Welcome!

ThreatHawk is an investigative threat intelligence platform that empowers cyberthreat analysts by placing at the power of open-source tools at their fingertips. It allows them to collect meaningful intelligence data about IP addresses, domains, emails, and file hashes through a single click.

Tools and Technologies

  • Python (Django and FastAPI)
  • PostgreSQL
  • Kafka
  • React (Next.js and MaterialUI)
  • Docker and Docker Compose
  • Meilisearch

Main Features

  • Aggregation of different kinds of Indicators of Compromise (IOCs) from open-source threat intelligence feeds such as AbuseIPDB and Darklist, with support for paid feeds as well. An IOC is any IP address, domain, email, or file hash that has been deemed malicious.
  • Enrichment of the collected IOCs using open-source threat intelligence tools such as EmailRep (for email reputation scores) and MaxMind GeoIP (for IP address geolocation data). This enrichment module is incredibly scalable and new tools can easily be integrated in the future.
  • Iterative, drill-down investigations on a specified IOC. This allows us to dig deeper and actually understand the threat landscape (by drawing links between the IOC, the attacker, and the techniques that were used by the attacker).
  • All data is converted to the industry-recognized STIX standard. This means that any external systems can access ThreatHawk's data too.
  • ThreatHawk is incredibly fault-tolerant and scalable since each feature is a separate microservice. This means that future contributions to this project can also be made very easily.

Significance

  • Cyber threats have become increasingly globalized, which is why it is essential to move towards open-source threat intelligence.
  • Traditionally, analyzing a single IOC with multiple tools is an incredibly painful process, since analysts must manually configure each tool separately. This is very difficult because different tools work differently (some use REST APIs, whereas others use client libraries or SDKs). ThreatHawk automates the usage of these different tools through one click.
  • Investigating an IOC iteratively (beyond the first level) is a relatively novel approach since most competing solutions only provide data up till the first layer. With ThreatHawk, analysts can drill-down to deeper layers until they find satisfactory insights.

Deployment

  • Ensure the following packages are installed and setup correctly

    • python3-pip
    • docker.io
    • docker-compose
    • curl
  • Run the following command to download a script to clone all required repositories

    • curl -L https://raw.githubusercontent.com/4W4I5/scripts/main/threathawk/deployment.sh -o deployment.sh
  • Mark it as executable

    • chmod +x deployment.sh
  • Run the script (use --build to force rebuild of containers if needed)

    • ./deployment.sh
  • Follow instructions listed below after all containers have been successfully deployed

Step 1: Install the required dependencies

cd threathawk/ioc-aggregator
pip install -r requirements.txt 

Step 2: Migrations

python3 manage.py makemigrations
python3 manage.py migrate

Step 3: Run the server

python3 manage.py runserver 8004

Step 4: Run the consumer, in a new terminal

python3 manage.py ioc_feeds_consumer

Step 5: Run the producer, in a new terminal

python3 manage.py ioc_feeds_job

Contributors

Popular repositories Loading

  1. ioc_aggregator ioc_aggregator Public

    An aggregator for scheduled indicator collection

    Python 1

  2. frontend frontend Public

    Dashboard

    JavaScript 1

  3. db_loader db_loader Public

    A database loader for indicator storage

    Python

  4. enrichment enrichment Public

    Indicator enrichment through open-source tools

    Python 1

  5. .github .github Public

    1

  6. encoding encoding Public

    To convert our enriched data into proper STIX objects (SDOs, SCOs, and SROs)

    Python 1

Repositories

Showing 7 of 7 repositories
  • frontend Public

    Dashboard

    threathawkproject/frontend’s past year of commit activity
    JavaScript 0 1 0 0 Updated Oct 25, 2024
  • investigation Public

    Perform drill-down investigations to fully understand the threat landscape

    threathawkproject/investigation’s past year of commit activity
    Python 0 1 0 0 Updated Oct 25, 2024
  • enrichment Public

    Indicator enrichment through open-source tools

    threathawkproject/enrichment’s past year of commit activity
    Python 0 1 0 0 Updated Oct 25, 2024
  • .github Public
    threathawkproject/.github’s past year of commit activity
    0 1 0 0 Updated Oct 25, 2024
  • ioc_aggregator Public

    An aggregator for scheduled indicator collection

    threathawkproject/ioc_aggregator’s past year of commit activity
    Python 0 1 0 0 Updated Jun 20, 2024
  • encoding Public

    To convert our enriched data into proper STIX objects (SDOs, SCOs, and SROs)

    threathawkproject/encoding’s past year of commit activity
    Python 0 1 0 0 Updated Jun 20, 2024
  • db_loader Public

    A database loader for indicator storage

    threathawkproject/db_loader’s past year of commit activity
    Python 0 0 0 0 Updated Mar 31, 2023

Top languages

Loading…

Most used topics

Loading…