Skip to content

Use grepaddr to extract (grep) all kinds of addresses from stdin like URLs (incl. IPv4/IPv6), IP addresses & ranges (IPv4/IPv6), e-mail addresses, MAC addresses.

License

Notifications You must be signed in to change notification settings

Zarcolio/grepaddr

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

About GrepAddr

GrepAddr takes input from stdin and extracts different kinds of addresses from stdin like URLs, IP addresses, e-mail addresses, MAC addresses and more. Essentially it's grep, but for interesting addresses.

Because the script uses regexp, it's very common to get false positives for FQDNs. The options --resolve (FQDNs only) --iana and --private can be used to reduce the number of false positives.

Why use GrepAddr?

Several tools and scripts exist on the internet which similar functionality but usually only for a single address type, for example xurlsx, xpath-url-extraction and relative-url-extractor. Little/none of them can grep as many addresses type like GrepAddr can. You can use GrepAddr when doing a pen test or CTF, or when persuing a bug bounty.

Install

GrepAddr should be able to grep addreases with a default Kali Linux installation without installing additional Python packages. Just run:

git clone https://github.com/Zarcolio/grepaddr
cd grepaddr
sudo bash install.sh

When using the installer in an automated environment, use the following command for an automated installation:

sudo bash install.sh -auto

If you're running into trouble running GrepAddr, please drop me an issue and I'll try to fix it :)

Usage

usage: grepaddr [-h] [-fqdn] [-srv] [-email] [--port] [--iana] [--private] [--resolve] [-ipv4] [-cidr4]
[-ipv6] [-cidr6] [-mac] [-url] [-relurl] [--baseurl <url>] [--basetag] [-csv <file>] [-decode <rounds>] 
[-unescape <rounds>]

Use GrepAddr to extract different kinds of addresses from stdin. If no arguments are given, addresses
of all types are shown.

optional arguments:
  -h, --help         show this help message and exit
  -fqdn              Extract fully qualified domain names.
  -srv               Extract DNS SRV records.
  -email             Extract e-mail addresses.
  --port             Include :port for extraction.
  --iana             Extract FQDNs with IANA registered TLDs , use with -fqdn, -srv or -email . No
                     impact on other options.
  --private          Extract FQDNs with TLDs for private use, use with -fqdn. No impact on other options.
  --resolve          Display only those FQDNs that can be resolved. Cannot be used together with
                     --iana or --private. No impact on other options.
  -ipv4              Extract IP version 4 addresses.
  -cidr4             Extract IP version 4 addresses in CIDR notation.
  -ipv6              Extract IP version 6 addresses.
  -cidr6             Extract IP version 6 addresses in CIDR notation.
  -mac               Extract MAC addresses.
  -url               Extract URLs (FQDN, IPv4, IPv6, mailto and generic detection of schemes).
  -relurl            Extract relative URLs.
  --basetag          Search for base URL in <BASE> and prepend it to URLS. Use with -url and/or -relurl.
  --baseurl <url>    Provide a base URL which is prepended to relative URLS starting at root. Use
                     with -url and/or -relurl.
  -csv <file>        Save addresses found to this CSV file.
  -decode <rounds>   URL decode input this many times before extracting FQDNs.
  -unslash <rounds>  Unescape slashes within input this many times before extracting FQDNs.

Examples

It's really easy to grep all supported addresses from stdin, just run:

wget -qO - https://twitter.com|grepaddr -csv twitter.csv

Want to extract addresses of certain type? Choose one of the options, for example -mac and run:

wget -qO - https://nl.wikipedia.org/wiki/MAC-adres|grepaddr -mac

Want to grep FQDNs with a private TLD, just run:

wget -qO - https://serverfault.com/questions/17255/top-level-domain-domain-suffix-for-private-network|grepaddr -fqdn --private

Want to extract FQDNs and show only resolved FQDNs URLs needed to be decoded first, just run:

wget -qO - https://twitter.com|grepaddr -fqdn --resolve --decode 1

Want to grep all addresses with the least chance of false positives without having to wait for resolving FQDNs:

wget -qO - https://twitter.com|grepaddr --iana

Want to extract all addresses and convert relative URLs starting at the root to an absolute URL:

wget -qO - https://twitter.com|grepaddr --base https://twitter.com

Want to extract addresses from a binary, use it together with strings:

string binary.ext|grepaddr

When looking for URLs from the current user hive within Windows' registry, run these commands:

Within Windows:

reg export HKCU hkcu.reg

Within your favorite Unix-like OS:

dos2unix -f hkcu.reg
strings hkcu.reg|grepaddr -url

Contribute?

Do you have some usefull additions to GrepAddr:

  • PR's Welcome
  • Twitter

About

Use grepaddr to extract (grep) all kinds of addresses from stdin like URLs (incl. IPv4/IPv6), IP addresses & ranges (IPv4/IPv6), e-mail addresses, MAC addresses.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published