This software is no longer maintained. We advise replacing your production use of this software with the swap-in replacement StayRTR
GoRTR is an open-source implementation of RPKI to Router protocol (RFC 6810) using the the Go Programming Language.
/libcontains a library to create your own server and client./prefixfilecontains the structure of a JSON export file and signing capabilities./cmd/gortr/gortr.gois a simple implementation that fetches a list and offers it to a router./cmd/rtrdump/rtrdump.goallows copying the PDUs sent by a RTR server as a JSON file./cmd/rtrmon/rtrmon.gocompare and monitor two RTR servers (using RTR and/or JSON), outputs diff and Prometheus metrics.
This software comes with no warranty.
Cloudflare operates 200+ GoRTR globally. They provide redundancy in at the PoP level. This provides increased reliability by computing a unique prefix list and providing a secure distribution of the file over its CDN before being sent to the routers.
GoRTR also powers the public RTR server available on rtr.rpki.cloudflare.com on port 8282 and 8283 for SSH (rpki/rpki)
Telia has deployed RPKI and uses GoRTR connected with OctoRPKI and rpki-client to distribute the ROAs to its routers. Instances of the RTR servers handle around 250 sessions each.
NTT has deployed OpenBSD's rpki-client together with GoRTR to facilitate rejecting RPKI Invalid BGP route announcements towards it's Global IP Network (AS 2914). More information is available here.
GTT deployed GoRTR along with OctoRPKI. The setup currently provides 400+ RTR sessions to their routers for filtering RPKI invalids.
Cogent deployed GoRTR and OctoRPKI at the end of May 2020. 8 validators feed approximately 2500 routers.
Router vendors also used this software to develop their implementations.
Do you use this tool at scale? Let us know!
- Refreshes a JSON list of prefixes (from either Cloudflare or a RIPE Validator)
- Prometheus metrics
- Lightweight
- TLS
- SSH
- Signature verification and expiration control
- Generate a list of prefixes sent via RTR (similar to Cloudflare JSON input, or RIPE RPKI Validator)
- Lightweight
- TLS
- SSH
You need a working Go environment (1.10 or newer). This project also uses Go Modules.
$ git clone git@github.com:cloudflare/gortr.git && cd gortr
$ go build cmd/gortr/gortr.goIf you do not want to use Docker, please go to the next section.
If you have Docker, you can start GoRTR with docker run -ti -p 8082:8082 cloudflare/gortr.
The containers contain Cloudflare's public signing key and an testing ECDSA private
key for the SSH server.
It will automatically download Cloudflare's prefix list and use the public key to validate it.
You can now use any CLI attributes as long as they are after the image name:
$ docker run -ti -p 8083:8083 cloudflare/gortr -bind :8083If you want to build your own image of GoRTR:
$ docker build -t mygortr -f Dockerfile.gortr.prod .
$ docker run -ti mygortr -hIt will download the code from GitHub and compile it with Go and also generate an ECDSA key for SSH.
Please note: if you plan to use SSH with Cloudflare's default container (cloudflare/gortr),
replace the key private.pem since it is a testing key that has been published.
An example is given below:
$ docker run -ti -v $PWD/mynewkey.pem:/private.pem cloudflare/gortr -ssh.bind :8083There are a few solutions to install it.
Go can directly fetch it from the source
$ go get github.com/cloudflare/gortr/cmd/gortrCopy cf.pub to your local directory if you want to use Cloudflare's signed JSON file.
You can use the Makefile (by default it will be compiled for Linux, add GOOS=darwin for Mac)
$ make dist-key build-gortrThe compiled file will be in /dist.
Or you can use a package (or binary) file from the Releases page:
$ sudo dpkg -i gortr[...].deb
$ sudo systemctl start gortrIf you want to sign your list of prefixes, generate an ECDSA key. Then generate the public key to be used in GoRTR. You will have to setup your validator to use this key or have another tool to sign the JSON file before passing it to GoRTR.
$ openssl ecparam -genkey -name prime256v1 -noout -outform pem > private.pem
$ openssl ec -in private.pem -pubout -outform pem > public.pemOnce you have a binary:
$ ./gortr -tls.bind 127.0.0.1:8282Make sure cf.pub is in the current directory. Or pass -verify.key=path/to/cf.pub
If you want to package it (deb/rpm), you can use the pre-built docker-compose file.
$ docker-compose -f docker-compose-pkg.yml upYou can find both files in the dist/ directory.
This was tested with a basic Squid proxy. The User-Agent header is passed
in the CONNECT.
You have to export the following two variables in order for GoRTR to use the proxy.
export HTTP_PROXY=schema://host:port
export HTTPS_PROXY=schema://host:port
You can run GoRTR and listen for TLS connections only (just pass -bind "").
First, you will have to create a SSL certificate.
$ openssl ecparam -genkey -name prime256v1 -noout -outform pem > private.pem
$ openssl req -new -x509 -key private.pem -out server.pemThen, you have to run
$ ./gortr -ssh.bind :8282 -tls.key private.pem -tls.cert server.pemYou can run GoRTR and listen for SSH connections only (just pass -bind "").
You will have to create an ECDSA key. You can use the following command:
$ openssl ecparam -genkey -name prime256v1 -noout -outform pem > private.pemThen you can start:
$ ./gortr -ssh.bind :8282 -ssh.key private.pem -bind ""By default, there is no authentication.
You can use password and key authentication:
For example, to configure user rpki and password rpki:
$ ./gortr -ssh.bind :8282 -ssh.key private.pem -ssh.method.password=true -ssh.auth.user rpki -ssh.auth.password rpki -bind ""And to configure a bypass for every SSH key:
$ ./gortr -ssh.bind :8282 -ssh.key private.pem -ssh.method.key=true -ssh.auth.key.bypass=true -bind ""GoRTR supports SLURM configuration files (RFC8416).
Create a json file (slurm.json):
{
"slurmVersion": 1,
"validationOutputFilters": {
"prefixFilters": [
{
"prefix": "10.0.0.0/8",
"comment": "Everything inside will be removed"
},
{
"asn": 65001,
},
{
"asn": 65002,
"prefix": "192.168.0.0/24",
},
],
"bgpsecFilters": []
},
"locallyAddedAssertions": {
"prefixAssertions": [
{
"asn": 65001,
"prefix": "2001:db8::/32",
"maxPrefixLength": 48,
"comment": "Manual add"
}
],
"bgpsecAssertions": [
]
}
}
When starting GoRTR, add the -slurm ./slurm.json argument.
The log should display something similar to the following:
INFO[0001] Slurm filtering: 112214 kept, 159 removed, 1 asserted
INFO[0002] New update (112215 uniques, 112215 total prefixes).
For instance, if the original JSON fetched contains the ROA: 10.0.0.0/24-24 AS65001,
it will be removed.
The JSON exported by GoRTR will contain the overrides and the file can be signed again. Others GoRTR can be configured to fetch the ROAs from the filtering GoRTR: the operator manages one SLURM file on a leader GoRTR.
You can check the content provided over RTR with rtrdump tool
$ ./rtrdump -connect 127.0.0.1:8282 -file debug.jsonYou can also fetch the re-generated JSON from the -export.path endpoint (default: http://localhost:8080/rpki.json)
Use your own validator, as long as the JSON source follows the following schema:
{
"roas": [
{
"prefix": "10.0.0.0/24",
"maxLength": 24,
"asn": "AS65001"
},
...
]
}
- Cloudflare (list curated, signed, compressed and cached in +160 PoPs)
- Third-party JSON formatted VRP exports:
To use a data source that do not contain signatures or validity information, pass:
-verify=false -checktime=false
Note: for boolean flags, it requires the equal sign
Cloudflare's prefix list removes duplicates and entries that are not routed on the Internet (>/24 IPv4 and >/48 IPv6).
By default, the session ID will be randomly generated. The serial will start at zero.
You can define a serial to start with the following way:
- the JSON must contain a
serialfield inmetadata; and - the flag
-useserialmust be set to 1 or 2
When flag is set to 1, every change of file will increment the serial regardless of the current serial field.
Make sure the refresh rate of GoRTR is more frequent than the refresh rate of the JSON.
When flag is set to 2, GoRTR will set the value of the serial in the JSON. If an ID is missed or not updated, it will cause discrepancies on the client.
A simple comparison between software and devices. Implementations on versions may vary.
| Device/software | Plaintext | TLS | SSH | Notes |
|---|---|---|---|---|
| RTRdump | Yes | Yes | Yes | |
| RTRlib | Yes | No | Yes | Only SSH key |
| Juniper | Yes | No | No | |
| Cisco | Yes | No | Yes | Only SSH password |
| Nokia SR OS | Yes | No | No | Since 9.0.R1(2010) |
| Arista | Yes | No | No | |
| FRRouting | Yes | No | Yes | Only SSH key |
| Bird2 | Yes | No | Yes | Only SSH key |
| Quagga | Yes | No | No |
Configure a session to the RTR server (assuming it runs on 192.168.1.100:8282)
louis@router> show configuration routing-options validation
group TEST-RPKI {
session 192.168.1.100 {
port 8282;
}
}
Add policies to validate or invalidate prefixes
louis@router> show configuration policy-options policy-statement STATEMENT-EXAMPLE
term RPKI-TEST-VAL {
from {
protocol bgp;
validation-database valid;
}
then {
validation-state valid;
next term;
}
}
term RPKI-TEST-INV {
from {
protocol bgp;
validation-database invalid;
}
then {
validation-state invalid;
reject;
}
}
Display status of the session to the RTR server.
louis@router> show validation session 192.168.1.100 detail
Session 192.168.1.100, State: up, Session index: 1
Group: TEST-RPKI, Preference: 100
Port: 8282
Refresh time: 300s
Hold time: 600s
Record Life time: 3600s
Serial (Full Update): 1
Serial (Incremental Update): 1
Session flaps: 2
Session uptime: 00:25:07
Last PDU received: 00:04:50
IPv4 prefix count: 46478
IPv6 prefix count: 8216
Show content of the database (list the PDUs)
louis@router> show validation database brief
RV database for instance master
Prefix Origin-AS Session State Mismatch
1.0.0.0/24-24 13335 192.168.1.100 valid
1.1.1.0/24-24 13335 192.168.1.100 valid
You may want to use the option to do SSH-based connection.
On Cisco, you can have only one RTR server per IP.
To configure a session for 192.168.1.100:8282:
Replace 65001 by the configured ASN:
router bgp 65001
rpki server 192.168.1.100
transport tcp port 8282
!
!
For an SSH session, you will also have to configure
router bgp 65001 rpki server 192.168.1.100 password xxx
where xxx is the password.
Some experimentations showed you have to configure
the username/password first, otherwise it will not accept the port.
router bgp 65001
rpki server 192.168.1.100
username rpki
transport ssh port 8282
!
!
ssh client tcp-window-scale 14
ssh timeout 120
The last two SSH statements solved an issue causing the connection to break before receiving all the PDUs (TCP window full problem).
To visualize the state of the session:
RP/0/RP0/CPU0:ios#sh bgp rpki server 192.168.1.100
RPKI Cache-Server 192.168.1.100
Transport: SSH port 8282
Connect state: ESTAB
Conn attempts: 1
Total byte RX: 1726892
Total byte TX: 452
Last reset
Timest: Apr 05 01:19:32 (04:26:58 ago)
Reason: protocol error
SSH information
Username: rpki
Password: *****
SSH PID: 18576
RPKI-RTR protocol information
Serial number: 15
Cache nonce: 0x0
Protocol state: DATA_END
Refresh time: 600 seconds
Response time: 30 seconds
Purge time: 60 seconds
Protocol exchange
ROAs announced: 67358 IPv4 11754 IPv6
ROAs withdrawn: 80 IPv4 34 IPv6
Error Reports : 0 sent 0 rcvd
Last protocol error
Reason: response timeout
Detail: response timeout while in DATA_START state
To visualize the accepted PDUs:
RP/0/RP0/CPU0:ios#sh bgp rpki table
Network Maxlen Origin-AS Server
1.0.0.0/24 24 13335 192.168.1.100
1.1.1.0/24 24 13335 192.168.1.100
router bgp <asn>
rpki cache <name>
host <ipv4|ipv6|hostname> [vrf <vrfname>] [port <1-65535>] # default port is 323
local-interface <interface>
preference <1-10> # the lower the value, the more preferred
# default is 5
refresh-interval <1-86400 seconds> # default is 3600
expire-interval <600-172800 seconds> # default is 7200
retry-interval <1-7200 seconds> # default is 600
If multiple caches are configured, the preference controls the priority.
Caches which are more preferred will be connected to first, if they are not reachable then connections will be attempted to less preferred caches.
If caches have the same preference value, they will all be connected to and the ROAs that are synced from them will be merged together.
To visualize the state of the session:
show bgp rpki cache [<name>]
show bgp rpki cache counters [errors]
show bgp rpki roa summary
To visualize the accepted PDUs:
show bgp rpki roa (ipv4|ipv6) [prefix]
/configure router "management" # or "Base" for in-band
origin-validation {
rpki-session 172.65.0.2 {
admin-state enable
description "rtr.rpki.cloudflare.com"
port 8282
}
}
See this NANOG 67 presentation for context
To check the state:
A:admin@IXPRouter# show router "management" origin-validation rpki-session detail
===============================================================================
RPKI Session Information
===============================================================================
IP Address : 172.65.0.2
Description : rtr.rpki.cloudflare.com
-------------------------------------------------------------------------------
Port : 8282 Oper State : established
Uptime : 0d 00:29:07 Flaps : 0
Active IPv4 Records: 327768 Active IPv6 Records: 67796
Admin State : Up Local Address : n/a
Hold Time : 600 Refresh Time : 300
Stale Route Time : 3600 Connect Retry : 120
Serial ID : 1 Session ID : 9944
===============================================================================
No. of Sessions : 1
===============================================================================
Licensed under the BSD 3 License.