Skip to content

Latest commit

 

History

History
142 lines (94 loc) · 8.91 KB

tls-performance-testing.md

File metadata and controls

142 lines (94 loc) · 8.91 KB
Raw

Automated PQC TLS Performance Benchmarking Tool Usage Guide

Contents

Overview

This tool allows for the automatic testing of PQC TLS 1.3 handshake performance and algorithmic efficiency when integrated within the OpenSSL library through the OQS-Provider. It tests empty TLS handshakes using varying combinations of PQC and Hybrid-PQC algorithms for authentication and KEMs. Furthermore, the testing tools conduct speed tests which evaluate PQC algorithmic performance when integrated within the OpenSSL library using the Liboqs library. The automated tools also perform benchmarking for classic cryptographic algorithms to provide a comparative baseline.

The tests can be conducted on either a single machine or across two machines connected via a physical or virtual network.

Supported Hardware

The automated testing tool is currently only supported on the following devices:

  • x86 Linux Machines using a Debian based operating system
  • ARM Linux devices using a 64-bit Debian based Operating System

Getting Started

To begin testing the performance of PQC algorithms when integrated within TLS, there are various steps that must be completed and they differ depending on whether a single machine or two machines are being used. Please fully review this section before conducting the tests to ensure all configurations are correct.

The scripts required for conducting the automated testing are stored in the scripts/test-scripts directory which can be found within the project's root directory.

Generating Required Certificates and Private Keys

Before beginning testing, it is necessary to first generate the required server certificate and private key files needed for the TLS performance testing tools. This can be done by executing the following command from within the scripts/testing-scripts directory:

./oqsssl-generate-keys.sh

This will generate the certificates and private keys and output them to the test-data/keys directory in the project's root directory. If testing performance over a physical/virtual network using two machines, the keys directory will need to be transferred to the second testing machine as well.

Testing Tool Execution

To start the automated testing tool, open a terminal in the directory containing the Full PQC TLS Test tool and run the following command:

./full-pqc-tls-test.sh

Please refer to the Testing Options section before beginning testing to ensure all configurations are correct

Upon executing the script, the testing tool will prompt you to enter the parameters for the test. Depending on the testing scenario, either using a single machine or separate server and client machines, different setup techniques and options will be required.

Testing Options

Before discussing the execution of the testing script, a list of the testing parameters used are detailed below, to provide guidance on the information needed before performing the tests:

The testing tool will prompt you to enter the parameters for the test. These parameters include:

  • Machine type (server or client)
  • Machine Comparison Option
  • Machine Results Number (if comparison option selected)
  • Number of test runs
  • Duration of each TLS handshake tests (if machine is client)
  • Duration of TLS speed tests (if machine is client)
  • IP address of the other machine (single machine - localhost / separate machines server IP and Client IP)

Single Machine Testing

If using a single machine for the testing, once the certificates and private keys have been generated, perform the following steps:

Server Setup:

  1. First, execute the full-pqc-tls-test.sh script and select the server machine type when prompted.

  2. Follow the prompts to enter the test parameters. When asked for the other machine's IP, use the localhost address (127.0.0.1).

  3. Ensure the server setup is complete and ready before proceeding to the client setup.

Client Setup:

  1. Ensure the server script is active and listening, this can be seen in the first terminals output.

  2. Open a separate terminal and execute the full-pqc-tls-test.sh script again.

  3. Select the client machine type when prompted.

  4. Follow the prompts to enter the test parameters. When asked for the other machine's IP, use the localhost address (127.0.0.1).

Separate Server and Client Machine Testing

When utilising two separate machines for testing, one machine will be set up as the server and the other as the client. Before initiating testing, ensure that both machines are properly set up and have the required certificates and keys as discussed previously. The server machine should be fully set up and ready before activating the client machine.

Server Machine Setup:

  1. Initiate the test server by executing the full-pqc-tls-test.sh script.

  2. When prompted, select the server machine type.

  3. Follow the prompts to enter the test parameters.

  4. When asked for the other machine's IP, provide the IP address of the client machine.

Client Setup:

  1. Ensure the server script is active and listening, this can be seen in the first terminals output.

  2. Initiate the test client by executing the full-pqc-tls-test.sh script.

  3. Follow the prompts to enter the test parameters.

  4. When asked for the other machine's IP, provide the IP address of the server machine.

Outputted Results

The results from the Full PQC TLS Test will be stored in the test-data/up-results/ops-openssl/machine-x directory, which can be found within the code's root directory. The results include handshake and speed test results for both PQC and classic ciphersuites. This directory is used to store all of the unparsed results from the automated testing tools.

Please note that when using more than one machine for testing, the results will only be stored on the client machine, not the server machine.

However, the data stored is not yet ready for interpretation or graph generation. To parse the data into a format that can be used for further analysis please refer to the parsing results section within the readme file.

Included Testing Scripts

The Full PQC TLS Test tool uses several scripts to perform the TLS handshake tests. These include:

  • oqsssl-test-server.sh
  • oqsssl-test-client.sh
  • oqsssl-test-speed.sh
  • oqsssl-generate-keys.sh

oqsssl-test-server.sh:

This script sets up and runs the server-side operations for the TLS handshake tests. It performs handshake tests for various combinations of PQC signature algorithms and KEM algorithms, as well as classical handshake tests. The script includes error handling to deal with test failures and will coordinate a reattempt with the client. The script also skips tests when both the signature and KEM algorithms are classical.

oqsssl-test-client.sh:

This script is responsible for client-side operations for the TLS handshake tests. It performs the client-side handshake operations and checks the test status. If the test fails, it sends a signal to the server to restart the test. The script also skips tests when both the signature and KEM algorithms are classical.

oqsssl-test-speed.sh:

This script performs speed tests for the PQC algorithmic operations when integrated into the OpenSSL library. It performs these tests for both PQC and classic and stores the results in a specific directory.

oqsssl-generate-keys.sh:

This script generates the necessary PQC and classical certificates and private keys needed for the TLS handshake tests. It creates a new key for each PQC signature algorithm and each ECC curve.

These scripts together enable the execution of the Full PQC TLS Test, with each one handling a specific aspect of the process. They are designed to work together seamlessly, with different scripts signalling each other as needed to coordinate the process.

Useful Documentation