2021 06 01 load testing analysis

Load testing results analysis

CIIP v2.6.0

Testing Date: June 1st, 2021

Testing Methodology

The load testing was conducted with scripts using the k6 tool, which are located in the app/tests/perf folder.

Due to the absence of server-side rendering in this version of the application, the most realistic way to conduct load testing was to query the /graphql endpoint, with each pages' query. Due to the usage of Relay and GraphQL fragments, all of the data that is used to render a page can be fetched in a single HTTP request.

To keep the testing scenario simple, we decided to only test two mutations: the "application creation" and "edition of application form", with our testing data being provided with 1000 facilities for which applications could be created.

The aim of the test was to ensure that our application, with its production deployment settings, could withstand a heavy traffic and respond to queries within an acceptable time limit.

Test Monitoring and Results Analysis

While running the test, team members were monitoring the progress of the testing scenarios via the OpenShift Console, and the behaviour of the application by using Sysdig dashboards, to ensure proper load balancing and resiliency of the application.

In terms of performance, only one operation appeared to have an average time greater than 1s, the CreateApplicationMutation. Analysis of that query's performance and optimization work will be performed in #1919.

Further testing is necessary to identify performance bottlenecks, and optimal settings for CPU limits and number of replicas. Given that the application server CPU usage was maxed out during the load testing, a Horizontal Pod Autoscaler should be added to the application server deployment, but will require multiple tests to determine the metrics thresholds that need to be used to provide optimal performance (i.e. the smallest difference between database query execution time and HTTP response time) (#1920).