From d66a3754a792e168eca662421a8bd0d4496dab82 Mon Sep 17 00:00:00 2001 From: Jessica Mitchell Date: Thu, 7 Mar 2024 11:35:13 +0100 Subject: [PATCH 1/3] add cmake option for benchmarks, add seealso in benchmarking --- doc/htmldoc/hpc/benchmarking.rst | 14 +++++++------- doc/htmldoc/installation/cmake_options.rst | 22 ++++++++++++++++++++++ 2 files changed, 29 insertions(+), 7 deletions(-) diff --git a/doc/htmldoc/hpc/benchmarking.rst b/doc/htmldoc/hpc/benchmarking.rst index cd0f17f5fe..a6420e981c 100644 --- a/doc/htmldoc/hpc/benchmarking.rst +++ b/doc/htmldoc/hpc/benchmarking.rst @@ -3,22 +3,25 @@ Benchmarking NEST ================= +.. seealso:: + + When compiling NEST to perform benchmarks, see :ref:`our cmake options ` for improved performance and energy saving. beNNch ~~~~~~ Computational efficiency is essential to simulate complex neuronal networks and study long-term effects such as learning. The scaling performance of neuronal network simulators on high-performance computing systems can be assessed with benchmark simulations. -However, maintaining comparability of benchmark results across different systems, software environments, network models, and researchers from potentially different labs poses a challenge. +However, maintaining comparability of benchmark results across different systems, software environments, network models, and researchers from potentially different labs poses a challenge. -The software framework `beNNch `_ tackles this challenge by implementing a unified, modular workflow for configuring, executing, and analyzing such benchmarks. +The software framework `beNNch `_ tackles this challenge by implementing a unified, modular workflow for configuring, executing, and analyzing such benchmarks. beNNch builds around the `JUBE Benchmarking Environment `_, installs simulation software, provides an interface to benchmark models, automates data and metadata annotation, and accounts for storage and presentation of results. For more details on the conceptual ideas behind beNNch, refer to Albers et al. (2022) [1]_. .. figure:: ../static/img/multi-area-model_5faa0e9c.png - Example ``beNNch`` output (Figure 5C of [1]_) + Example ``beNNch`` output (Figure 5C of [1]_) Strong-scaling performance of the `multi-area model `_ simulated with NEST on JURECA-DC. The left graph shows the absolute wall-clock time measured with Python-level timers for both network construction and state propagation. @@ -29,7 +32,7 @@ For more details on the conceptual ideas behind beNNch, refer to Albers et al. ( The lower right graph shows the relative contribution of these phases to the state-propagation time. -.. seealso:: +.. seealso:: For further details, see the accompanying `beNNch GitHub Page `_. And for a detailed step-by-step walk though see `Walk through guide `_. @@ -43,6 +46,3 @@ References .. [1] Albers J., et al (2022). A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations. Frontiers in Neuroinformatics(16):837549. https://doi.org/10.3389/fninf.2022.837549 - - - diff --git a/doc/htmldoc/installation/cmake_options.rst b/doc/htmldoc/installation/cmake_options.rst index 39c69c0dc8..be0dcdfe80 100644 --- a/doc/htmldoc/installation/cmake_options.rst +++ b/doc/htmldoc/installation/cmake_options.rst @@ -99,6 +99,28 @@ Use Python to build PyNEST For more details, see the :ref:`Python binding ` section below. +.. _benchmark_cmake: + +Performance and energy saving in benchmarks +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For benchmarking, we recommend that the following options are used to improve performance. + ++------------------------------------------+-----------------------------------------------+ +| ``-Dwith-optimize="-O3; -march=native"`` | optimized for making code as fast as possible | +| | for the specific type of CPU that you're | +| | using to compile NEST | ++------------------------------------------+-----------------------------------------------+ +| ``-Dwith-defines=-DNDEBUG`` | disables debugging of code | ++------------------------------------------+-----------------------------------------------+ + +.. note:: + + These options should *only be used after* you have performed several simulations of a given model with the + ``-Dwith-optimize="-O2"`` option. + + + Select parallelization scheme ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From a65a4eefddd527bbc58f53bcdc867a0f24a6c67a Mon Sep 17 00:00:00 2001 From: jessica-mitchell Date: Thu, 7 Mar 2024 12:16:45 +0100 Subject: [PATCH 2/3] Apply suggestions from code review Co-authored-by: Hans Ekkehard Plesser --- doc/htmldoc/hpc/benchmarking.rst | 2 +- doc/htmldoc/installation/cmake_options.rst | 29 ++++++++++++++-------- 2 files changed, 20 insertions(+), 11 deletions(-) diff --git a/doc/htmldoc/hpc/benchmarking.rst b/doc/htmldoc/hpc/benchmarking.rst index a6420e981c..e32b0fd7c2 100644 --- a/doc/htmldoc/hpc/benchmarking.rst +++ b/doc/htmldoc/hpc/benchmarking.rst @@ -5,7 +5,7 @@ Benchmarking NEST .. seealso:: - When compiling NEST to perform benchmarks, see :ref:`our cmake options ` for improved performance and energy saving. + When compiling NEST to perform benchmarks, see :ref:`our cmake options ` for improved performance and energy saving. beNNch ~~~~~~ diff --git a/doc/htmldoc/installation/cmake_options.rst b/doc/htmldoc/installation/cmake_options.rst index be0dcdfe80..ed72a32f5b 100644 --- a/doc/htmldoc/installation/cmake_options.rst +++ b/doc/htmldoc/installation/cmake_options.rst @@ -99,25 +99,34 @@ Use Python to build PyNEST For more details, see the :ref:`Python binding ` section below. -.. _benchmark_cmake: +.. _performance_cmake: -Performance and energy saving in benchmarks -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Maximize performance, reduce energy consumption +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -For benchmarking, we recommend that the following options are used to improve performance. +The following options help to optimize NEST for maximal performance and thus reduced energy consumption. +------------------------------------------+-----------------------------------------------+ -| ``-Dwith-optimize="-O3; -march=native"`` | optimized for making code as fast as possible | -| | for the specific type of CPU that you're | -| | using to compile NEST | +| ``-Dwith-optimize="-O3 -march=native"`` | Activate most compiler options that do not | +| | affect compliance with IEEE754 numerics and | +| | optimize for CPU type used | +------------------------------------------+-----------------------------------------------+ -| ``-Dwith-defines=-DNDEBUG`` | disables debugging of code | +| ``-Dwith-defines=-DNDEBUG`` | Disable all ``assert()`` statements in NEST | +------------------------------------------+-----------------------------------------------+ .. note:: - These options should *only be used after* you have performed several simulations of a given model with the - ``-Dwith-optimize="-O2"`` option. + * In our experience, gains from these optimizations are not very large. I can still be sensible to test them, + especially if you are going to perform large numbers of simulations. + * Your particular use case may touch upon corner cases in NEST execution that our extensive test suite has + not covered. Internal consistency tests in NEST in form of ``assert()`` statements can help to detect such + corner cases. Using the optimization options above removes these internal checks and thus increases the + risk that NEST will produce incorrect results. Therefore, use these options *only after you have performed + multiple simulations of your specific model with default optimization settings* (i.e., ``-O2``) and assertions + thus in place. + * Using ``-march=native`` requires that you build NEST on the same CPU architecture as you will use to run it. + * For the technically minded: Even just using ``-O3`` removes some ``assert()``s from NEST since we + have wrapped some of them in functions which get eliminated due to interprocedural optimization. ``` From 78530652e89a8fd0fd24ef81d684cbf1eedb788c Mon Sep 17 00:00:00 2001 From: jessica-mitchell Date: Thu, 7 Mar 2024 12:27:55 +0100 Subject: [PATCH 3/3] Improve language and fix typos --- doc/htmldoc/installation/cmake_options.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/doc/htmldoc/installation/cmake_options.rst b/doc/htmldoc/installation/cmake_options.rst index ed72a32f5b..f6c70c1c5a 100644 --- a/doc/htmldoc/installation/cmake_options.rst +++ b/doc/htmldoc/installation/cmake_options.rst @@ -116,17 +116,17 @@ The following options help to optimize NEST for maximal performance and thus red .. note:: - * In our experience, gains from these optimizations are not very large. I can still be sensible to test them, - especially if you are going to perform large numbers of simulations. - * Your particular use case may touch upon corner cases in NEST execution that our extensive test suite has - not covered. Internal consistency tests in NEST in form of ``assert()`` statements can help to detect such - corner cases. Using the optimization options above removes these internal checks and thus increases the + * In our experience, gains from these optimizations are not very large. It can still be sensible to test them, + especially if you are going to perform a large number of simulations. + * Your particular use case may contain edge cases during NEST execution that our extensive test suite has + not covered. Internal consistency tests in NEST in the form of ``assert()`` statements can help to detect such + edge cases. Using the optimization options above removes these internal checks and thus increases the risk that NEST will produce incorrect results. Therefore, use these options *only after you have performed - multiple simulations of your specific model with default optimization settings* (i.e., ``-O2``) and assertions - thus in place. + multiple simulations of your specific model with default optimization settings* (i.e., ``-O2``), which leaves the assertions + in place. * Using ``-march=native`` requires that you build NEST on the same CPU architecture as you will use to run it. - * For the technically minded: Even just using ``-O3`` removes some ``assert()``s from NEST since we - have wrapped some of them in functions which get eliminated due to interprocedural optimization. ``` + * For the technically minded: Even just using ``-O3`` removes some ``assert()`` statements from NEST since we + have wrapped some of them in functions, which get eliminated due to interprocedural optimization.