Simulation "Total CPU Time" is much larger than "Total Real Time"

[Amarelio]

Dear all,

I noticed today after running an OpenFAST 2.4.0 simulation that the "Total CPU Time" was much larger than "Total Real Time". The OS is Ubuntu 20.4. This is the execution information:

**************************************************************************************************
OpenFAST

 Copyright (C) 2021 National Renewable Energy Laboratory
 Copyright (C) 2021 Envision Energy USA LTD

 This program is licensed under Apache License Version 2.0 and comes with ABSOLUTELY NO WARRANTY.
 See the "LICENSE" file distributed with this software for details.
 **************************************************************************************************

 OpenFAST-GIT-NOTFOUND
 Compile Info:
  - Compiler: Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R)
  64, Version 2021.2.0 Build 20210228_000000
  - Architecture: 64 bit
  - Precision: single
  - Date: Jun 06 2021
  - Time: 18:46:54
 Execution Info:
  - Date: 07/14/2021
  - Time: 20:04:52+0200

Total Real Time:       10.673 minutes
  Total CPU Time:        24.913 minutes
  Simulation CPU Time:   24.892 minutes
  Simulated Time:        16.667 minutes
  Time Ratio (Sim/CPU):  0.66955

  OpenFAST terminated normally.
***********************************************

Has this ever happened before? Since OpenFAST 2.4.0 is not parallelized, I was expecting one simulation to use up one core so it would be impossible for the CPU time be much higher than the real time. Am I thinking correctly?

P.S: I checked the process with the "top" command and indeed the simulation was consuming a percentage of around 200% of cpu.

Best regards

[rafmudaf]

Hi @Amarelio I've noticed this, as well. Though I don't fully understand it, I think its due to the Intel MKL libraries being threaded on their own. @andrew-platt @bjonkman do you have any more insight?

I have tried setting the MKL_NUM_THREADS environment variable to 1, but no change.
There's also this output in CMake about pthread

-- Looking for pthread.h
-- Looking for pthread.h - found

[sayerhs]

@rafmudaf What are your compiler flags during Intel build? Does it auto-parallelize/vectorize loops? If so, just setting MKL_NUM_THREADS might not be sufficient, you'll also want to explicitly set OMP_NUM_THREADS=1 to control how the loops in OpenFAST themselves are run.

[bjonkman]

@rafmudaf, there is also the -nothreads flag.

OpenFAST uses CPU_TIME to get a "processor-dependent approximation of the processor time". A quick look at the documentation shows that this maybe isn't the best routine to call for performance of multi-threaded applications.