llama : NvAPI performance state change support

[sasha0552]

Note: I will continue this PR a bit later

Related: #8084

Reference implementation

TODO:

• Implement performance state switching functions
• Place performance state switching calls in a common function before/after inference start/end
  • Switch only if Pascal GPU(s) present
• Compile only if CUDA enabled
  • Enable by default if CUDA enabled, otherwise disable
• Log performance state changes and library loading status
• Synchronize pstate changes between n instances of llama.cpp on a single GPU
• Clean up temporary/debug code

---

Alternative implementations (just thoughts):

• A separate daemon process?
• Add options in main/server/etc to allow calling processes before/after inference? (probably the simplest solution)

---

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[sasha0552]

Superseded with nvidia-pstated.

[mirh]

I mean.. is it though?
I see why a datacenter gpu that doesn't even have a display output could even call it a day with just a rough loop that checks for "any activity at all", but for the seemingly best results you would need some kind of "token cycles awareness".

Besides on top of that, a very nice feature that could also be integrated with direct nvapi support is bus activity monitoring.
It's supposedly not super accurate, but printing a note when pcie is busy more than 50 or 70 percent of the time could help many people to quickly diagnose when a model isn't slow just because it is slow but because they are depending on RAM swapping for non-trivial amounts of layers.