feat: Discover Instance Type Capacity Memory Overhead

[jukie]

Fixes #5161

Description
This adds a new controller for managing a cache on the instancetype provider that stores the memory capacity overhead for each instance type by comparing the actual value after a Kubernetes Node gets registered with the cluster.
The cache is then implemented by NewInstanceType() when calculating memory capacity. If a cached value doesn't exist it will fall back to the existing logic of vmMemoryOverheadPercent.

The cache is keyed by a combination of instance type name and hash of nodeClass.Status.AMIs so this should always be accurate and when an AMI update is triggered will use the existing logic of calculating against vmMemoryOverheadPercent to ensure safe instance creation every time.

How was this change tested?
Suite tests

Does this change impact docs?

• Yes, PR includes docs updates
• Yes, issue opened:
• No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[netlify]

✅ Deploy Preview for karpenter-docs-prod canceled.

Name	Link
🔨 Latest commit	065f328
🔍 Latest deploy log	https://app.netlify.com/sites/karpenter-docs-prod/deploys/66faef18e9c75d000842516a

[njtran]

Thanks for the PR! Looks really good!

[njtran]

Not sure you want to do the hash, as that encompasses more details than we care about for resource allocation.

[jukie]

Further down there's a match against this map and the node's nodeclass hash annotation so that we're only storing cache values for the latest version of the nodeclass. Would you instead prefer keying this map on the nodeclass name and then while during the node loop having a check that the node's ami is within the nodeclass status.amis list?

[jukie]

CC @jmdeal (I think you were the one on the call today?) There's a negligible amount of variance when measuring overhead in Ki but I was unable to find any variance between the same instance type when measuring in Mi which my PR is doing.

Here's a simple script that can be used to verify: https://gist.github.com/jukie/df045af8fec68941f5d119044bf04aee

Instance Type: m6i.24xlarge
Variance detected in memory capacities:
 - 389990024
 - 389990016
 - 389990004
 - 389989996
 - 389990000
 - 389990004
 - 389989988
 - 389990000
 - 389990012
 - 389990012
 - 389989988
 - 389989980
 - 389990020