Recommenders
Ottoscalr provides a pluggable recommender framework which makes it extensible to additional recommendation algorithms specific to the load patterns and environment in which the workloads operate.
This recommender framework defines a recommendation workflow that's executed by the ottoscalr controllers and interfaces to plug in bespoke recommenders. At the core, recommenders should retrospectively (or even adopt techniques to predict future trends) look at the utilization metrics for a workload over a configurable time period and recommend a HPA configuration.
The framework integrates with standard promql compliant metric sources and provide the data in a simple consumable format to the recommenders. This ensures the recommender implementations needn't be concerned with the querying of metric sources.
For more information regarding the metrics required and promql queries for ottoscalr to function, refer the Metrics Source Wiki Page
The framework also provides the capabilities to interpolate data where there's data holes or any data that needs splicing. This is particularly useful when there are expected spiky patterns in the data which need not be accounted for in the recommendation generated.
A recommender should recommend a target HPA configuration for a workload. The interface for a recommender is as follows:
type Recommender interface {
Recommend(ctx context.Context, wm WorkloadMeta) (*v1alpha1.HPAConfiguration, error)
}
We'll go over the details of how a good recommender is implemented in the "Guidelines for recommenders" section.
Policy Iterator selects a policy from the sorted list of graded policies. Policy iterator looks into the current policy that's applied for a workload and selects the next applicable policy. It's named after the Iterator design pattern. The interface for a policy iterator is as follows:
type PolicyIterator interface {
NextPolicy(ctx context.Context, wm WorkloadMeta) (*Policy, error)
GetName() string
}
The recommendation workflow chains a recommender and a set of policy iterators to derive the HPA configuration for a workload.
A well implemented recommender would recommend an ideal HPA which would yield best possible savings by optimizing the workload slack all the while keeping the number of breaches to SLOs to 0 for the entire period of historical data.
The default CPU utilization recommender currently bundled with ottoscalr derives the HPA such that the cpu utilization doesn't go beyond a redline threshold considering the ACL for a workload and utilization demand surges. The safety buffer ensured by the HPA by virtue of the utilization threshold should guarantee that any surges to the demand would be absorbed by the current set of pods before the next set of pods (triggered by the scale out event) become ready. This autoscaling safety buffer (ASB) required (illustrated in the following diagram) will ensure that workloads will not face disruptions due to autoscaling. The HPA recommended would yield the best savings and slack given the criteria defined.
During the ACL window , the traffic on the workload can continue to increase and needs to be served by the current pods. This extra traffic needs to be absorbed by what we define as an Autoscaling Safety Buffer (ASB).
Autoscaling Safety Buffer(ASB) = ( redlineMetricValue - desiredMetricValue ) * currentReplicas * perPodCapacity
For eg. We take a workload with a pod size of 8 cores and set the desiredMetricValue for cpu utilization based trigger as 60%.We assume that the workload pods redline at 85% avg cpu utilization. At the point when currentReplicas are 100, ASB would be (.85 - .60) * 100 * 8 = 200 cores.
To contribute a recommender to the project, please raise a PR detailing the algorithm. The implementation to adhere to the published recommender interfaces.