Extend the simple UDAF interface with function-level states

[kagamiori]

The simple function interface for UDAFs currently doesn't allow function-level states. Function-level states are variables hold by a UDAF instance that are typically computed once and used at every row when adding inputs to accumulators or extracting values from accumulators. With the traditional vector function interface, UDAF authors use function-level states by simply defining them as data members of the function class. On the contrary, the simple function interface doesn’t expose any data member of the function class to the author-defined accumulator logics.

Below are a few examples where function-level states are necessary and useful.

1. Decimal functions usually need to know the precision and scale of the result Decimal type when extract values from accumulators. The result type is currently not exposed to author-defined accumulator type in the simple function interface.
2. Some functions perform heavy computation on a constant argument before processing all input rows, such as approx_most_frequent, and store the computation result in a function-level state. With the current simple function interface, the UDAF author would have to do the computation on the constant argument at every row.

To enable function-level states, we can extend SimpleAggregateAdapter to allow the UDAF author to define a FunctionState struct in the function class and let SimpleAggregateAdapter hold an instance of FunctionState. The author then implement a void initialize(FunctionState& state, const TypePtr& resultType, const std::vector<VectorPtr>& constantInputs) in their function class that assign values to the FunctionState instance. This initialize() function will be called only once when the aggregation function instance is created. Finally, all methods in the author-defined accumulator type receive this FunctionState instance as a const argument.

A prototype with an example can be found in #8710.

[liujiayi771]

Thank you, it has been very helpful to me.

[liujiayi771]

Hi @kagamiori. While I was testing the newly developed agg function, I needed to obtain the resultType, so I used the FunctionState method you introduced. I found that it is usually usable, but it seems that streaming agg does not call the initialize method.

[liujiayi771]

velox/velox/exec/AggregateCompanionAdapter.cpp

Line 211 in e4f33f4

fn_->initializeNewGroups(groups, allSelectedRange);

Besides, do I also need to call initialize(outputType, {}) before this line? Otherwise, in some test scenarios, resultType hasn't been set.