Feature request: Rolling Window Batching

[yarncraft]

It would be great to have the option to aggregate incoming messages on a rolling window of fixed size. (https://github.com/asecurityteam/rolling)

[Jeffail]

Hey @yarncraft, sounds interesting. It could make sense to implement this as some sort of general purpose buffer type, which would emit discrete windows as a batch.

[yarncraft]

Hey @yarncraft, sounds interesting. It could make sense to implement this as some sort of general purpose buffer type, which would emit discrete windows as a batch.

Exactly, we're planning to use Benthos in streams mode at our IoT company. It has proven to be very useful and flexible to work with so far. The rolling window mode could be of use for certain analytics use cases such as anomaly detection etc.

[Jeffail]

Hey @yarncraft, I forgot to update this issue but there's now a window buffer to try out: https://www.benthos.dev/docs/components/buffers/system_window, it works off the system clock but if it goes well I'll eventually add more options.