Support Batch Execution Mode

[aglinxinyuan]

I would like to introduce the idea of supporting multiple runtime execution modes that users can choose from based on the requirements of their use case and the characteristics of their jobs.

The current (and default) execution behavior of our engine is what we call pipelined, or STREAMING, execution mode. In this mode, each operator performs continuous, incremental processing as data flows through the pipeline.

In addition, we plan to support a batch-style execution mode, referred to as BATCH execution mode. This mode executes jobs in a manner more reminiscent of traditional batch processing. We intend to enable this mode via a configuration flag.

Our unified approach to stream and batch processing ensures that applications executed over bounded inputs will produce the same final results regardless of the selected execution mode. Enabling BATCH execution allows the engine to apply additional optimizations that are only possible when operators know that their inputs are bounded.

Below, I will provide an example illustrating the differences between these execution modes:

[chenlica]

This is an exciting development! With this improvement, the engine can support two execution modes, and a user can choose one based on their preference (e.g., performance versus latency). The batch mode also provides additional optimization opportunities such as vectorization/SIMD.

[Yicong-Huang]

In general I like the idea of supporting multiple execution modes, in one engine. Some things to consider:

1. I don't think it is a good idea to call it batch vs streaming. They are different concepts. "batch" is for bounded and complete data. "streaming" is for unbounded and continued data. it is more of operator-at-a-time and tuple-at-a-time. Maybe you can call them Materialized mode vs Pipelined mode.
2. please correct me if I am wrong, the backend difference is on the region size. the materialized mode is equivalent to create a region per operator, while the pipelined mode is just using pasta to cut regions. If so, I actually think it might be better to define a parameter called region size with a sliding bar, where a user can choose from 1 operator/region, 2 operator/region, all the way to auto operator/region (i.e., pasta). this gives more flexility to configure and can include your two modes as well. Also would be more easier to understand to users: in their language, it's a matter of how many operators can execute "at the same time".

[Xiao-zhen-Liu]

I agree with the Materialized vs. Pipelined name.
The sliding region size idea is nice to have, but would require changing the cost model of Pasta. I think Materialized mode vs. Pipelined mode is good enough.

[chenlica]

I also agree with the Materialized vs. Pipelined name.

I also agree with @Xiao-zhen-Liu about the idea of "sliding region size."

[aglinxinyuan]

I will go with this plan then.