[GLUTEN-7548][VL][test] Optimize BHJ in velox backend

[JkSelf]

What changes were proposed in this pull request?

This PR implements the optimization from the CK backend in the BHJ to the Velox backend, ensuring that the hash table is built only once per executor. The detailed design document can be found here.

How was this patch tested?

Existing tests

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

• Other pull requests

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

#7548

[github-actions]

Run Gluten Clickhouse CI on x86

[FelixYBW]

In long term, we need to implement the Spark way. Broadcast hashtable instead of raw table data.

[JkSelf]

In long term, we need to implement the Spark way. Broadcast hashtable instead of raw table data.

@FelixYBW Yes, we will support broadcasting the hash table approach after adding serialization/deserialization support to Velox's HashTable.

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[FelixYBW]

@zhztheplayer Is there memory management issue in this solution? Is the memory allocated in storage memory?

@JkSelf will this solution helpful to the final solution?

[JkSelf]

@JkSelf will this solution helpful to the final solution?

@FelixYBW Yes, the primary difference between Design 1 and Design 2 is the need for serialization and deserialization of Velox's hash table. Most of the remaining code can be shared between the two designs. We will evaluate the TPCH performance after addressing the result mismatch issue. If the performance does not meet expectations, we will proceed with implementing Design 2.

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86