[GLUTEN-8889][CORE] Bump Spark version from 3.5.2 to 3.5.5 #8890
Conversation
|
Run Gluten ClickHouse CI on ARM |
|
@jlfsdtc please fix spark 35 pipeline of ch |
jackylee-ch
force-pushed
the
bump_spark_3.5.5
branch
from
50ca277 to
83a868d
Compare
|
Run Gluten ClickHouse CI on ARM |
|
Run Gluten ClickHouse CI on ARM |
|
Run Gluten ClickHouse CI on ARM |
|
Run Gluten ClickHouse CI on ARM |
|
Let's first upgrade to 3.5.5 and then upgrade Icerberg? After upgrading to 1.8.1, Icerberg only supports JDK 17, and clickhouse backend need take some time to support. @jlfsdtc will open pr for upgradeing to 3.5.5. |
@baibaichen We need upgrade them both, as Iceberg 1.8.1 fix the core_dump problem with Spark 3.5.4. |
I understand this，We will try to support JDK 17 in the next month. My suggestion is to upgrade to 3.5.5 first, then use different iceberg versions for different JDKs, and finally, Clickhouse support JDK 17. Some users are still using JDK 8, and we still need to support them. |
It's a block issue for iceberg with Spark 3.5.5. To upgrade to Spark 3.5.5, we must use Iceberg 1.8.1. |
What changes were proposed in this pull request?
Support Spark 3.5.5 and upgrade iceberg version to 1.8.1 to fix the compatibility issue with Spark 3.5.4.
How was this patch tested?
GA