Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] fix spark352-scala2.13 test home #8938

Merged
merged 1 commit into from
Mar 9, 2025

Conversation

zhouyuan
Copy link
Contributor

@zhouyuan zhouyuan commented Mar 8, 2025

What changes were proposed in this pull request?

followup #8881

How was this patch tested?

pass GHA

@github-actions github-actions bot added the INFRA label Mar 8, 2025
Copy link

github-actions bot commented Mar 8, 2025

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

mkdir -p ${INSTALL_DIR}/shims/spark"${spark_version_short}"/spark_home/assembly/target/scala-"${scala_version}"
mv jars ${INSTALL_DIR}/shims/spark"${spark_version_short}"/spark_home/assembly/target/scala-"${scala_version}"
mkdir -p ${INSTALL_DIR}/shims/spark"${spark_version_short}""scala_suffix_short"/spark_home/assembly/target/scala-"${scala_version}"
mv jars ${INSTALL_DIR}/shims/spark"${spark_version_short}""scala_suffix_short"/spark_home/assembly/target/scala-"${scala_version}"
Copy link
Contributor Author

@zhouyuan zhouyuan Mar 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cc: @yaooqinn it looks the folder path is wrong for scala-2.13

@zhouyuan zhouyuan force-pushed the wip_fix_spark35_scala213_image branch 2 times, most recently from 8894253 to 5f0871a Compare March 8, 2025 13:26
Signed-off-by: Yuan <yuanzhou@apache.org>
@zhouyuan zhouyuan force-pushed the wip_fix_spark35_scala213_image branch from 5f0871a to 9eb402d Compare March 8, 2025 13:59
@PHILO-HE PHILO-HE merged commit 257dbc6 into apache:main Mar 9, 2025
4 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_03_09_2025_time.csv log/native_master_03_08_2025_7a772e8e9c_time.csv difference percentage
q1 11.93 11.78 -0.149 98.75%
q2 11.41 12.31 0.899 107.88%
q3 3.68 1.56 -2.124 42.35%
q4 51.25 52.27 1.019 101.99%
q5 8.36 8.85 0.486 105.82%
q6 4.60 4.79 0.195 104.24%
q7 3.54 4.67 1.131 131.94%
q8 3.44 4.87 1.429 141.51%
q9 16.07 16.03 -0.039 99.76%
q10 12.17 12.87 0.693 105.69%
q11 28.71 28.46 -0.249 99.13%
q12 1.67 1.64 -0.033 98.04%
q13 6.61 5.66 -0.954 85.57%
q14a 42.89 42.83 -0.056 99.87%
q14b 38.74 40.14 1.396 103.60%
q15 3.33 3.99 0.657 119.72%
q16 5.67 4.59 -1.078 80.99%
q17 7.18 5.62 -1.563 78.25%
q18 7.13 7.75 0.621 108.71%
q19 3.63 3.95 0.316 108.71%
q20 2.24 3.81 1.573 170.28%
q21 0.79 1.65 0.857 208.37%
q22 3.38 3.29 -0.087 97.41%
q23a 62.87 61.70 -1.169 98.14%
q23b 72.18 72.69 0.507 100.70%
q24a 69.82 70.47 0.658 100.94%
q24b 67.86 68.90 1.046 101.54%
q25 5.02 5.68 0.657 113.08%
q26 2.75 2.23 -0.522 81.03%
q27 2.73 2.33 -0.397 85.44%
q28 17.62 17.07 -0.554 96.86%
q29 7.54 7.79 0.247 103.28%
q30 5.20 4.92 -0.277 94.68%
q31 7.88 7.97 0.085 101.07%
q32 1.89 1.42 -0.476 74.85%
q33 4.03 3.87 -0.161 96.01%
q34 3.80 3.65 -0.146 96.17%
q35 7.47 7.68 0.212 102.84%
q36 2.41 2.34 -0.064 97.32%
q37 3.72 3.88 0.162 104.34%
q38 11.95 11.68 -0.270 97.74%
q39a 4.40 4.29 -0.109 97.53%
q39b 3.93 3.45 -0.476 87.88%
q40 4.12 3.70 -0.420 89.81%
q41 0.64 0.62 -0.013 97.96%
q42 0.86 0.92 0.060 106.93%
q43 2.79 2.64 -0.156 94.42%
q44 6.03 6.01 -0.024 99.59%
q45 3.59 3.13 -0.454 87.35%
q46 4.40 5.35 0.955 121.73%
q47 9.78 11.05 1.273 113.01%
q48 3.72 3.28 -0.440 88.17%
q49 5.58 5.43 -0.148 97.35%
q50 17.18 17.60 0.421 102.45%
q51 7.24 7.07 -0.170 97.65%
q52 0.80 0.99 0.191 123.87%
q53 1.47 1.91 0.447 130.47%
q54 5.52 5.20 -0.320 94.21%
q55 0.92 1.02 0.097 110.52%
q56 3.92 4.37 0.446 111.37%
q57 6.87 6.39 -0.485 92.94%
q58 3.59 2.82 -0.776 78.40%
q59 4.35 4.33 -0.015 99.65%
q60 4.78 4.61 -0.167 96.51%
q61 4.41 4.48 0.072 101.62%
q62 2.78 2.61 -0.164 94.09%
q63 1.67 1.30 -0.368 77.94%
q64 36.38 35.87 -0.509 98.60%
q65 10.92 11.13 0.215 101.97%
q66 3.87 2.99 -0.886 77.12%
q67 58.13 57.86 -0.270 99.54%
q68 3.32 3.18 -0.135 95.94%
q69 4.95 5.40 0.457 109.23%
q70 6.09 5.78 -0.311 94.90%
q71 4.66 5.07 0.415 108.92%
q72 20.94 20.92 -0.021 99.90%
q73 2.67 2.53 -0.138 94.81%
q74 17.20 17.62 0.422 102.46%
q75 24.06 22.92 -1.136 95.28%
q76 6.76 7.10 0.343 105.07%
q77 3.36 2.22 -1.144 66.00%
q78 32.89 32.65 -0.240 99.27%
q79 3.79 3.54 -0.252 93.35%
q80 10.25 10.81 0.561 105.47%
q81 7.01 7.24 0.226 103.22%
q82 5.53 5.36 -0.168 96.96%
q83 2.32 1.99 -0.325 85.98%
q84 3.08 4.39 1.315 142.71%
q85 5.54 6.32 0.778 114.03%
q86 2.12 2.01 -0.109 94.85%
q87 11.87 11.98 0.111 100.93%
q88 15.07 15.64 0.574 103.81%
q89 2.15 2.25 0.103 104.78%
q90 2.11 2.32 0.217 110.31%
q91 3.40 4.05 0.654 119.26%
q92 1.64 1.75 0.112 106.81%
q93 23.58 23.48 -0.093 99.61%
q94 8.19 8.79 0.597 107.29%
q9 55.94 56.01 0.074 100.13%
q5 2.45 1.84 -0.610 75.11%
q96 11.01 11.44 0.423 103.84%
q97 2.09 2.23 0.137 106.56%
q98 5.13 6.03 0.901 117.55%
q99 0.47 0.56 0.086 118.30%
total 1171.41 1177.52 6.110 100.52%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_03_09_2025_time.csv log/native_master_03_08_2025_7a772e8e9c_time.csv difference percentage
q1 24.63 24.99 0.370 101.50%
q2 26.55 25.99 -0.563 97.88%
q3 33.01 31.52 -1.491 95.48%
q4 27.39 28.17 0.780 102.85%
q5 61.44 58.76 -2.675 95.65%
q6 9.72 6.71 -3.016 68.98%
q7 40.57 40.52 -0.048 99.88%
q8 67.42 62.60 -4.822 92.85%
q9 90.90 97.39 6.490 107.14%
q10 41.87 38.48 -3.385 91.91%
q11 16.38 15.58 -0.799 95.12%
q12 16.99 16.96 -0.037 99.78%
q13 24.45 24.89 0.435 101.78%
q14 11.18 12.21 1.033 109.24%
q15 26.41 26.26 -0.149 99.44%
q16 12.99 12.96 -0.024 99.82%
q17 74.57 75.82 1.244 101.67%
q18 114.45 114.83 0.384 100.34%
q19 14.30 20.91 6.613 146.25%
q20 25.33 23.60 -1.731 93.17%
q21 175.63 177.21 1.580 100.90%
q22 10.07 10.06 -0.017 99.83%
total 946.24 946.42 0.172 100.02%

@yaooqinn
Copy link
Member

LGTM, nice catch!

yikf pushed a commit to yikf/incubator-gluten that referenced this pull request Mar 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants