Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to use RandomAccessInput.readBytes bulk method #17555

Merged
merged 2 commits into from
Mar 11, 2025

Conversation

asimmahmood1
Copy link
Contributor

@asimmahmood1 asimmahmood1 commented Mar 7, 2025

Description

This is a follow-up to Lucene change apache/lucene#12599. This class is used in StarTree feature, so maybe some performance benefits.

Other place I checked was FileCachedIndexInput and OnDemandBlockIndexInput but didn't require a change.

Note: I think the change is small enough to skip the ChangeLog. LGK if anyone feels differently.

Related Issues

Part of #16934

Check List

  • Functionality includes testing.
  • [n/a] API changes companion pull request created, if applicable.
  • [n/a] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Asim Mahmood <asim.seng@gmail.com>
@expani
Copy link
Contributor

expani commented Mar 7, 2025

Thanks for the change @asimmahmood1 LGTM

IMO we can wait for a couple of days until Nightlies run with JDK 21 then we can see the improvement with this change as well.

@mch2 Please have a look.

Copy link
Contributor

github-actions bot commented Mar 7, 2025

❌ Gradle check result for b5b5751:

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@expani
Copy link
Contributor

expani commented Mar 7, 2025

IMO we can wait for a couple of days until Nightlies run with JDK 21 then we can see the improvement with this change as well.

Traced back the usage and looks like it's exclusively for StarTreeIndexing. @bharath-techie/@sandeshkr419 are there any benchmarks for this ? If not, we can merge this PR without waiting for Nightlies.

@asimmahmood1 asimmahmood1 reopened this Mar 10, 2025
Copy link
Contributor

❌ Gradle check result for b5b5751: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for b5b5751: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for b5b5751: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@asimmahmood1
Copy link
Contributor Author

So odd, the gradle check says failed but the actual run has no failures: https://build.ci.opensearch.org/job/gradle-check/54399/

time pass: 1980
Workflow running status :false
Complete the run, checking results now......
Please check jenkins url for logs: https://build.ci.opensearch.org/job/gradle-check/54399/
Result: FAILURE
Error: Process completed with exit code 1.

On the bwc test failed to start: https://build.ci.opensearch.org/job/gradle-check/54399/consoleText

> Task :distribution:bwc:bugfix:buildBwcLinuxTar FAILED
 [2.19.1] > Task :distribution:archives:linux-tar:assemble
 [2.19.1] build complete, generating: /var/jenkins/workspace/gradle-check/search/distribution/bwc/bugfix/build/bwc/checkout-2.19/build/54399.tar.bz2
 [2.19.1] 
 [2.19.1] [Incubating] Problems report is available at: file:///var/jenkins/workspace/gradle-check/search/distribution/bwc/bugfix/build/bwc/checkout-2.19/build/reports/problems/problems-report.html
 [2.19.1] 
 [2.19.1] BUILD SUCCESSFUL in 5m 20s
 [2.19.1] 188 actionable tasks: 175 executed, 13 from cache

Copy link
Contributor

❌ Gradle check result for b5b5751: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@asimmahmood1
Copy link
Contributor Author

Now it failed with these. I think these are flaky tests.

[Test Result](https://build.ci.opensearch.org/job/gradle-check/54402/testReport/) (3 failures / +2)
[org.opensearch.repositories.azure.AzureBlobStoreRepositoryTests.testDeleteBlobs](https://build.ci.opensearch.org/job/gradle-check/54402/testReport/junit/org.opensearch.repositories.azure/AzureBlobStoreRepositoryTests/testDeleteBlobs/)
[org.opensearch.rest.ReactorNetty4StreamingStressIT.testCloseClientStreamingRequest](https://build.ci.opensearch.org/job/gradle-check/54402/testReport/junit/org.opensearch.rest/ReactorNetty4StreamingStressIT/testCloseClientStreamingRequest/)
[org.opensearch.rest.ReactorNetty4StreamingStressIT.classMethod](https://build.ci.opensearch.org/job/gradle-check/54402/testReport/junit/org.opensearch.rest/ReactorNetty4StreamingStressIT/classMethod/)

Copy link
Contributor

❌ Gradle check result for b5b5751: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@expani
Copy link
Contributor

expani commented Mar 10, 2025

@asimmahmood1 Also, sync your custom branch read_bytes_bulk if someone has fixed any flaky tests.
I see it's 3 commits behind.

Copy link
Contributor

❕ Gradle check result for d3e4e8c: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT.testSnapshotWithStuckNode

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link

codecov bot commented Mar 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.38%. Comparing base (ffa46ca) to head (d3e4e8c).
Report is 7 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #17555      +/-   ##
============================================
- Coverage     72.43%   72.38%   -0.06%     
+ Complexity    65721    65686      -35     
============================================
  Files          5311     5311              
  Lines        304941   304934       -7     
  Branches      44228    44225       -3     
============================================
- Hits         220890   220715     -175     
- Misses        65952    66101     +149     
- Partials      18099    18118      +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@msfroh msfroh merged commit e80b907 into opensearch-project:main Mar 11, 2025
34 of 35 checks passed
@asimmahmood1 asimmahmood1 deleted the read_bytes_bulk branch March 13, 2025 01:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants