Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(rust): Add equi joins to new streaming engine #19869

Merged
merged 32 commits into from
Dec 17, 2024

Conversation

orlp
Copy link
Collaborator

@orlp orlp commented Nov 19, 2024

There is still a lot of (very) low-hanging performance optimization to be done, but with this PR all equijoins should be fully functional on the new streaming engine.

@orlp orlp changed the title refactor(rust): Add equi join to new streaming engine refactor(rust): Add equi joins to new streaming engine Nov 19, 2024
@github-actions github-actions bot added internal An internal refactor or improvement rust Related to Rust Polars labels Nov 19, 2024
Copy link

codecov bot commented Nov 19, 2024

Codecov Report

Attention: Patch coverage is 0.81893% with 1090 lines in your changes missing coverage. Please review.

Project coverage is 79.13%. Comparing base (2fc26cf) to head (1e48d2c).
Report is 42 commits behind head on main.

Files with missing lines Patch % Lines
crates/polars-stream/src/nodes/joins/equi_join.rs 0.00% 652 Missing ⚠️
...s/polars-expr/src/chunked_idx_table/row_encoded.rs 0.00% 235 Missing ⚠️
crates/polars-expr/src/hash_keys.rs 0.00% 126 Missing ⚠️
...es/polars-arrow/src/compute/take/generic_binary.rs 0.00% 20 Missing ⚠️
crates/polars-utils/src/idx_map/bytes_idx_map.rs 0.00% 14 Missing ⚠️
crates/polars-core/src/frame/mod.rs 30.00% 7 Missing ⚠️
crates/polars-utils/src/index.rs 25.00% 6 Missing ⚠️
crates/polars-core/src/datatypes/field.rs 0.00% 4 Missing ⚠️
crates/polars-ops/src/frame/join/args.rs 0.00% 4 Missing ⚠️
crates/polars-stream/src/nodes/in_memory_map.rs 0.00% 4 Missing ⚠️
... and 6 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #19869      +/-   ##
==========================================
- Coverage   79.60%   79.13%   -0.48%     
==========================================
  Files        1567     1572       +5     
  Lines      218528   219839    +1311     
  Branches     2462     2462              
==========================================
- Hits       173969   173959      -10     
- Misses      43992    45312    +1320     
- Partials      567      568       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@orlp orlp force-pushed the new-streaming-equi-join branch from c7def77 to 958df17 Compare November 20, 2024 18:22
@orlp orlp marked this pull request as ready for review November 25, 2024 15:31
@orlp orlp force-pushed the new-streaming-equi-join branch from a83f38e to a601af2 Compare November 26, 2024 16:41
@orlp orlp force-pushed the new-streaming-equi-join branch from a601af2 to 145ccc1 Compare December 17, 2024 00:18
@ritchie46 ritchie46 merged commit a06a5e9 into pola-rs:main Dec 17, 2024
26 checks passed
@c-peters c-peters added the accepted Ready for implementation label Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted Ready for implementation internal An internal refactor or improvement rust Related to Rust Polars
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants