Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify Iceberg positional delete file read #12488

Open
yingsu00 opened this issue Feb 28, 2025 · 0 comments · May be fixed by #12590
Open

Simplify Iceberg positional delete file read #12488

yingsu00 opened this issue Feb 28, 2025 · 0 comments · May be fixed by #12590
Assignees
Labels
enhancement New feature or request iceberg

Comments

@yingsu00
Copy link
Collaborator

Description

IcebergSplitReader maintains a deleteBitmap_. In IcebergSplitReader::next(), we needed to copy the unconsumed bytes from last batch from the end of deleteBitmap_ to the beginning of deleteBitmap_, because at that time we could not know the exact number of rows to read. However this can be avoided since int64_t nextReadSize(uint64_t size) was added to the Reader interface. With this function, we can know the exact number of rows that will be read in the next batch, so maintaining the deleteBitmapBitOffset_ and copying the left over deleteBitmap_ from last batch is not needed.

@yingsu00 yingsu00 added the enhancement New feature or request label Feb 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request iceberg
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants