You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
IcebergSplitReader maintains a deleteBitmap_. In IcebergSplitReader::next(), we needed to copy the unconsumed bytes from last batch from the end of deleteBitmap_ to the beginning of deleteBitmap_, because at that time we could not know the exact number of rows to read. However this can be avoided since int64_t nextReadSize(uint64_t size) was added to the Reader interface. With this function, we can know the exact number of rows that will be read in the next batch, so maintaining the deleteBitmapBitOffset_ and copying the left over deleteBitmap_ from last batch is not needed.
The text was updated successfully, but these errors were encountered:
Description
IcebergSplitReader maintains a deleteBitmap_. In
IcebergSplitReader::next()
, we needed to copy the unconsumed bytes from last batch from the end of deleteBitmap_ to the beginning of deleteBitmap_, because at that time we could not know the exact number of rows to read. However this can be avoided sinceint64_t nextReadSize(uint64_t size)
was added to the Reader interface. With this function, we can know the exact number of rows that will be read in the next batch, so maintaining the deleteBitmapBitOffset_ and copying the left over deleteBitmap_ from last batch is not needed.The text was updated successfully, but these errors were encountered: