Skip to content

Commit

Permalink
fix(iceberg): Position delete bug while doing upper bound check
Browse files Browse the repository at this point in the history
  While updating the bitmap for delete positions, we are
  including the exclusive upper bound, leading to incorrectly
  setting the buffer size
  • Loading branch information
nmahadevuni committed Feb 26, 2025
1 parent f5c25ab commit 9ad3530
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ void PositionalDeleteFileReader::updateDeleteBitmap(
static_cast<uint64_t>(deleteBitmapBuffer->size()),
deletePositionsOffset_ == 0 ||
(deletePositionsOffset_ < deletePositionsVector->size() &&
deletePositions[deletePositionsOffset_] > rowNumberUpperBound)
deletePositions[deletePositionsOffset_] >= rowNumberUpperBound)
? 0
: bits::nbytes(
deletePositions[deletePositionsOffset_ - 1] + 1 -
Expand Down
4 changes: 4 additions & 0 deletions velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -689,6 +689,10 @@ TEST_F(HiveIcebergTest, positionalDeletesMultipleSplits) {
{{"data_file_1", makeRandomIncreasingValues(0, 20000)}}}},
0,
3);

// Include only upper bound(which is exclusive) in delete positions for the
// second 10k batch of rows.
assertMultipleSplits({1000, 9000, 20000}, 1, 0, 20000, 3);
}

TEST_F(HiveIcebergTest, testPartitionedRead) {
Expand Down

0 comments on commit 9ad3530

Please sign in to comment.