Skip to content

[GLUTEN-7548][VL][test] Optimize BHJ in velox backend #15535

[GLUTEN-7548][VL][test] Optimize BHJ in velox backend

[GLUTEN-7548][VL][test] Optimize BHJ in velox backend #15535

Triggered via pull request March 12, 2025 10:56
@JkSelfJkSelf
synchronize #8931
Status Success
Total duration 32s
Artifacts

clickhouse_be_trigger.yml

on: pull_request_target
add-comment
24s
add-comment
Fit to window
Zoom out
Zoom in

Annotations

29 errors
VeloxOrcDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L185
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#48958] +- Filter (int#48958 = int#48988) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#48956,short#48957,int#48958,long#48959L,float#48960,double#48961,decimal#48962,string#48963,binary#48964,bool#48965,date#48966,timestamp#48967,array#48968,struct#48969,map#48970]) : +- RelationV2[byte#48956, short#48957, int#48958, long#48959L, float#48960, double#48961, decimal#48962, string#48963, binary#48964, bool#48965, date#48966, timestamp#48967, array#48968, struct#48969, map#48970] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#48986,short#48987,int#48988,long#48989L,float#48990,double#48991,decimal#48992,string#48993,binary#48994,bool#48995,date#48996,timestamp#48997,array#48998,struct#48999,map#49000]) +- RelationV2[byte#48986, short#48987, int#48988, long#48989L, float#48990, double#48991, decimal#48992, string#48993, binary#48994, bool#48995, date#48996, timestamp#48997, array#48998, struct#48999, map#49000] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [int#48958] +- Join Inner, (int#48958 = int#48988) :- Filter isnotnull(int#48958) : +- RelationV2[int#48958] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(int#48988) +- RelationV2[int#48988] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(1937) ProjectExecTransformer [int#48958] +- ^(1937) BroadcastHashJoinExecTransformer [int#48958], [int#48988], Inner, BuildRight, false :- ^(1937) FilterExecTransformer isnotnull(int#48958) : +- ^(1937) BatchScanExecTransformer[int#48958] OrcScan DataFilters: [isnotnull(int#48958)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#48958)] +- ^(1937) InputIteratorTransformer[int#48988] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=511644] +- ^(1936) FilterExecTransformer isnotnull(int#48988) +- ^(1936) BatchScanExecTransformer[int#48988] OrcScan DataFilters: [isnotnull(int#48988)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#48988)] +- == Initial Plan == Project [int#48958] +- BroadcastHashJoin [int#48958], [int#48988], Inner, BuildRight, false :- Filter isnotnull(int#48958) : +- BatchScan[int#48958] OrcScan DataFilters: [isnotnull(int#48958)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=511435] +- Filter isnotnull(int#48988) +- BatchScan[int#48988] OrcScan DataFilters: [isnotnull(int#48988)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxOrcDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L328
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#48956] +- Filter (byte#48956 = byte#48986) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#48956,short#48957,int#48958,long#48959L,float#48960,double#48961,decimal#48962,string#48963,binary#48964,bool#48965,date#48966,timestamp#48967,array#48968,struct#48969,map#48970]) : +- RelationV2[byte#48956, short#48957, int#48958, long#48959L, float#48960, double#48961, decimal#48962, string#48963, binary#48964, bool#48965, date#48966, timestamp#48967, array#48968, struct#48969, map#48970] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#48986,short#48987,int#48988,long#48989L,float#48990,double#48991,decimal#48992,string#48993,binary#48994,bool#48995,date#48996,timestamp#48997,array#48998,struct#48999,map#49000]) +- RelationV2[byte#48986, short#48987, int#48988, long#48989L, float#48990, double#48991, decimal#48992, string#48993, binary#48994, bool#48995, date#48996, timestamp#48997, array#48998, struct#48999, map#49000] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [byte#48956] +- Join Inner, (byte#48956 = byte#48986) :- Filter isnotnull(byte#48956) : +- RelationV2[byte#48956] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(byte#48986) +- RelationV2[byte#48986] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(1992) ProjectExecTransformer [byte#48956] +- ^(1992) BroadcastHashJoinExecTransformer [byte#48956], [byte#48986], Inner, BuildRight, false :- ^(1992) FilterExecTransformer isnotnull(byte#48956) : +- ^(1992) BatchScanExecTransformer[byte#48956] OrcScan DataFilters: [isnotnull(byte#48956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] NativeFilters: [isnotnull(byte#48956)] +- ^(1992) InputIteratorTransformer[byte#48986] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=517361] +- ^(1991) FilterExecTransformer isnotnull(byte#48986) +- ^(1991) BatchScanExecTransformer[byte#48986] OrcScan DataFilters: [isnotnull(byte#48986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] NativeFilters: [isnotnull(byte#48986)] +- == Initial Plan == Project [byte#48956] +- BroadcastHashJoin [byte#48956], [byte#48986], Inner, BuildRight, false :- Filter isnotnull(byte#48956) : +- BatchScan[byte#48956] OrcScan DataFilters: [isnotnull(byte#48956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=517152] +- Filter isnotnull(byte#48986) +- BatchScan[byte#48986] OrcScan DataFilters: [isnotnull(byte#48986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxParquetDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#50408] +- Filter (int#50408 = int#50438) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#50406,short#50407,int#50408,long#50409L,float#50410,double#50411,decimal#50412,string#50413,binary#50414,bool#50415,date#50416,timestamp#50417,array#50418,struct#50419,map#50420]) : +- RelationV2[byte#50406, short#50407, int#50408, long#50409L, float#50410, double#50411, decimal#50412, string#50413, binary#50414, bool#50415, date#50416, timestamp#50417, array#50418, struct#50419, map#50420] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#50436,short#50437,int#50438,long#50439L,float#50440,double#50441,decimal#50442,string#50443,binary#50444,bool#50445,date#50446,timestamp#50447,array#50448,struct#50449,map#50450]) +- RelationV2[byte#50436, short#50437, int#50438, long#50439L, float#50440, double#50441, decimal#50442, string#50443, binary#50444, bool#50445, date#50446, timestamp#50447, array#50448, struct#50449, map#50450] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [int#50408] +- Join Inner, (int#50408 = int#50438) :- Filter isnotnull(int#50408) : +- RelationV2[int#50408] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(int#50438) +- RelationV2[int#50438] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2115) ProjectExecTransformer [int#50408] +- ^(2115) BroadcastHashJoinExecTransformer [int#50408], [int#50438], Inner, BuildRight, false :- ^(2115) FilterExecTransformer isnotnull(int#50408) : +- ^(2115) BatchScanExecTransformer[int#50408] ParquetScan DataFilters: [isnotnull(int#50408)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#50408)] +- ^(2115) InputIteratorTransformer[int#50438] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=529566] +- ^(2114) FilterExecTransformer isnotnull(int#50438) +- ^(2114) BatchScanExecTransformer[int#50438] ParquetScan DataFilters: [isnotnull(int#50438)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#50438)] +- == Initial Plan == Project [int#50408] +- BroadcastHashJoin [int#50408], [int#50438], Inner, BuildRight, false :- Filter isnotnull(int#50408) : +- BatchScan[int#50408] ParquetScan DataFilters: [isnotnull(int#50408)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=529357] +- Filter isnotnull(int#50438) +- BatchScan[int#50438] ParquetScan DataFilters: [isnotnull(int#50438)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxParquetDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#50407] +- Filter (short#50407 = short#50437) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#50406,short#50407,int#50408,long#50409L,float#50410,double#50411,decimal#50412,string#50413,binary#50414,bool#50415,date#50416,timestamp#50417,array#50418,struct#50419,map#50420]) : +- RelationV2[byte#50406, short#50407, int#50408, long#50409L, float#50410, double#50411, decimal#50412, string#50413, binary#50414, bool#50415, date#50416, timestamp#50417, array#50418, struct#50419, map#50420] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#50436,short#50437,int#50438,long#50439L,float#50440,double#50441,decimal#50442,string#50443,binary#50444,bool#50445,date#50446,timestamp#50447,array#50448,struct#50449,map#50450]) +- RelationV2[byte#50436, short#50437, int#50438, long#50439L, float#50440, double#50441, decimal#50442, string#50443, binary#50444, bool#50445, date#50446, timestamp#50447, array#50448, struct#50449, map#50450] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [short#50407] +- Join Inner, (short#50407 = short#50437) :- Filter isnotnull(short#50407) : +- RelationV2[short#50407] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(short#50437) +- RelationV2[short#50437] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2124) ProjectExecTransformer [short#50407] +- ^(2124) BroadcastHashJoinExecTransformer [short#50407], [short#50437], Inner, BuildRight, false :- ^(2124) FilterExecTransformer isnotnull(short#50407) : +- ^(2124) BatchScanExecTransformer[short#50407] ParquetScan DataFilters: [isnotnull(short#50407)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] NativeFilters: [isnotnull(short#50407)] +- ^(2124) InputIteratorTransformer[short#50437] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=530481] +- ^(2123) FilterExecTransformer isnotnull(short#50437) +- ^(2123) BatchScanExecTransformer[short#50437] ParquetScan DataFilters: [isnotnull(short#50437)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] NativeFilters: [isnotnull(short#50437)] +- == Initial Plan == Project [short#50407] +- BroadcastHashJoin [short#50407], [short#50437], Inner, BuildRight, false :- Filter isnotnull(short#50407) : +- BatchScan[short#50407] ParquetScan DataFilters: [isnotnull(short#50407)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=530272] +- Filter isnotnull(short#50437) +- BatchScan[short#50437] ParquetScan DataFilters: [isnotnull(short#50437)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#56483] +- Filter (short#56483 = short#56513) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#56482,short#56483,int#56484,long#56485L,float#56486,double#56487,decimal#56488,string#56489,binary#56490,bool#56491,date#56492,timestamp#56493,array#56494,struct#56495,map#56496]) : +- RelationV2[byte#56482, short#56483, int#56484, long#56485L, float#56486, double#56487, decimal#56488, string#56489, binary#56490, bool#56491, date#56492, timestamp#56493, array#56494, struct#56495, map#56496] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#56512,short#56513,int#56514,long#56515L,float#56516,double#56517,decimal#56518,string#56519,binary#56520,bool#56521,date#56522,timestamp#56523,array#56524,struct#56525,map#56526]) +- RelationV2[byte#56512, short#56513, int#56514, long#56515L, float#56516, double#56517, decimal#56518, string#56519, binary#56520, bool#56521, date#56522, timestamp#56523, array#56524, struct#56525, map#56526] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [short#56483] +- Join Inner, (short#56483 = short#56513) :- Filter isnotnull(short#56483) : +- RelationV2[short#56483] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(short#56513) +- RelationV2[short#56513] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2064) ProjectExecTransformer [short#56483] +- ^(2064) BroadcastHashJoinExecTransformer [short#56483], [short#56513], Inner, BuildRight, false :- ^(2064) FilterExecTransformer isnotnull(short#56483) : +- ^(2064) BatchScanExecTransformer[short#56483] OrcScan DataFilters: [isnotnull(short#56483)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#56483)] +- ^(2064) InputIteratorTransformer[short#56513] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=525865] +- ^(2063) FilterExecTransformer isnotnull(short#56513) +- ^(2063) BatchScanExecTransformer[short#56513] OrcScan DataFilters: [isnotnull(short#56513)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#56513)] +- == Initial Plan == Project [short#56483] +- BroadcastHashJoin [short#56483], [short#56513], Inner, BuildRight, false :- Filter isnotnull(short#56483) : +- BatchScan[short#56483] OrcScan DataFilters: [isnotnull(short#56483)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=525656] +- Filter isnotnull(short#56513) +- BatchScan[short#56513] OrcScan DataFilters: [isnotnull(short#56513)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#56482] +- Filter (byte#56482 = byte#56512) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#56482,short#56483,int#56484,long#56485L,float#56486,double#56487,decimal#56488,string#56489,binary#56490,bool#56491,date#56492,timestamp#56493,array#56494,struct#56495,map#56496]) : +- RelationV2[byte#56482, short#56483, int#56484, long#56485L, float#56486, double#56487, decimal#56488, string#56489, binary#56490, bool#56491, date#56492, timestamp#56493, array#56494, struct#56495, map#56496] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#56512,short#56513,int#56514,long#56515L,float#56516,double#56517,decimal#56518,string#56519,binary#56520,bool#56521,date#56522,timestamp#56523,array#56524,struct#56525,map#56526]) +- RelationV2[byte#56512, short#56513, int#56514, long#56515L, float#56516, double#56517, decimal#56518, string#56519, binary#56520, bool#56521, date#56522, timestamp#56523, array#56524, struct#56525, map#56526] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [byte#56482] +- Join Inner, (byte#56482 = byte#56512) :- Filter isnotnull(byte#56482) : +- RelationV2[byte#56482] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(byte#56512) +- RelationV2[byte#56512] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2102) ProjectExecTransformer [byte#56482] +- ^(2102) BroadcastHashJoinExecTransformer [byte#56482], [byte#56512], Inner, BuildRight, false :- ^(2102) FilterExecTransformer isnotnull(byte#56482) : +- ^(2102) BatchScanExecTransformer[byte#56482] OrcScan DataFilters: [isnotnull(byte#56482)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(byte#56482)] +- ^(2102) InputIteratorTransformer[byte#56512] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=529891] +- ^(2101) FilterExecTransformer isnotnull(byte#56512) +- ^(2101) BatchScanExecTransformer[byte#56512] OrcScan DataFilters: [isnotnull(byte#56512)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(byte#56512)] +- == Initial Plan == Project [byte#56482] +- BroadcastHashJoin [byte#56482], [byte#56512], Inner, BuildRight, false :- Filter isnotnull(byte#56482) : +- BatchScan[byte#56482] OrcScan DataFilters: [isnotnull(byte#56482)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=529682] +- Filter isnotnull(byte#56512) +- BatchScan[byte#56512] OrcScan DataFilters: [isnotnull(byte#56512)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxParquetDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#57878] +- Filter (int#57878 = int#57908) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#57876,short#57877,int#57878,long#57879L,float#57880,double#57881,decimal#57882,string#57883,binary#57884,bool#57885,date#57886,timestamp#57887,array#57888,struct#57889,map#57890]) : +- RelationV2[byte#57876, short#57877, int#57878, long#57879L, float#57880, double#57881, decimal#57882, string#57883, binary#57884, bool#57885, date#57886, timestamp#57887, array#57888, struct#57889, map#57890] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#57906,short#57907,int#57908,long#57909L,float#57910,double#57911,decimal#57912,string#57913,binary#57914,bool#57915,date#57916,timestamp#57917,array#57918,struct#57919,map#57920]) +- RelationV2[byte#57906, short#57907, int#57908, long#57909L, float#57910, double#57911, decimal#57912, string#57913, binary#57914, bool#57915, date#57916, timestamp#57917, array#57918, struct#57919, map#57920] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [int#57878] +- Join Inner, (int#57878 = int#57908) :- Filter isnotnull(int#57878) : +- RelationV2[int#57878] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(int#57908) +- RelationV2[int#57908] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2225) ProjectExecTransformer [int#57878] +- ^(2225) BroadcastHashJoinExecTransformer [int#57878], [int#57908], Inner, BuildRight, false :- ^(2225) FilterExecTransformer isnotnull(int#57878) : +- ^(2225) BatchScanExecTransformer[int#57878] ParquetScan DataFilters: [isnotnull(int#57878)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#57878)] +- ^(2225) InputIteratorTransformer[int#57908] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=542096] +- ^(2224) FilterExecTransformer isnotnull(int#57908) +- ^(2224) BatchScanExecTransformer[int#57908] ParquetScan DataFilters: [isnotnull(int#57908)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#57908)] +- == Initial Plan == Project [int#57878] +- BroadcastHashJoin [int#57878], [int#57908], Inner, BuildRight, false :- Filter isnotnull(int#57878) : +- BatchScan[int#57878] ParquetScan DataFilters: [isnotnull(int#57878)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=541887] +- Filter isnotnull(int#57908) +- BatchScan[int#57908] ParquetScan DataFilters: [isnotnull(int#57908)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxParquetDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#57877] +- Filter (short#57877 = short#57907) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#57876,short#57877,int#57878,long#57879L,float#57880,double#57881,decimal#57882,string#57883,binary#57884,bool#57885,date#57886,timestamp#57887,array#57888,struct#57889,map#57890]) : +- RelationV2[byte#57876, short#57877, int#57878, long#57879L, float#57880, double#57881, decimal#57882, string#57883, binary#57884, bool#57885, date#57886, timestamp#57887, array#57888, struct#57889, map#57890] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#57906,short#57907,int#57908,long#57909L,float#57910,double#57911,decimal#57912,string#57913,binary#57914,bool#57915,date#57916,timestamp#57917,array#57918,struct#57919,map#57920]) +- RelationV2[byte#57906, short#57907, int#57908, long#57909L, float#57910, double#57911, decimal#57912, string#57913, binary#57914, bool#57915, date#57916, timestamp#57917, array#57918, struct#57919, map#57920] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [short#57877] +- Join Inner, (short#57877 = short#57907) :- Filter isnotnull(short#57877) : +- RelationV2[short#57877] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(short#57907) +- RelationV2[short#57907] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2234) ProjectExecTransformer [short#57877] +- ^(2234) BroadcastHashJoinExecTransformer [short#57877], [short#57907], Inner, BuildRight, false :- ^(2234) FilterExecTransformer isnotnull(short#57877) : +- ^(2234) BatchScanExecTransformer[short#57877] ParquetScan DataFilters: [isnotnull(short#57877)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#57877)] +- ^(2234) InputIteratorTransformer[short#57907] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=543011] +- ^(2233) FilterExecTransformer isnotnull(short#57907) +- ^(2233) BatchScanExecTransformer[short#57907] ParquetScan DataFilters: [isnotnull(short#57907)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#57907)] +- == Initial Plan == Project [short#57877] +- BroadcastHashJoin [short#57877], [short#57907], Inner, BuildRight, false :- Filter isnotnull(short#57877) : +- BatchScan[short#57877] ParquetScan DataFilters: [isnotnull(short#57877)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=542802] +- Filter isnotnull(short#57907) +- BatchScan[short#57907] ParquetScan DataFilters: [isnotnull(short#57907)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxParquetDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L327
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#57876] +- Filter (byte#57876 = byte#57906) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#57876,short#57877,int#57878,long#57879L,float#57880,double#57881,decimal#57882,string#57883,binary#57884,bool#57885,date#57886,timestamp#57887,array#57888,struct#57889,map#57890]) : +- RelationV2[byte#57876, short#57877, int#57878, long#57879L, float#57880, double#57881, decimal#57882, string#57883, binary#57884, bool#57885, date#57886, timestamp#57887, array#57888, struct#57889, map#57890] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#57906,short#57907,int#57908,long#57909L,float#57910,double#57911,decimal#57912,string#57913,binary#57914,bool#57915,date#57916,timestamp#57917,array#57918,struct#57919,map#57920]) +- RelationV2[byte#57906, short#57907, int#57908, long#57909L, float#57910, double#57911, decimal#57912, string#57913, binary#57914, bool#57915, date#57916, timestamp#57917, array#57918, struct#57919, map#57920] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [byte#57876] +- Join Inner, (byte#57876 = byte#57906) :- Filter isnotnull(byte#57876) : +- RelationV2[byte#57876] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(byte#57906) +- RelationV2[byte#57906] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2276) ProjectExecTransformer [byte#57876] +- ^(2276) BroadcastHashJoinExecTransformer [byte#57876], [byte#57906], Inner, BuildRight, false :- ^(2276) FilterExecTransformer isnotnull(byte#57876) : +- ^(2276) BatchScanExecTransformer[byte#57876] ParquetScan DataFilters: [isnotnull(byte#57876)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(byte#57876)] +- ^(2276) InputIteratorTransformer[byte#57906] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=547425] +- ^(2275) FilterExecTransformer isnotnull(byte#57906) +- ^(2275) BatchScanExecTransformer[byte#57906] ParquetScan DataFilters: [isnotnull(byte#57906)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(byte#57906)] +- == Initial Plan == Project [byte#57876] +- BroadcastHashJoin [byte#57876], [byte#57906], Inner, BuildRight, false :- Filter isnotnull(byte#57876) : +- BatchScan[byte#57876] ParquetScan DataFilters: [isnotnull(byte#57876)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=547216] +- Filter isnotnull(byte#57906) +- BatchScan[byte#57906] ParquetScan DataFilters: [isnotnull(byte#57906)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxOrcDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L191
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#60958] +- Filter (int#60958 = int#60988) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#60956,short#60957,int#60958,long#60959L,float#60960,double#60961,decimal#60962,string#60963,binary#60964,bool#60965,date#60966,timestamp#60967,array#60968,struct#60969,map#60970]) : +- RelationV2[byte#60956, short#60957, int#60958, long#60959L, float#60960, double#60961, decimal#60962, string#60963, binary#60964, bool#60965, date#60966, timestamp#60967, array#60968, struct#60969, map#60970] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#60986,short#60987,int#60988,long#60989L,float#60990,double#60991,decimal#60992,string#60993,binary#60994,bool#60995,date#60996,timestamp#60997,array#60998,struct#60999,map#61000]) +- RelationV2[byte#60986, short#60987, int#60988, long#60989L, float#60990, double#60991, decimal#60992, string#60993, binary#60994, bool#60995, date#60996, timestamp#60997, array#60998, struct#60999, map#61000] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [int#60958] +- Join Inner, (int#60958 = int#60988) :- Filter isnotnull(int#60958) : +- RelationV2[int#60958] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(int#60988) +- RelationV2[int#60988] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2157) ProjectExecTransformer [int#60958] +- ^(2157) BroadcastHashJoinExecTransformer [int#60958], [int#60988], Inner, BuildRight, false :- ^(2157) FilterExecTransformer isnotnull(int#60958) : +- ^(2157) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#60958] OrcScan DataFilters: [isnotnull(int#60958)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#60958)] +- ^(2157) InputIteratorTransformer[int#60988] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=526343] +- ^(2156) FilterExecTransformer isnotnull(int#60988) +- ^(2156) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#60988] OrcScan DataFilters: [isnotnull(int#60988)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#60988)] +- == Initial Plan == Project [int#60958] +- BroadcastHashJoin [int#60958], [int#60988], Inner, BuildRight, false :- Filter isnotnull(int#60958) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#60958] OrcScan DataFilters: [isnotnull(int#60958)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=526134] +- Filter isnotnull(int#60988) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#60988] OrcScan DataFilters: [isnotnull(int#60988)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxOrcDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#60957] +- Filter (short#60957 = short#60987) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#60956,short#60957,int#60958,long#60959L,float#60960,double#60961,decimal#60962,string#60963,binary#60964,bool#60965,date#60966,timestamp#60967,array#60968,struct#60969,map#60970]) : +- RelationV2[byte#60956, short#60957, int#60958, long#60959L, float#60960, double#60961, decimal#60962, string#60963, binary#60964, bool#60965, date#60966, timestamp#60967, array#60968, struct#60969, map#60970] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#60986,short#60987,int#60988,long#60989L,float#60990,double#60991,decimal#60992,string#60993,binary#60994,bool#60995,date#60996,timestamp#60997,array#60998,struct#60999,map#61000]) +- RelationV2[byte#60986, short#60987, int#60988, long#60989L, float#60990, double#60991, decimal#60992, string#60993, binary#60994, bool#60995, date#60996, timestamp#60997, array#60998, struct#60999, map#61000] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [short#60957] +- Join Inner, (short#60957 = short#60987) :- Filter isnotnull(short#60957) : +- RelationV2[short#60957] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(short#60987) +- RelationV2[short#60987] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2166) ProjectExecTransformer [short#60957] +- ^(2166) BroadcastHashJoinExecTransformer [short#60957], [short#60987], Inner, BuildRight, false :- ^(2166) FilterExecTransformer isnotnull(short#60957) : +- ^(2166) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#60957] OrcScan DataFilters: [isnotnull(short#60957)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#60957)] +- ^(2166) InputIteratorTransformer[short#60987] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=527258] +- ^(2165) FilterExecTransformer isnotnull(short#60987) +- ^(2165) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#60987] OrcScan DataFilters: [isnotnull(short#60987)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#60987)] +- == Initial Plan == Project [short#60957] +- BroadcastHashJoin [short#60957], [short#60987], Inner, BuildRight, false :- Filter isnotnull(short#60957) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#60957] OrcScan DataFilters: [isnotnull(short#60957)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=527049] +- Filter isnotnull(short#60987) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#60987] OrcScan DataFilters: [isnotnull(short#60987)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#60956] +- Filter (byte#60956 = byte#60986) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#60956,short#60957,int#60958,long#60959L,float#60960,double#60961,decimal#60962,string#60963,binary#60964,bool#60965,date#60966,timestamp#60967,array#60968,struct#60969,map#60970]) : +- RelationV2[byte#60956, short#60957, int#60958, long#60959L, float#60960, double#60961, decimal#60962, string#60963, binary#60964, bool#60965, date#60966, timestamp#60967, array#60968, struct#60969, map#60970] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#60986,short#60987,int#60988,long#60989L,float#60990,double#60991,decimal#60992,string#60993,binary#60994,bool#60995,date#60996,timestamp#60997,array#60998,struct#60999,map#61000]) +- RelationV2[byte#60986, short#60987, int#60988, long#60989L, float#60990, double#60991, decimal#60992, string#60993, binary#60994, bool#60995, date#60996, timestamp#60997, array#60998, struct#60999, map#61000] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [byte#60956] +- Join Inner, (byte#60956 = byte#60986) :- Filter isnotnull(byte#60956) : +- RelationV2[byte#60956] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(byte#60986) +- RelationV2[byte#60986] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2204) ProjectExecTransformer [byte#60956] +- ^(2204) BroadcastHashJoinExecTransformer [byte#60956], [byte#60986], Inner, BuildRight, false :- ^(2204) FilterExecTransformer isnotnull(byte#60956) : +- ^(2204) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#60956] OrcScan DataFilters: [isnotnull(byte#60956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#60956)] +- ^(2204) InputIteratorTransformer[byte#60986] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=531284] +- ^(2203) FilterExecTransformer isnotnull(byte#60986) +- ^(2203) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#60986] OrcScan DataFilters: [isnotnull(byte#60986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#60986)] +- == Initial Plan == Project [byte#60956] +- BroadcastHashJoin [byte#60956], [byte#60986], Inner, BuildRight, false :- Filter isnotnull(byte#60956) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#60956] OrcScan DataFilters: [isnotnull(byte#60956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=531075] +- Filter isnotnull(byte#60986) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#60986] OrcScan DataFilters: [isnotnull(byte#60986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxParquetDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L190
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#64523] +- Filter (int#64523 = int#64553) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#64521,short#64522,int#64523,long#64524L,float#64525,double#64526,decimal#64527,string#64528,binary#64529,bool#64530,date#64531,timestamp#64532,array#64533,struct#64534,map#64535]) : +- RelationV2[byte#64521, short#64522, int#64523, long#64524L, float#64525, double#64526, decimal#64527, string#64528, binary#64529, bool#64530, date#64531, timestamp#64532, array#64533, struct#64534, map#64535] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#64551,short#64552,int#64553,long#64554L,float#64555,double#64556,decimal#64557,string#64558,binary#64559,bool#64560,date#64561,timestamp#64562,array#64563,struct#64564,map#64565]) +- RelationV2[byte#64551, short#64552, int#64553, long#64554L, float#64555, double#64556, decimal#64557, string#64558, binary#64559, bool#64560, date#64561, timestamp#64562, array#64563, struct#64564, map#64565] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [int#64523] +- Join Inner, (int#64523 = int#64553) :- Filter isnotnull(int#64523) : +- RelationV2[int#64523] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(int#64553) +- RelationV2[int#64553] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2331) ProjectExecTransformer [int#64523] +- ^(2331) BroadcastHashJoinExecTransformer [int#64523], [int#64553], Inner, BuildRight, false :- ^(2331) FilterExecTransformer isnotnull(int#64523) : +- ^(2331) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#64523] ParquetScan DataFilters: [isnotnull(int#64523)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#64523)] +- ^(2331) InputIteratorTransformer[int#64553] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=544108] +- ^(2330) FilterExecTransformer isnotnull(int#64553) +- ^(2330) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#64553] ParquetScan DataFilters: [isnotnull(int#64553)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#64553)] +- == Initial Plan == Project [int#64523] +- BroadcastHashJoin [int#64523], [int#64553], Inner, BuildRight, false :- Filter isnotnull(int#64523) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#64523] ParquetScan DataFilters: [isnotnull(int#64523)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=543899] +- Filter isnotnull(int#64553) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#64553] ParquetScan DataFilters: [isnotnull(int#64553)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxParquetDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#64522] +- Filter (short#64522 = short#64552) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#64521,short#64522,int#64523,long#64524L,float#64525,double#64526,decimal#64527,string#64528,binary#64529,bool#64530,date#64531,timestamp#64532,array#64533,struct#64534,map#64535]) : +- RelationV2[byte#64521, short#64522, int#64523, long#64524L, float#64525, double#64526, decimal#64527, string#64528, binary#64529, bool#64530, date#64531, timestamp#64532, array#64533, struct#64534, map#64535] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#64551,short#64552,int#64553,long#64554L,float#64555,double#64556,decimal#64557,string#64558,binary#64559,bool#64560,date#64561,timestamp#64562,array#64563,struct#64564,map#64565]) +- RelationV2[byte#64551, short#64552, int#64553, long#64554L, float#64555, double#64556, decimal#64557, string#64558, binary#64559, bool#64560, date#64561, timestamp#64562, array#64563, struct#64564, map#64565] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [short#64522] +- Join Inner, (short#64522 = short#64552) :- Filter isnotnull(short#64522) : +- RelationV2[short#64522] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(short#64552) +- RelationV2[short#64552] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2340) ProjectExecTransformer [short#64522] +- ^(2340) BroadcastHashJoinExecTransformer [short#64522], [short#64552], Inner, BuildRight, false :- ^(2340) FilterExecTransformer isnotnull(short#64522) : +- ^(2340) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#64522] ParquetScan DataFilters: [isnotnull(short#64522)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#64522)] +- ^(2340) InputIteratorTransformer[short#64552] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=545023] +- ^(2339) FilterExecTransformer isnotnull(short#64552) +- ^(2339) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#64552] ParquetScan DataFilters: [isnotnull(short#64552)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#64552)] +- == Initial Plan == Project [short#64522] +- BroadcastHashJoin [short#64522], [short#64552], Inner, BuildRight, false :- Filter isnotnull(short#64522) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#64522] ParquetScan DataFilters: [isnotnull(short#64522)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=544814] +- Filter isnotnull(short#64552) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#64552] ParquetScan DataFilters: [isnotnull(short#64552)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxParquetDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#64521] +- Filter (byte#64521 = byte#64551) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#64521,short#64522,int#64523,long#64524L,float#64525,double#64526,decimal#64527,string#64528,binary#64529,bool#64530,date#64531,timestamp#64532,array#64533,struct#64534,map#64535]) : +- RelationV2[byte#64521, short#64522, int#64523, long#64524L, float#64525, double#64526, decimal#64527, string#64528, binary#64529, bool#64530, date#64531, timestamp#64532, array#64533, struct#64534, map#64535] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#64551,short#64552,int#64553,long#64554L,float#64555,double#64556,decimal#64557,string#64558,binary#64559,bool#64560,date#64561,timestamp#64562,array#64563,struct#64564,map#64565]) +- RelationV2[byte#64551, short#64552, int#64553, long#64554L, float#64555, double#64556, decimal#64557, string#64558, binary#64559, bool#64560, date#64561, timestamp#64562, array#64563, struct#64564, map#64565] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [byte#64521] +- Join Inner, (byte#64521 = byte#64551) :- Filter isnotnull(byte#64521) : +- RelationV2[byte#64521] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(byte#64551) +- RelationV2[byte#64551] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2378) ProjectExecTransformer [byte#64521] +- ^(2378) BroadcastHashJoinExecTransformer [byte#64521], [byte#64551], Inner, BuildRight, false :- ^(2378) FilterExecTransformer isnotnull(byte#64521) : +- ^(2378) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#64521] ParquetScan DataFilters: [isnotnull(byte#64521)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#64521)] +- ^(2378) InputIteratorTransformer[byte#64551] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=549049] +- ^(2377) FilterExecTransformer isnotnull(byte#64551) +- ^(2377) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#64551] ParquetScan DataFilters: [isnotnull(byte#64551)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#64551)] +- == Initial Plan == Project [byte#64521] +- BroadcastHashJoin [byte#64521], [byte#64551], Inner, BuildRight, false :- Filter isnotnull(byte#64521) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#64521] ParquetScan DataFilters: [isnotnull(byte#64521)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=548840] +- Filter isnotnull(byte#64551) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#64551] ParquetScan DataFilters: [isnotnull(byte#64551)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxOrcDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#73724] +- Filter (short#73724 = short#73754) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73723,short#73724,int#73725,long#73726L,float#73727,double#73728,decimal#73729,string#73730,binary#73731,bool#73732,date#73733,timestamp#73734,array#73735,struct#73736,map#73737]) : +- RelationV2[byte#73723, short#73724, int#73725, long#73726L, float#73727, double#73728, decimal#73729, string#73730, binary#73731, bool#73732, date#73733, timestamp#73734, array#73735, struct#73736, map#73737] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73753,short#73754,int#73755,long#73756L,float#73757,double#73758,decimal#73759,string#73760,binary#73761,bool#73762,date#73763,timestamp#73764,array#73765,struct#73766,map#73767]) +- RelationV2[byte#73753, short#73754, int#73755, long#73756L, float#73757, double#73758, decimal#73759, string#73760, binary#73761, bool#73762, date#73763, timestamp#73764, array#73765, struct#73766, map#73767] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [short#73724] +- Join Inner, (short#73724 = short#73754) :- Filter isnotnull(short#73724) : +- RelationV2[short#73724] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(short#73754) +- RelationV2[short#73754] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2390) ProjectExecTransformer [short#73724] +- ^(2390) BroadcastHashJoinExecTransformer [short#73724], [short#73754], Inner, BuildRight, false :- ^(2390) FilterExecTransformer isnotnull(short#73724) : +- ^(2390) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73724] OrcScan DataFilters: [isnotnull(short#73724)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73724)] +- ^(2390) InputIteratorTransformer[short#73754] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=550160] +- ^(2389) FilterExecTransformer isnotnull(short#73754) +- ^(2389) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73754] OrcScan DataFilters: [isnotnull(short#73754)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73754)] +- == Initial Plan == Project [short#73724] +- BroadcastHashJoin [short#73724], [short#73754], Inner, BuildRight, false :- Filter isnotnull(short#73724) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73724] OrcScan DataFilters: [isnotnull(short#73724)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=549951] +- Filter isnotnull(short#73754) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73754] OrcScan DataFilters: [isnotnull(short#73754)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxParquetDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#77599] +- Filter (int#77599 = int#77629) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#77597,short#77598,int#77599,long#77600L,float#77601,double#77602,decimal#77603,string#77604,binary#77605,bool#77606,date#77607,timestamp#77608,array#77609,struct#77610,map#77611]) : +- RelationV2[byte#77597, short#77598, int#77599, long#77600L, float#77601, double#77602, decimal#77603, string#77604, binary#77605, bool#77606, date#77607, timestamp#77608, array#77609, struct#77610, map#77611] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#77627,short#77628,int#77629,long#77630L,float#77631,double#77632,decimal#77633,string#77634,binary#77635,bool#77636,date#77637,timestamp#77638,array#77639,struct#77640,map#77641]) +- RelationV2[byte#77627, short#77628, int#77629, long#77630L, float#77631, double#77632, decimal#77633, string#77634, binary#77635, bool#77636, date#77637, timestamp#77638, array#77639, struct#77640, map#77641] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [int#77599] +- Join Inner, (int#77599 = int#77629) :- Filter isnotnull(int#77599) : +- RelationV2[int#77599] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(int#77629) +- RelationV2[int#77629] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2569) ProjectExecTransformer [int#77599] +- ^(2569) BroadcastHashJoinExecTransformer [int#77599], [int#77629], Inner, BuildRight, false :- ^(2569) FilterExecTransformer isnotnull(int#77599) : +- ^(2569) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77599] ParquetScan DataFilters: [isnotnull(int#77599)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77599)] +- ^(2569) InputIteratorTransformer[int#77629] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=567762] +- ^(2568) FilterExecTransformer isnotnull(int#77629) +- ^(2568) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77629] ParquetScan DataFilters: [isnotnull(int#77629)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77629)] +- == Initial Plan == Project [int#77599] +- BroadcastHashJoin [int#77599], [int#77629], Inner, BuildRight, false :- Filter isnotnull(int#77599) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77599] ParquetScan DataFilters: [isnotnull(int#77599)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=567553] +- Filter isnotnull(int#77629) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77629] ParquetScan DataFilters: [isnotnull(int#77629)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxParquetDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#77598] +- Filter (short#77598 = short#77628) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#77597,short#77598,int#77599,long#77600L,float#77601,double#77602,decimal#77603,string#77604,binary#77605,bool#77606,date#77607,timestamp#77608,array#77609,struct#77610,map#77611]) : +- RelationV2[byte#77597, short#77598, int#77599, long#77600L, float#77601, double#77602, decimal#77603, string#77604, binary#77605, bool#77606, date#77607, timestamp#77608, array#77609, struct#77610, map#77611] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#77627,short#77628,int#77629,long#77630L,float#77631,double#77632,decimal#77633,string#77634,binary#77635,bool#77636,date#77637,timestamp#77638,array#77639,struct#77640,map#77641]) +- RelationV2[byte#77627, short#77628, int#77629, long#77630L, float#77631, double#77632, decimal#77633, string#77634, binary#77635, bool#77636, date#77637, timestamp#77638, array#77639, struct#77640, map#77641] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [short#77598] +- Join Inner, (short#77598 = short#77628) :- Filter isnotnull(short#77598) : +- RelationV2[short#77598] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(short#77628) +- RelationV2[short#77628] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2578) ProjectExecTransformer [short#77598] +- ^(2578) BroadcastHashJoinExecTransformer [short#77598], [short#77628], Inner, BuildRight, false :- ^(2578) FilterExecTransformer isnotnull(short#77598) : +- ^(2578) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77598] ParquetScan DataFilters: [isnotnull(short#77598)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77598)] +- ^(2578) InputIteratorTransformer[short#77628] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=568677] +- ^(2577) FilterExecTransformer isnotnull(short#77628) +- ^(2577) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77628] ParquetScan DataFilters: [isnotnull(short#77628)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77628)] +- == Initial Plan == Project [short#77598] +- BroadcastHashJoin [short#77598], [short#77628], Inner, BuildRight, false :- Filter isnotnull(short#77598) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77598] ParquetScan DataFilters: [isnotnull(short#77598)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=568468] +- Filter isnotnull(short#77628) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77628] ParquetScan DataFilters: [isnotnull(short#77628)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxParquetDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#77597] +- Filter (byte#77597 = byte#77627) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#77597,short#77598,int#77599,long#77600L,float#77601,double#77602,decimal#77603,string#77604,binary#77605,bool#77606,date#77607,timestamp#77608,array#77609,struct#77610,map#77611]) : +- RelationV2[byte#77597, short#77598, int#77599, long#77600L, float#77601, double#77602, decimal#77603, string#77604, binary#77605, bool#77606, date#77607, timestamp#77608, array#77609, struct#77610, map#77611] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#77627,short#77628,int#77629,long#77630L,float#77631,double#77632,decimal#77633,string#77634,binary#77635,bool#77636,date#77637,timestamp#77638,array#77639,struct#77640,map#77641]) +- RelationV2[byte#77627, short#77628, int#77629, long#77630L, float#77631, double#77632, decimal#77633, string#77634, binary#77635, bool#77636, date#77637, timestamp#77638, array#77639, struct#77640, map#77641] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [byte#77597] +- Join Inner, (byte#77597 = byte#77627) :- Filter isnotnull(byte#77597) : +- RelationV2[byte#77597] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(byte#77627) +- RelationV2[byte#77627] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2628) ProjectExecTransformer [byte#77597] +- ^(2628) BroadcastHashJoinExecTransformer [byte#77597], [byte#77627], Inner, BuildRight, false :- ^(2628) FilterExecTransformer isnotnull(byte#77597) : +- ^(2628) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77597] ParquetScan DataFilters: [isnotnull(byte#77597)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77597)] +- ^(2628) InputIteratorTransformer[byte#77627] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=573327] +- ^(2627) FilterExecTransformer isnotnull(byte#77627) +- ^(2627) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77627] ParquetScan DataFilters: [isnotnull(byte#77627)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77627)] +- == Initial Plan == Project [byte#77597] +- BroadcastHashJoin [byte#77597], [byte#77627], Inner, BuildRight, false :- Filter isnotnull(byte#77597) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77597] ParquetScan DataFilters: [isnotnull(byte#77597)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=573118] +- Filter isnotnull(byte#77627) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77627] ParquetScan DataFilters: [isnotnull(byte#77627)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxOrcDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L185
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#73704] +- Filter (int#73704 = int#73734) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73702,short#73703,int#73704,long#73705L,float#73706,double#73707,decimal#73708,string#73709,binary#73710,bool#73711,date#73712,timestamp#73713,array#73714,struct#73715,map#73716]) : +- RelationV2[byte#73702, short#73703, int#73704, long#73705L, float#73706, double#73707, decimal#73708, string#73709, binary#73710, bool#73711, date#73712, timestamp#73713, array#73714, struct#73715, map#73716] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73732,short#73733,int#73734,long#73735L,float#73736,double#73737,decimal#73738,string#73739,binary#73740,bool#73741,date#73742,timestamp#73743,array#73744,struct#73745,map#73746]) +- RelationV2[byte#73732, short#73733, int#73734, long#73735L, float#73736, double#73737, decimal#73738, string#73739, binary#73740, bool#73741, date#73742, timestamp#73743, array#73744, struct#73745, map#73746] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [int#73704] +- Join Inner, (int#73704 = int#73734) :- Filter isnotnull(int#73704) : +- RelationV2[int#73704] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(int#73734) +- RelationV2[int#73734] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2377) ProjectExecTransformer [int#73704] +- ^(2377) BroadcastHashJoinExecTransformer [int#73704], [int#73734], Inner, BuildRight, false :- ^(2377) FilterExecTransformer isnotnull(int#73704) : +- ^(2377) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#73704] OrcScan DataFilters: [isnotnull(int#73704)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73704)] +- ^(2377) InputIteratorTransformer[int#73734] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=542836] +- ^(2376) FilterExecTransformer isnotnull(int#73734) +- ^(2376) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#73734] OrcScan DataFilters: [isnotnull(int#73734)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73734)] +- == Initial Plan == Project [int#73704] +- BroadcastHashJoin [int#73704], [int#73734], Inner, BuildRight, false :- Filter isnotnull(int#73704) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#73704] OrcScan DataFilters: [isnotnull(int#73704)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=542627] +- Filter isnotnull(int#73734) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#73734] OrcScan DataFilters: [isnotnull(int#73734)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxOrcDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#73703] +- Filter (short#73703 = short#73733) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73702,short#73703,int#73704,long#73705L,float#73706,double#73707,decimal#73708,string#73709,binary#73710,bool#73711,date#73712,timestamp#73713,array#73714,struct#73715,map#73716]) : +- RelationV2[byte#73702, short#73703, int#73704, long#73705L, float#73706, double#73707, decimal#73708, string#73709, binary#73710, bool#73711, date#73712, timestamp#73713, array#73714, struct#73715, map#73716] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73732,short#73733,int#73734,long#73735L,float#73736,double#73737,decimal#73738,string#73739,binary#73740,bool#73741,date#73742,timestamp#73743,array#73744,struct#73745,map#73746]) +- RelationV2[byte#73732, short#73733, int#73734, long#73735L, float#73736, double#73737, decimal#73738, string#73739, binary#73740, bool#73741, date#73742, timestamp#73743, array#73744, struct#73745, map#73746] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [short#73703] +- Join Inner, (short#73703 = short#73733) :- Filter isnotnull(short#73703) : +- RelationV2[short#73703] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(short#73733) +- RelationV2[short#73733] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2386) ProjectExecTransformer [short#73703] +- ^(2386) BroadcastHashJoinExecTransformer [short#73703], [short#73733], Inner, BuildRight, false :- ^(2386) FilterExecTransformer isnotnull(short#73703) : +- ^(2386) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73703] OrcScan DataFilters: [isnotnull(short#73703)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73703)] +- ^(2386) InputIteratorTransformer[short#73733] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=543751] +- ^(2385) FilterExecTransformer isnotnull(short#73733) +- ^(2385) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73733] OrcScan DataFilters: [isnotnull(short#73733)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73733)] +- == Initial Plan == Project [short#73703] +- BroadcastHashJoin [short#73703], [short#73733], Inner, BuildRight, false :- Filter isnotnull(short#73703) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73703] OrcScan DataFilters: [isnotnull(short#73703)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=543542] +- Filter isnotnull(short#73733) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73733] OrcScan DataFilters: [isnotnull(short#73733)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#73702] +- Filter (byte#73702 = byte#73732) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73702,short#73703,int#73704,long#73705L,float#73706,double#73707,decimal#73708,string#73709,binary#73710,bool#73711,date#73712,timestamp#73713,array#73714,struct#73715,map#73716]) : +- RelationV2[byte#73702, short#73703, int#73704, long#73705L, float#73706, double#73707, decimal#73708, string#73709, binary#73710, bool#73711, date#73712, timestamp#73713, array#73714, struct#73715, map#73716] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73732,short#73733,int#73734,long#73735L,float#73736,double#73737,decimal#73738,string#73739,binary#73740,bool#73741,date#73742,timestamp#73743,array#73744,struct#73745,map#73746]) +- RelationV2[byte#73732, short#73733, int#73734, long#73735L, float#73736, double#73737, decimal#73738, string#73739, binary#73740, bool#73741, date#73742, timestamp#73743, array#73744, struct#73745, map#73746] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [byte#73702] +- Join Inner, (byte#73702 = byte#73732) :- Filter isnotnull(byte#73702) : +- RelationV2[byte#73702] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(byte#73732) +- RelationV2[byte#73732] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2436) ProjectExecTransformer [byte#73702] +- ^(2436) BroadcastHashJoinExecTransformer [byte#73702], [byte#73732], Inner, BuildRight, false :- ^(2436) FilterExecTransformer isnotnull(byte#73702) : +- ^(2436) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73702] OrcScan DataFilters: [isnotnull(byte#73702)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73702)] +- ^(2436) InputIteratorTransformer[byte#73732] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=548401] +- ^(2435) FilterExecTransformer isnotnull(byte#73732) +- ^(2435) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73732] OrcScan DataFilters: [isnotnull(byte#73732)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73732)] +- == Initial Plan == Project [byte#73702] +- BroadcastHashJoin [byte#73702], [byte#73732], Inner, BuildRight, false :- Filter isnotnull(byte#73702) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73702] OrcScan DataFilters: [isnotnull(byte#73702)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=548192] +- Filter isnotnull(byte#73732) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73732] OrcScan DataFilters: [isnotnull(byte#73732)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxParquetDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#77367] +- Filter (short#77367 = short#77397) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#77366,short#77367,int#77368,long#77369L,float#77370,double#77371,decimal#77372,string#77373,binary#77374,bool#77375,date#77376,timestamp#77377,array#77378,struct#77379,map#77380]) : +- RelationV2[byte#77366, short#77367, int#77368, long#77369L, float#77370, double#77371, decimal#77372, string#77373, binary#77374, bool#77375, date#77376, timestamp#77377, array#77378, struct#77379, map#77380] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#77396,short#77397,int#77398,long#77399L,float#77400,double#77401,decimal#77402,string#77403,binary#77404,bool#77405,date#77406,timestamp#77407,array#77408,struct#77409,map#77410]) +- RelationV2[byte#77396, short#77397, int#77398, long#77399L, float#77400, double#77401, decimal#77402, string#77403, binary#77404, bool#77405, date#77406, timestamp#77407, array#77408, struct#77409, map#77410] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [short#77367] +- Join Inner, (short#77367 = short#77397) :- Filter isnotnull(short#77367) : +- RelationV2[short#77367] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(short#77397) +- RelationV2[short#77397] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2574) ProjectExecTransformer [short#77367] +- ^(2574) BroadcastHashJoinExecTransformer [short#77367], [short#77397], Inner, BuildRight, false :- ^(2574) FilterExecTransformer isnotnull(short#77367) : +- ^(2574) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77367] ParquetScan DataFilters: [isnotnull(short#77367)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77367)] +- ^(2574) InputIteratorTransformer[short#77397] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=562268] +- ^(2573) FilterExecTransformer isnotnull(short#77397) +- ^(2573) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77397] ParquetScan DataFilters: [isnotnull(short#77397)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77397)] +- == Initial Plan == Project [short#77367] +- BroadcastHashJoin [short#77367], [short#77397], Inner, BuildRight, false :- Filter isnotnull(short#77367) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77367] ParquetScan DataFilters: [isnotnull(short#77367)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=562059] +- Filter isnotnull(short#77397) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77397] ParquetScan DataFilters: [isnotnull(short#77397)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L185
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#73137] +- Filter (int#73137 = int#73167) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73135,short#73136,int#73137,long#73138L,float#73139,double#73140,decimal#73141,string#73142,binary#73143,bool#73144,date#73145,timestamp#73146,array#73147,struct#73148,map#73149]) : +- RelationV2[byte#73135, short#73136, int#73137, long#73138L, float#73139, double#73140, decimal#73141, string#73142, binary#73143, bool#73144, date#73145, timestamp#73146, array#73147, struct#73148, map#73149] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73165,short#73166,int#73167,long#73168L,float#73169,double#73170,decimal#73171,string#73172,binary#73173,bool#73174,date#73175,timestamp#73176,array#73177,struct#73178,map#73179]) +- RelationV2[byte#73165, short#73166, int#73167, long#73168L, float#73169, double#73170, decimal#73171, string#73172, binary#73173, bool#73174, date#73175, timestamp#73176, array#73177, struct#73178, map#73179] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [int#73137] +- Join Inner, (int#73137 = int#73167) :- Filter isnotnull(int#73137) : +- RelationV2[int#73137] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(int#73167) +- RelationV2[int#73167] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2584) ProjectExecTransformer [int#73137] +- ^(2584) BroadcastHashJoinExecTransformer [int#73137], [int#73167], Inner, BuildRight, false :- ^(2584) FilterExecTransformer isnotnull(int#73137) : +- ^(2584) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[int#73137] OrcScan DataFilters: [isnotnull(int#73137)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73137)] +- ^(2584) InputIteratorTransformer[int#73167] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=366602] +- ^(2583) FilterExecTransformer isnotnull(int#73167) +- ^(2583) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[int#73167] OrcScan DataFilters: [isnotnull(int#73167)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73167)] +- == Initial Plan == Project [int#73137] +- BroadcastHashJoin [int#73137], [int#73167], Inner, BuildRight, false :- Filter isnotnull(int#73137) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[int#73137] OrcScan DataFilters: [isnotnull(int#73137)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=366393] +- Filter isnotnull(int#73167) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[int#73167] OrcScan DataFilters: [isnotnull(int#73167)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxOrcDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#73136] +- Filter (short#73136 = short#73166) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73135,short#73136,int#73137,long#73138L,float#73139,double#73140,decimal#73141,string#73142,binary#73143,bool#73144,date#73145,timestamp#73146,array#73147,struct#73148,map#73149]) : +- RelationV2[byte#73135, short#73136, int#73137, long#73138L, float#73139, double#73140, decimal#73141, string#73142, binary#73143, bool#73144, date#73145, timestamp#73146, array#73147, struct#73148, map#73149] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73165,short#73166,int#73167,long#73168L,float#73169,double#73170,decimal#73171,string#73172,binary#73173,bool#73174,date#73175,timestamp#73176,array#73177,struct#73178,map#73179]) +- RelationV2[byte#73165, short#73166, int#73167, long#73168L, float#73169, double#73170, decimal#73171, string#73172, binary#73173, bool#73174, date#73175, timestamp#73176, array#73177, struct#73178, map#73179] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [short#73136] +- Join Inner, (short#73136 = short#73166) :- Filter isnotnull(short#73136) : +- RelationV2[short#73136] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(short#73166) +- RelationV2[short#73166] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2593) ProjectExecTransformer [short#73136] +- ^(2593) BroadcastHashJoinExecTransformer [short#73136], [short#73166], Inner, BuildRight, false :- ^(2593) FilterExecTransformer isnotnull(short#73136) : +- ^(2593) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[short#73136] OrcScan DataFilters: [isnotnull(short#73136)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73136)] +- ^(2593) InputIteratorTransformer[short#73166] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=367517] +- ^(2592) FilterExecTransformer isnotnull(short#73166) +- ^(2592) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[short#73166] OrcScan DataFilters: [isnotnull(short#73166)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73166)] +- == Initial Plan == Project [short#73136] +- BroadcastHashJoin [short#73136], [short#73166], Inner, BuildRight, false :- Filter isnotnull(short#73136) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[short#73136] OrcScan DataFilters: [isnotnull(short#73136)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=367308] +- Filter isnotnull(short#73166) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[short#73166] OrcScan DataFilters: [isnotnull(short#73166)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L328
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#73135] +- Filter (byte#73135 = byte#73165) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73135,short#73136,int#73137,long#73138L,float#73139,double#73140,decimal#73141,string#73142,binary#73143,bool#73144,date#73145,timestamp#73146,array#73147,struct#73148,map#73149]) : +- RelationV2[byte#73135, short#73136, int#73137, long#73138L, float#73139, double#73140, decimal#73141, string#73142, binary#73143, bool#73144, date#73145, timestamp#73146, array#73147, struct#73148, map#73149] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73165,short#73166,int#73167,long#73168L,float#73169,double#73170,decimal#73171,string#73172,binary#73173,bool#73174,date#73175,timestamp#73176,array#73177,struct#73178,map#73179]) +- RelationV2[byte#73165, short#73166, int#73167, long#73168L, float#73169, double#73170, decimal#73171, string#73172, binary#73173, bool#73174, date#73175, timestamp#73176, array#73177, struct#73178, map#73179] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [byte#73135] +- Join Inner, (byte#73135 = byte#73165) :- Filter isnotnull(byte#73135) : +- RelationV2[byte#73135] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(byte#73165) +- RelationV2[byte#73165] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2647) ProjectExecTransformer [byte#73135] +- ^(2647) BroadcastHashJoinExecTransformer [byte#73135], [byte#73165], Inner, BuildRight, false :- ^(2647) FilterExecTransformer isnotnull(byte#73135) : +- ^(2647) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[byte#73135] OrcScan DataFilters: [isnotnull(byte#73135)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73135)] +- ^(2647) InputIteratorTransformer[byte#73165] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=372555] +- ^(2646) FilterExecTransformer isnotnull(byte#73165) +- ^(2646) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[byte#73165] OrcScan DataFilters: [isnotnull(byte#73165)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73165)] +- == Initial Plan == Project [byte#73135] +- BroadcastHashJoin [byte#73135], [byte#73165], Inner, BuildRight, false :- Filter isnotnull(byte#73135) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[byte#73135] OrcScan DataFilters: [isnotnull(byte#73135)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=372346] +- Filter isnotnull(byte#73165) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[byte#73165] OrcScan DataFilters: [isnotnull(byte#73165)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxParquetDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#76906] +- Filter (int#76906 = int#76936) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#76904,short#76905,int#76906,long#76907L,float#76908,double#76909,decimal#76910,string#76911,binary#76912,bool#76913,date#76914,timestamp#76915,array#76916,struct#76917,map#76918]) : +- RelationV2[byte#76904, short#76905, int#76906, long#76907L, float#76908, double#76909, decimal#76910, string#76911, binary#76912, bool#76913, date#76914, timestamp#76915, array#76916, struct#76917, map#76918] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#76934,short#76935,int#76936,long#76937L,float#76938,double#76939,decimal#76940,string#76941,binary#76942,bool#76943,date#76944,timestamp#76945,array#76946,struct#76947,map#76948]) +- RelationV2[byte#76934, short#76935, int#76936, long#76937L, float#76938, double#76939, decimal#76940, string#76941, binary#76942, bool#76943, date#76944, timestamp#76945, array#76946, struct#76947, map#76948] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [int#76906] +- Join Inner, (int#76906 = int#76936) :- Filter isnotnull(int#76906) : +- RelationV2[int#76906] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1 +- Filter isnotnull(int#76936) +- RelationV2[int#76936] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2772) ProjectExecTransformer [int#76906] +- ^(2772) BroadcastHashJoinExecTransformer [int#76906], [int#76936], Inner, BuildRight, false :- ^(2772) FilterExecTransformer isnotnull(int#76906) : +- ^(2772) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[int#76906] ParquetScan DataFilters: [isnotnull(int#76906)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#76906)] +- ^(2772) InputIteratorTransformer[int#76936] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=385119] +- ^(2771) FilterExecTransformer isnotnull(int#76936) +- ^(2771) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[int#76936] ParquetScan DataFilters: [isnotnull(int#76936)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#76936)] +- == Initial Plan == Project [int#76906] +- BroadcastHashJoin [int#76906], [int#76936], Inner, BuildRight, false :- Filter isnotnull(int#76906) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[int#76906] ParquetScan DataFilters: [isnotnull(int#76906)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=384910] +- Filter isnotnull(int#76936) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[int#76936] ParquetScan DataFilters: [isnotnull(int#76936)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxParquetDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#76905] +- Filter (short#76905 = short#76935) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#76904,short#76905,int#76906,long#76907L,float#76908,double#76909,decimal#76910,string#76911,binary#76912,bool#76913,date#76914,timestamp#76915,array#76916,struct#76917,map#76918]) : +- RelationV2[byte#76904, short#76905, int#76906, long#76907L, float#76908, double#76909, decimal#76910, string#76911, binary#76912, bool#76913, date#76914, timestamp#76915, array#76916, struct#76917, map#76918] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#76934,short#76935,int#76936,long#76937L,float#76938,double#76939,decimal#76940,string#76941,binary#76942,bool#76943,date#76944,timestamp#76945,array#76946,struct#76947,map#76948]) +- RelationV2[byte#76934, short#76935, int#76936, long#76937L, float#76938, double#76939, decimal#76940, string#76941, binary#76942, bool#76943, date#76944, timestamp#76945, array#76946, struct#76947, map#76948] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [short#76905] +- Join Inner, (short#76905 = short#76935) :- Filter isnotnull(short#76905) : +- RelationV2[short#76905] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1 +- Filter isnotnull(short#76935) +- RelationV2[short#76935] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2781) ProjectExecTransformer [short#76905] +- ^(2781) BroadcastHashJoinExecTransformer [short#76905], [short#76935], Inner, BuildRight, false :- ^(2781) FilterExecTransformer isnotnull(short#76905) : +- ^(2781) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[short#76905] ParquetScan DataFilters: [isnotnull(short#76905)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#76905)] +- ^(2781) InputIteratorTransformer[short#76935] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=386034] +- ^(2780) FilterExecTransformer isnotnull(short#76935) +- ^(2780) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[short#76935] ParquetScan DataFilters: [isnotnull(short#76935)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#76935)] +- == Initial Plan == Project [short#76905] +- BroadcastHashJoin [short#76905], [short#76935], Inner, BuildRight, false :- Filter isnotnull(short#76905) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[short#76905] ParquetScan DataFilters: [isnotnull(short#76905)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=385825] +- Filter isnotnull(short#76935) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[short#76935] ParquetScan DataFilters: [isnotnull(short#76935)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxParquetDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#76904] +- Filter (byte#76904 = byte#76934) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#76904,short#76905,int#76906,long#76907L,float#76908,double#76909,decimal#76910,string#76911,binary#76912,bool#76913,date#76914,timestamp#76915,array#76916,struct#76917,map#76918]) : +- RelationV2[byte#76904, short#76905, int#76906, long#76907L, float#76908, double#76909, decimal#76910, string#76911, binary#76912, bool#76913, date#76914, timestamp#76915, array#76916, struct#76917, map#76918] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#76934,short#76935,int#76936,long#76937L,float#76938,double#76939,decimal#76940,string#76941,binary#76942,bool#76943,date#76944,timestamp#76945,array#76946,struct#76947,map#76948]) +- RelationV2[byte#76934, short#76935, int#76936, long#76937L, float#76938, double#76939, decimal#76940, string#76941, binary#76942, bool#76943, date#76944, timestamp#76945, array#76946, struct#76947, map#76948] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [byte#76904] +- Join Inner, (byte#76904 = byte#76934) :- Filter isnotnull(byte#76904) : +- RelationV2[byte#76904] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1 +- Filter isnotnull(byte#76934) +- RelationV2[byte#76934] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2831) ProjectExecTransformer [byte#76904] +- ^(2831) BroadcastHashJoinExecTransformer [byte#76904], [byte#76934], Inner, BuildRight, false :- ^(2831) FilterExecTransformer isnotnull(byte#76904) : +- ^(2831) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[byte#76904] ParquetScan DataFilters: [isnotnull(byte#76904)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#76904)] +- ^(2831) InputIteratorTransformer[byte#76934] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=390684] +- ^(2830) FilterExecTransformer isnotnull(byte#76934) +- ^(2830) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[byte#76934] ParquetScan DataFilters: [isnotnull(byte#76934)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#76934)] +- == Initial Plan == Project [byte#76904] +- BroadcastHashJoin [byte#76904], [byte#76934], Inner, BuildRight, false :- Filter isnotnull(byte#76904) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[byte#76904] ParquetScan DataFilters: [isnotnull(byte#76904)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=390475] +- Filter isnotnull(byte#76934) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[byte#76934] ParquetScan DataFilters: [isnotnull(byte#76934)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]