[GLUTEN-7548][VL][test] Optimize BHJ in velox backend #15535
clickhouse_be_trigger.yml
on: pull_request_target
add-comment
24s
Annotations
29 errors
VeloxOrcDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L185
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#48958]
+- Filter (int#48958 = int#48988)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#48956,short#48957,int#48958,long#48959L,float#48960,double#48961,decimal#48962,string#48963,binary#48964,bool#48965,date#48966,timestamp#48967,array#48968,struct#48969,map#48970])
: +- RelationV2[byte#48956, short#48957, int#48958, long#48959L, float#48960, double#48961, decimal#48962, string#48963, binary#48964, bool#48965, date#48966, timestamp#48967, array#48968, struct#48969, map#48970] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#48986,short#48987,int#48988,long#48989L,float#48990,double#48991,decimal#48992,string#48993,binary#48994,bool#48995,date#48996,timestamp#48997,array#48998,struct#48999,map#49000])
+- RelationV2[byte#48986, short#48987, int#48988, long#48989L, float#48990, double#48991, decimal#48992, string#48993, binary#48994, bool#48995, date#48996, timestamp#48997, array#48998, struct#48999, map#49000] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [int#48958]
+- Join Inner, (int#48958 = int#48988)
:- Filter isnotnull(int#48958)
: +- RelationV2[int#48958] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(int#48988)
+- RelationV2[int#48988] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(1937) ProjectExecTransformer [int#48958]
+- ^(1937) BroadcastHashJoinExecTransformer [int#48958], [int#48988], Inner, BuildRight, false
:- ^(1937) FilterExecTransformer isnotnull(int#48958)
: +- ^(1937) BatchScanExecTransformer[int#48958] OrcScan DataFilters: [isnotnull(int#48958)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#48958)]
+- ^(1937) InputIteratorTransformer[int#48988]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=511644]
+- ^(1936) FilterExecTransformer isnotnull(int#48988)
+- ^(1936) BatchScanExecTransformer[int#48988] OrcScan DataFilters: [isnotnull(int#48988)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#48988)]
+- == Initial Plan ==
Project [int#48958]
+- BroadcastHashJoin [int#48958], [int#48988], Inner, BuildRight, false
:- Filter isnotnull(int#48958)
: +- BatchScan[int#48958] OrcScan DataFilters: [isnotnull(int#48958)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=511435]
+- Filter isnotnull(int#48988)
+- BatchScan[int#48988] OrcScan DataFilters: [isnotnull(int#48988)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxOrcDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L328
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#48956]
+- Filter (byte#48956 = byte#48986)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#48956,short#48957,int#48958,long#48959L,float#48960,double#48961,decimal#48962,string#48963,binary#48964,bool#48965,date#48966,timestamp#48967,array#48968,struct#48969,map#48970])
: +- RelationV2[byte#48956, short#48957, int#48958, long#48959L, float#48960, double#48961, decimal#48962, string#48963, binary#48964, bool#48965, date#48966, timestamp#48967, array#48968, struct#48969, map#48970] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#48986,short#48987,int#48988,long#48989L,float#48990,double#48991,decimal#48992,string#48993,binary#48994,bool#48995,date#48996,timestamp#48997,array#48998,struct#48999,map#49000])
+- RelationV2[byte#48986, short#48987, int#48988, long#48989L, float#48990, double#48991, decimal#48992, string#48993, binary#48994, bool#48995, date#48996, timestamp#48997, array#48998, struct#48999, map#49000] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [byte#48956]
+- Join Inner, (byte#48956 = byte#48986)
:- Filter isnotnull(byte#48956)
: +- RelationV2[byte#48956] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(byte#48986)
+- RelationV2[byte#48986] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(1992) ProjectExecTransformer [byte#48956]
+- ^(1992) BroadcastHashJoinExecTransformer [byte#48956], [byte#48986], Inner, BuildRight, false
:- ^(1992) FilterExecTransformer isnotnull(byte#48956)
: +- ^(1992) BatchScanExecTransformer[byte#48956] OrcScan DataFilters: [isnotnull(byte#48956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] NativeFilters: [isnotnull(byte#48956)]
+- ^(1992) InputIteratorTransformer[byte#48986]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=517361]
+- ^(1991) FilterExecTransformer isnotnull(byte#48986)
+- ^(1991) BatchScanExecTransformer[byte#48986] OrcScan DataFilters: [isnotnull(byte#48986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] NativeFilters: [isnotnull(byte#48986)]
+- == Initial Plan ==
Project [byte#48956]
+- BroadcastHashJoin [byte#48956], [byte#48986], Inner, BuildRight, false
:- Filter isnotnull(byte#48956)
: +- BatchScan[byte#48956] OrcScan DataFilters: [isnotnull(byte#48956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=517152]
+- Filter isnotnull(byte#48986)
+- BatchScan[byte#48986] OrcScan DataFilters: [isnotnull(byte#48986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxParquetDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#50408]
+- Filter (int#50408 = int#50438)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#50406,short#50407,int#50408,long#50409L,float#50410,double#50411,decimal#50412,string#50413,binary#50414,bool#50415,date#50416,timestamp#50417,array#50418,struct#50419,map#50420])
: +- RelationV2[byte#50406, short#50407, int#50408, long#50409L, float#50410, double#50411, decimal#50412, string#50413, binary#50414, bool#50415, date#50416, timestamp#50417, array#50418, struct#50419, map#50420] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#50436,short#50437,int#50438,long#50439L,float#50440,double#50441,decimal#50442,string#50443,binary#50444,bool#50445,date#50446,timestamp#50447,array#50448,struct#50449,map#50450])
+- RelationV2[byte#50436, short#50437, int#50438, long#50439L, float#50440, double#50441, decimal#50442, string#50443, binary#50444, bool#50445, date#50446, timestamp#50447, array#50448, struct#50449, map#50450] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [int#50408]
+- Join Inner, (int#50408 = int#50438)
:- Filter isnotnull(int#50408)
: +- RelationV2[int#50408] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(int#50438)
+- RelationV2[int#50438] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2115) ProjectExecTransformer [int#50408]
+- ^(2115) BroadcastHashJoinExecTransformer [int#50408], [int#50438], Inner, BuildRight, false
:- ^(2115) FilterExecTransformer isnotnull(int#50408)
: +- ^(2115) BatchScanExecTransformer[int#50408] ParquetScan DataFilters: [isnotnull(int#50408)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#50408)]
+- ^(2115) InputIteratorTransformer[int#50438]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=529566]
+- ^(2114) FilterExecTransformer isnotnull(int#50438)
+- ^(2114) BatchScanExecTransformer[int#50438] ParquetScan DataFilters: [isnotnull(int#50438)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#50438)]
+- == Initial Plan ==
Project [int#50408]
+- BroadcastHashJoin [int#50408], [int#50438], Inner, BuildRight, false
:- Filter isnotnull(int#50408)
: +- BatchScan[int#50408] ParquetScan DataFilters: [isnotnull(int#50408)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=529357]
+- Filter isnotnull(int#50438)
+- BatchScan[int#50438] ParquetScan DataFilters: [isnotnull(int#50438)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxParquetDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#50407]
+- Filter (short#50407 = short#50437)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#50406,short#50407,int#50408,long#50409L,float#50410,double#50411,decimal#50412,string#50413,binary#50414,bool#50415,date#50416,timestamp#50417,array#50418,struct#50419,map#50420])
: +- RelationV2[byte#50406, short#50407, int#50408, long#50409L, float#50410, double#50411, decimal#50412, string#50413, binary#50414, bool#50415, date#50416, timestamp#50417, array#50418, struct#50419, map#50420] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#50436,short#50437,int#50438,long#50439L,float#50440,double#50441,decimal#50442,string#50443,binary#50444,bool#50445,date#50446,timestamp#50447,array#50448,struct#50449,map#50450])
+- RelationV2[byte#50436, short#50437, int#50438, long#50439L, float#50440, double#50441, decimal#50442, string#50443, binary#50444, bool#50445, date#50446, timestamp#50447, array#50448, struct#50449, map#50450] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [short#50407]
+- Join Inner, (short#50407 = short#50437)
:- Filter isnotnull(short#50407)
: +- RelationV2[short#50407] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(short#50437)
+- RelationV2[short#50437] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2124) ProjectExecTransformer [short#50407]
+- ^(2124) BroadcastHashJoinExecTransformer [short#50407], [short#50437], Inner, BuildRight, false
:- ^(2124) FilterExecTransformer isnotnull(short#50407)
: +- ^(2124) BatchScanExecTransformer[short#50407] ParquetScan DataFilters: [isnotnull(short#50407)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] NativeFilters: [isnotnull(short#50407)]
+- ^(2124) InputIteratorTransformer[short#50437]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=530481]
+- ^(2123) FilterExecTransformer isnotnull(short#50437)
+- ^(2123) BatchScanExecTransformer[short#50437] ParquetScan DataFilters: [isnotnull(short#50437)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] NativeFilters: [isnotnull(short#50437)]
+- == Initial Plan ==
Project [short#50407]
+- BroadcastHashJoin [short#50407], [short#50437], Inner, BuildRight, false
:- Filter isnotnull(short#50407)
: +- BatchScan[short#50407] ParquetScan DataFilters: [isnotnull(short#50407)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=530272]
+- Filter isnotnull(short#50437)
+- BatchScan[short#50437] ParquetScan DataFilters: [isnotnull(short#50437)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#56483]
+- Filter (short#56483 = short#56513)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#56482,short#56483,int#56484,long#56485L,float#56486,double#56487,decimal#56488,string#56489,binary#56490,bool#56491,date#56492,timestamp#56493,array#56494,struct#56495,map#56496])
: +- RelationV2[byte#56482, short#56483, int#56484, long#56485L, float#56486, double#56487, decimal#56488, string#56489, binary#56490, bool#56491, date#56492, timestamp#56493, array#56494, struct#56495, map#56496] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#56512,short#56513,int#56514,long#56515L,float#56516,double#56517,decimal#56518,string#56519,binary#56520,bool#56521,date#56522,timestamp#56523,array#56524,struct#56525,map#56526])
+- RelationV2[byte#56512, short#56513, int#56514, long#56515L, float#56516, double#56517, decimal#56518, string#56519, binary#56520, bool#56521, date#56522, timestamp#56523, array#56524, struct#56525, map#56526] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [short#56483]
+- Join Inner, (short#56483 = short#56513)
:- Filter isnotnull(short#56483)
: +- RelationV2[short#56483] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(short#56513)
+- RelationV2[short#56513] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2064) ProjectExecTransformer [short#56483]
+- ^(2064) BroadcastHashJoinExecTransformer [short#56483], [short#56513], Inner, BuildRight, false
:- ^(2064) FilterExecTransformer isnotnull(short#56483)
: +- ^(2064) BatchScanExecTransformer[short#56483] OrcScan DataFilters: [isnotnull(short#56483)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#56483)]
+- ^(2064) InputIteratorTransformer[short#56513]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=525865]
+- ^(2063) FilterExecTransformer isnotnull(short#56513)
+- ^(2063) BatchScanExecTransformer[short#56513] OrcScan DataFilters: [isnotnull(short#56513)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#56513)]
+- == Initial Plan ==
Project [short#56483]
+- BroadcastHashJoin [short#56483], [short#56513], Inner, BuildRight, false
:- Filter isnotnull(short#56483)
: +- BatchScan[short#56483] OrcScan DataFilters: [isnotnull(short#56483)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=525656]
+- Filter isnotnull(short#56513)
+- BatchScan[short#56513] OrcScan DataFilters: [isnotnull(short#56513)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#56482]
+- Filter (byte#56482 = byte#56512)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#56482,short#56483,int#56484,long#56485L,float#56486,double#56487,decimal#56488,string#56489,binary#56490,bool#56491,date#56492,timestamp#56493,array#56494,struct#56495,map#56496])
: +- RelationV2[byte#56482, short#56483, int#56484, long#56485L, float#56486, double#56487, decimal#56488, string#56489, binary#56490, bool#56491, date#56492, timestamp#56493, array#56494, struct#56495, map#56496] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#56512,short#56513,int#56514,long#56515L,float#56516,double#56517,decimal#56518,string#56519,binary#56520,bool#56521,date#56522,timestamp#56523,array#56524,struct#56525,map#56526])
+- RelationV2[byte#56512, short#56513, int#56514, long#56515L, float#56516, double#56517, decimal#56518, string#56519, binary#56520, bool#56521, date#56522, timestamp#56523, array#56524, struct#56525, map#56526] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [byte#56482]
+- Join Inner, (byte#56482 = byte#56512)
:- Filter isnotnull(byte#56482)
: +- RelationV2[byte#56482] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(byte#56512)
+- RelationV2[byte#56512] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2102) ProjectExecTransformer [byte#56482]
+- ^(2102) BroadcastHashJoinExecTransformer [byte#56482], [byte#56512], Inner, BuildRight, false
:- ^(2102) FilterExecTransformer isnotnull(byte#56482)
: +- ^(2102) BatchScanExecTransformer[byte#56482] OrcScan DataFilters: [isnotnull(byte#56482)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(byte#56482)]
+- ^(2102) InputIteratorTransformer[byte#56512]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=529891]
+- ^(2101) FilterExecTransformer isnotnull(byte#56512)
+- ^(2101) BatchScanExecTransformer[byte#56512] OrcScan DataFilters: [isnotnull(byte#56512)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(byte#56512)]
+- == Initial Plan ==
Project [byte#56482]
+- BroadcastHashJoin [byte#56482], [byte#56512], Inner, BuildRight, false
:- Filter isnotnull(byte#56482)
: +- BatchScan[byte#56482] OrcScan DataFilters: [isnotnull(byte#56482)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=529682]
+- Filter isnotnull(byte#56512)
+- BatchScan[byte#56512] OrcScan DataFilters: [isnotnull(byte#56512)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxParquetDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#57878]
+- Filter (int#57878 = int#57908)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#57876,short#57877,int#57878,long#57879L,float#57880,double#57881,decimal#57882,string#57883,binary#57884,bool#57885,date#57886,timestamp#57887,array#57888,struct#57889,map#57890])
: +- RelationV2[byte#57876, short#57877, int#57878, long#57879L, float#57880, double#57881, decimal#57882, string#57883, binary#57884, bool#57885, date#57886, timestamp#57887, array#57888, struct#57889, map#57890] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#57906,short#57907,int#57908,long#57909L,float#57910,double#57911,decimal#57912,string#57913,binary#57914,bool#57915,date#57916,timestamp#57917,array#57918,struct#57919,map#57920])
+- RelationV2[byte#57906, short#57907, int#57908, long#57909L, float#57910, double#57911, decimal#57912, string#57913, binary#57914, bool#57915, date#57916, timestamp#57917, array#57918, struct#57919, map#57920] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [int#57878]
+- Join Inner, (int#57878 = int#57908)
:- Filter isnotnull(int#57878)
: +- RelationV2[int#57878] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(int#57908)
+- RelationV2[int#57908] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2225) ProjectExecTransformer [int#57878]
+- ^(2225) BroadcastHashJoinExecTransformer [int#57878], [int#57908], Inner, BuildRight, false
:- ^(2225) FilterExecTransformer isnotnull(int#57878)
: +- ^(2225) BatchScanExecTransformer[int#57878] ParquetScan DataFilters: [isnotnull(int#57878)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#57878)]
+- ^(2225) InputIteratorTransformer[int#57908]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=542096]
+- ^(2224) FilterExecTransformer isnotnull(int#57908)
+- ^(2224) BatchScanExecTransformer[int#57908] ParquetScan DataFilters: [isnotnull(int#57908)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#57908)]
+- == Initial Plan ==
Project [int#57878]
+- BroadcastHashJoin [int#57878], [int#57908], Inner, BuildRight, false
:- Filter isnotnull(int#57878)
: +- BatchScan[int#57878] ParquetScan DataFilters: [isnotnull(int#57878)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=541887]
+- Filter isnotnull(int#57908)
+- BatchScan[int#57908] ParquetScan DataFilters: [isnotnull(int#57908)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxParquetDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#57877]
+- Filter (short#57877 = short#57907)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#57876,short#57877,int#57878,long#57879L,float#57880,double#57881,decimal#57882,string#57883,binary#57884,bool#57885,date#57886,timestamp#57887,array#57888,struct#57889,map#57890])
: +- RelationV2[byte#57876, short#57877, int#57878, long#57879L, float#57880, double#57881, decimal#57882, string#57883, binary#57884, bool#57885, date#57886, timestamp#57887, array#57888, struct#57889, map#57890] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#57906,short#57907,int#57908,long#57909L,float#57910,double#57911,decimal#57912,string#57913,binary#57914,bool#57915,date#57916,timestamp#57917,array#57918,struct#57919,map#57920])
+- RelationV2[byte#57906, short#57907, int#57908, long#57909L, float#57910, double#57911, decimal#57912, string#57913, binary#57914, bool#57915, date#57916, timestamp#57917, array#57918, struct#57919, map#57920] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [short#57877]
+- Join Inner, (short#57877 = short#57907)
:- Filter isnotnull(short#57877)
: +- RelationV2[short#57877] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(short#57907)
+- RelationV2[short#57907] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2234) ProjectExecTransformer [short#57877]
+- ^(2234) BroadcastHashJoinExecTransformer [short#57877], [short#57907], Inner, BuildRight, false
:- ^(2234) FilterExecTransformer isnotnull(short#57877)
: +- ^(2234) BatchScanExecTransformer[short#57877] ParquetScan DataFilters: [isnotnull(short#57877)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#57877)]
+- ^(2234) InputIteratorTransformer[short#57907]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=543011]
+- ^(2233) FilterExecTransformer isnotnull(short#57907)
+- ^(2233) BatchScanExecTransformer[short#57907] ParquetScan DataFilters: [isnotnull(short#57907)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#57907)]
+- == Initial Plan ==
Project [short#57877]
+- BroadcastHashJoin [short#57877], [short#57907], Inner, BuildRight, false
:- Filter isnotnull(short#57877)
: +- BatchScan[short#57877] ParquetScan DataFilters: [isnotnull(short#57877)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=542802]
+- Filter isnotnull(short#57907)
+- BatchScan[short#57907] ParquetScan DataFilters: [isnotnull(short#57907)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxParquetDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L327
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#57876]
+- Filter (byte#57876 = byte#57906)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#57876,short#57877,int#57878,long#57879L,float#57880,double#57881,decimal#57882,string#57883,binary#57884,bool#57885,date#57886,timestamp#57887,array#57888,struct#57889,map#57890])
: +- RelationV2[byte#57876, short#57877, int#57878, long#57879L, float#57880, double#57881, decimal#57882, string#57883, binary#57884, bool#57885, date#57886, timestamp#57887, array#57888, struct#57889, map#57890] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#57906,short#57907,int#57908,long#57909L,float#57910,double#57911,decimal#57912,string#57913,binary#57914,bool#57915,date#57916,timestamp#57917,array#57918,struct#57919,map#57920])
+- RelationV2[byte#57906, short#57907, int#57908, long#57909L, float#57910, double#57911, decimal#57912, string#57913, binary#57914, bool#57915, date#57916, timestamp#57917, array#57918, struct#57919, map#57920] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [byte#57876]
+- Join Inner, (byte#57876 = byte#57906)
:- Filter isnotnull(byte#57876)
: +- RelationV2[byte#57876] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(byte#57906)
+- RelationV2[byte#57906] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2276) ProjectExecTransformer [byte#57876]
+- ^(2276) BroadcastHashJoinExecTransformer [byte#57876], [byte#57906], Inner, BuildRight, false
:- ^(2276) FilterExecTransformer isnotnull(byte#57876)
: +- ^(2276) BatchScanExecTransformer[byte#57876] ParquetScan DataFilters: [isnotnull(byte#57876)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(byte#57876)]
+- ^(2276) InputIteratorTransformer[byte#57906]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=547425]
+- ^(2275) FilterExecTransformer isnotnull(byte#57906)
+- ^(2275) BatchScanExecTransformer[byte#57906] ParquetScan DataFilters: [isnotnull(byte#57906)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(byte#57906)]
+- == Initial Plan ==
Project [byte#57876]
+- BroadcastHashJoin [byte#57876], [byte#57906], Inner, BuildRight, false
:- Filter isnotnull(byte#57876)
: +- BatchScan[byte#57876] ParquetScan DataFilters: [isnotnull(byte#57876)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=547216]
+- Filter isnotnull(byte#57906)
+- BatchScan[byte#57906] ParquetScan DataFilters: [isnotnull(byte#57906)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxOrcDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L191
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#60958]
+- Filter (int#60958 = int#60988)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#60956,short#60957,int#60958,long#60959L,float#60960,double#60961,decimal#60962,string#60963,binary#60964,bool#60965,date#60966,timestamp#60967,array#60968,struct#60969,map#60970])
: +- RelationV2[byte#60956, short#60957, int#60958, long#60959L, float#60960, double#60961, decimal#60962, string#60963, binary#60964, bool#60965, date#60966, timestamp#60967, array#60968, struct#60969, map#60970] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#60986,short#60987,int#60988,long#60989L,float#60990,double#60991,decimal#60992,string#60993,binary#60994,bool#60995,date#60996,timestamp#60997,array#60998,struct#60999,map#61000])
+- RelationV2[byte#60986, short#60987, int#60988, long#60989L, float#60990, double#60991, decimal#60992, string#60993, binary#60994, bool#60995, date#60996, timestamp#60997, array#60998, struct#60999, map#61000] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [int#60958]
+- Join Inner, (int#60958 = int#60988)
:- Filter isnotnull(int#60958)
: +- RelationV2[int#60958] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(int#60988)
+- RelationV2[int#60988] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2157) ProjectExecTransformer [int#60958]
+- ^(2157) BroadcastHashJoinExecTransformer [int#60958], [int#60988], Inner, BuildRight, false
:- ^(2157) FilterExecTransformer isnotnull(int#60958)
: +- ^(2157) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#60958] OrcScan DataFilters: [isnotnull(int#60958)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#60958)]
+- ^(2157) InputIteratorTransformer[int#60988]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=526343]
+- ^(2156) FilterExecTransformer isnotnull(int#60988)
+- ^(2156) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#60988] OrcScan DataFilters: [isnotnull(int#60988)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#60988)]
+- == Initial Plan ==
Project [int#60958]
+- BroadcastHashJoin [int#60958], [int#60988], Inner, BuildRight, false
:- Filter isnotnull(int#60958)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#60958] OrcScan DataFilters: [isnotnull(int#60958)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=526134]
+- Filter isnotnull(int#60988)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#60988] OrcScan DataFilters: [isnotnull(int#60988)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxOrcDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#60957]
+- Filter (short#60957 = short#60987)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#60956,short#60957,int#60958,long#60959L,float#60960,double#60961,decimal#60962,string#60963,binary#60964,bool#60965,date#60966,timestamp#60967,array#60968,struct#60969,map#60970])
: +- RelationV2[byte#60956, short#60957, int#60958, long#60959L, float#60960, double#60961, decimal#60962, string#60963, binary#60964, bool#60965, date#60966, timestamp#60967, array#60968, struct#60969, map#60970] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#60986,short#60987,int#60988,long#60989L,float#60990,double#60991,decimal#60992,string#60993,binary#60994,bool#60995,date#60996,timestamp#60997,array#60998,struct#60999,map#61000])
+- RelationV2[byte#60986, short#60987, int#60988, long#60989L, float#60990, double#60991, decimal#60992, string#60993, binary#60994, bool#60995, date#60996, timestamp#60997, array#60998, struct#60999, map#61000] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [short#60957]
+- Join Inner, (short#60957 = short#60987)
:- Filter isnotnull(short#60957)
: +- RelationV2[short#60957] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(short#60987)
+- RelationV2[short#60987] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2166) ProjectExecTransformer [short#60957]
+- ^(2166) BroadcastHashJoinExecTransformer [short#60957], [short#60987], Inner, BuildRight, false
:- ^(2166) FilterExecTransformer isnotnull(short#60957)
: +- ^(2166) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#60957] OrcScan DataFilters: [isnotnull(short#60957)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#60957)]
+- ^(2166) InputIteratorTransformer[short#60987]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=527258]
+- ^(2165) FilterExecTransformer isnotnull(short#60987)
+- ^(2165) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#60987] OrcScan DataFilters: [isnotnull(short#60987)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#60987)]
+- == Initial Plan ==
Project [short#60957]
+- BroadcastHashJoin [short#60957], [short#60987], Inner, BuildRight, false
:- Filter isnotnull(short#60957)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#60957] OrcScan DataFilters: [isnotnull(short#60957)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=527049]
+- Filter isnotnull(short#60987)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#60987] OrcScan DataFilters: [isnotnull(short#60987)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#60956]
+- Filter (byte#60956 = byte#60986)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#60956,short#60957,int#60958,long#60959L,float#60960,double#60961,decimal#60962,string#60963,binary#60964,bool#60965,date#60966,timestamp#60967,array#60968,struct#60969,map#60970])
: +- RelationV2[byte#60956, short#60957, int#60958, long#60959L, float#60960, double#60961, decimal#60962, string#60963, binary#60964, bool#60965, date#60966, timestamp#60967, array#60968, struct#60969, map#60970] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#60986,short#60987,int#60988,long#60989L,float#60990,double#60991,decimal#60992,string#60993,binary#60994,bool#60995,date#60996,timestamp#60997,array#60998,struct#60999,map#61000])
+- RelationV2[byte#60986, short#60987, int#60988, long#60989L, float#60990, double#60991, decimal#60992, string#60993, binary#60994, bool#60995, date#60996, timestamp#60997, array#60998, struct#60999, map#61000] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [byte#60956]
+- Join Inner, (byte#60956 = byte#60986)
:- Filter isnotnull(byte#60956)
: +- RelationV2[byte#60956] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(byte#60986)
+- RelationV2[byte#60986] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2204) ProjectExecTransformer [byte#60956]
+- ^(2204) BroadcastHashJoinExecTransformer [byte#60956], [byte#60986], Inner, BuildRight, false
:- ^(2204) FilterExecTransformer isnotnull(byte#60956)
: +- ^(2204) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#60956] OrcScan DataFilters: [isnotnull(byte#60956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#60956)]
+- ^(2204) InputIteratorTransformer[byte#60986]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=531284]
+- ^(2203) FilterExecTransformer isnotnull(byte#60986)
+- ^(2203) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#60986] OrcScan DataFilters: [isnotnull(byte#60986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#60986)]
+- == Initial Plan ==
Project [byte#60956]
+- BroadcastHashJoin [byte#60956], [byte#60986], Inner, BuildRight, false
:- Filter isnotnull(byte#60956)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#60956] OrcScan DataFilters: [isnotnull(byte#60956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=531075]
+- Filter isnotnull(byte#60986)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#60986] OrcScan DataFilters: [isnotnull(byte#60986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxParquetDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L190
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#64523]
+- Filter (int#64523 = int#64553)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#64521,short#64522,int#64523,long#64524L,float#64525,double#64526,decimal#64527,string#64528,binary#64529,bool#64530,date#64531,timestamp#64532,array#64533,struct#64534,map#64535])
: +- RelationV2[byte#64521, short#64522, int#64523, long#64524L, float#64525, double#64526, decimal#64527, string#64528, binary#64529, bool#64530, date#64531, timestamp#64532, array#64533, struct#64534, map#64535] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#64551,short#64552,int#64553,long#64554L,float#64555,double#64556,decimal#64557,string#64558,binary#64559,bool#64560,date#64561,timestamp#64562,array#64563,struct#64564,map#64565])
+- RelationV2[byte#64551, short#64552, int#64553, long#64554L, float#64555, double#64556, decimal#64557, string#64558, binary#64559, bool#64560, date#64561, timestamp#64562, array#64563, struct#64564, map#64565] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [int#64523]
+- Join Inner, (int#64523 = int#64553)
:- Filter isnotnull(int#64523)
: +- RelationV2[int#64523] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(int#64553)
+- RelationV2[int#64553] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2331) ProjectExecTransformer [int#64523]
+- ^(2331) BroadcastHashJoinExecTransformer [int#64523], [int#64553], Inner, BuildRight, false
:- ^(2331) FilterExecTransformer isnotnull(int#64523)
: +- ^(2331) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#64523] ParquetScan DataFilters: [isnotnull(int#64523)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#64523)]
+- ^(2331) InputIteratorTransformer[int#64553]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=544108]
+- ^(2330) FilterExecTransformer isnotnull(int#64553)
+- ^(2330) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#64553] ParquetScan DataFilters: [isnotnull(int#64553)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#64553)]
+- == Initial Plan ==
Project [int#64523]
+- BroadcastHashJoin [int#64523], [int#64553], Inner, BuildRight, false
:- Filter isnotnull(int#64523)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#64523] ParquetScan DataFilters: [isnotnull(int#64523)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=543899]
+- Filter isnotnull(int#64553)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#64553] ParquetScan DataFilters: [isnotnull(int#64553)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxParquetDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#64522]
+- Filter (short#64522 = short#64552)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#64521,short#64522,int#64523,long#64524L,float#64525,double#64526,decimal#64527,string#64528,binary#64529,bool#64530,date#64531,timestamp#64532,array#64533,struct#64534,map#64535])
: +- RelationV2[byte#64521, short#64522, int#64523, long#64524L, float#64525, double#64526, decimal#64527, string#64528, binary#64529, bool#64530, date#64531, timestamp#64532, array#64533, struct#64534, map#64535] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#64551,short#64552,int#64553,long#64554L,float#64555,double#64556,decimal#64557,string#64558,binary#64559,bool#64560,date#64561,timestamp#64562,array#64563,struct#64564,map#64565])
+- RelationV2[byte#64551, short#64552, int#64553, long#64554L, float#64555, double#64556, decimal#64557, string#64558, binary#64559, bool#64560, date#64561, timestamp#64562, array#64563, struct#64564, map#64565] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [short#64522]
+- Join Inner, (short#64522 = short#64552)
:- Filter isnotnull(short#64522)
: +- RelationV2[short#64522] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(short#64552)
+- RelationV2[short#64552] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2340) ProjectExecTransformer [short#64522]
+- ^(2340) BroadcastHashJoinExecTransformer [short#64522], [short#64552], Inner, BuildRight, false
:- ^(2340) FilterExecTransformer isnotnull(short#64522)
: +- ^(2340) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#64522] ParquetScan DataFilters: [isnotnull(short#64522)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#64522)]
+- ^(2340) InputIteratorTransformer[short#64552]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=545023]
+- ^(2339) FilterExecTransformer isnotnull(short#64552)
+- ^(2339) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#64552] ParquetScan DataFilters: [isnotnull(short#64552)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#64552)]
+- == Initial Plan ==
Project [short#64522]
+- BroadcastHashJoin [short#64522], [short#64552], Inner, BuildRight, false
:- Filter isnotnull(short#64522)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#64522] ParquetScan DataFilters: [isnotnull(short#64522)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=544814]
+- Filter isnotnull(short#64552)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#64552] ParquetScan DataFilters: [isnotnull(short#64552)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxParquetDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#64521]
+- Filter (byte#64521 = byte#64551)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#64521,short#64522,int#64523,long#64524L,float#64525,double#64526,decimal#64527,string#64528,binary#64529,bool#64530,date#64531,timestamp#64532,array#64533,struct#64534,map#64535])
: +- RelationV2[byte#64521, short#64522, int#64523, long#64524L, float#64525, double#64526, decimal#64527, string#64528, binary#64529, bool#64530, date#64531, timestamp#64532, array#64533, struct#64534, map#64535] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#64551,short#64552,int#64553,long#64554L,float#64555,double#64556,decimal#64557,string#64558,binary#64559,bool#64560,date#64561,timestamp#64562,array#64563,struct#64564,map#64565])
+- RelationV2[byte#64551, short#64552, int#64553, long#64554L, float#64555, double#64556, decimal#64557, string#64558, binary#64559, bool#64560, date#64561, timestamp#64562, array#64563, struct#64564, map#64565] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [byte#64521]
+- Join Inner, (byte#64521 = byte#64551)
:- Filter isnotnull(byte#64521)
: +- RelationV2[byte#64521] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(byte#64551)
+- RelationV2[byte#64551] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2378) ProjectExecTransformer [byte#64521]
+- ^(2378) BroadcastHashJoinExecTransformer [byte#64521], [byte#64551], Inner, BuildRight, false
:- ^(2378) FilterExecTransformer isnotnull(byte#64521)
: +- ^(2378) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#64521] ParquetScan DataFilters: [isnotnull(byte#64521)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#64521)]
+- ^(2378) InputIteratorTransformer[byte#64551]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=549049]
+- ^(2377) FilterExecTransformer isnotnull(byte#64551)
+- ^(2377) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#64551] ParquetScan DataFilters: [isnotnull(byte#64551)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#64551)]
+- == Initial Plan ==
Project [byte#64521]
+- BroadcastHashJoin [byte#64521], [byte#64551], Inner, BuildRight, false
:- Filter isnotnull(byte#64521)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#64521] ParquetScan DataFilters: [isnotnull(byte#64521)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=548840]
+- Filter isnotnull(byte#64551)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#64551] ParquetScan DataFilters: [isnotnull(byte#64551)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxOrcDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#73724]
+- Filter (short#73724 = short#73754)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73723,short#73724,int#73725,long#73726L,float#73727,double#73728,decimal#73729,string#73730,binary#73731,bool#73732,date#73733,timestamp#73734,array#73735,struct#73736,map#73737])
: +- RelationV2[byte#73723, short#73724, int#73725, long#73726L, float#73727, double#73728, decimal#73729, string#73730, binary#73731, bool#73732, date#73733, timestamp#73734, array#73735, struct#73736, map#73737] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73753,short#73754,int#73755,long#73756L,float#73757,double#73758,decimal#73759,string#73760,binary#73761,bool#73762,date#73763,timestamp#73764,array#73765,struct#73766,map#73767])
+- RelationV2[byte#73753, short#73754, int#73755, long#73756L, float#73757, double#73758, decimal#73759, string#73760, binary#73761, bool#73762, date#73763, timestamp#73764, array#73765, struct#73766, map#73767] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [short#73724]
+- Join Inner, (short#73724 = short#73754)
:- Filter isnotnull(short#73724)
: +- RelationV2[short#73724] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(short#73754)
+- RelationV2[short#73754] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2390) ProjectExecTransformer [short#73724]
+- ^(2390) BroadcastHashJoinExecTransformer [short#73724], [short#73754], Inner, BuildRight, false
:- ^(2390) FilterExecTransformer isnotnull(short#73724)
: +- ^(2390) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73724] OrcScan DataFilters: [isnotnull(short#73724)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73724)]
+- ^(2390) InputIteratorTransformer[short#73754]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=550160]
+- ^(2389) FilterExecTransformer isnotnull(short#73754)
+- ^(2389) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73754] OrcScan DataFilters: [isnotnull(short#73754)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73754)]
+- == Initial Plan ==
Project [short#73724]
+- BroadcastHashJoin [short#73724], [short#73754], Inner, BuildRight, false
:- Filter isnotnull(short#73724)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73724] OrcScan DataFilters: [isnotnull(short#73724)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=549951]
+- Filter isnotnull(short#73754)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73754] OrcScan DataFilters: [isnotnull(short#73754)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxParquetDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#77599]
+- Filter (int#77599 = int#77629)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#77597,short#77598,int#77599,long#77600L,float#77601,double#77602,decimal#77603,string#77604,binary#77605,bool#77606,date#77607,timestamp#77608,array#77609,struct#77610,map#77611])
: +- RelationV2[byte#77597, short#77598, int#77599, long#77600L, float#77601, double#77602, decimal#77603, string#77604, binary#77605, bool#77606, date#77607, timestamp#77608, array#77609, struct#77610, map#77611] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#77627,short#77628,int#77629,long#77630L,float#77631,double#77632,decimal#77633,string#77634,binary#77635,bool#77636,date#77637,timestamp#77638,array#77639,struct#77640,map#77641])
+- RelationV2[byte#77627, short#77628, int#77629, long#77630L, float#77631, double#77632, decimal#77633, string#77634, binary#77635, bool#77636, date#77637, timestamp#77638, array#77639, struct#77640, map#77641] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [int#77599]
+- Join Inner, (int#77599 = int#77629)
:- Filter isnotnull(int#77599)
: +- RelationV2[int#77599] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(int#77629)
+- RelationV2[int#77629] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2569) ProjectExecTransformer [int#77599]
+- ^(2569) BroadcastHashJoinExecTransformer [int#77599], [int#77629], Inner, BuildRight, false
:- ^(2569) FilterExecTransformer isnotnull(int#77599)
: +- ^(2569) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77599] ParquetScan DataFilters: [isnotnull(int#77599)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77599)]
+- ^(2569) InputIteratorTransformer[int#77629]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=567762]
+- ^(2568) FilterExecTransformer isnotnull(int#77629)
+- ^(2568) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77629] ParquetScan DataFilters: [isnotnull(int#77629)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77629)]
+- == Initial Plan ==
Project [int#77599]
+- BroadcastHashJoin [int#77599], [int#77629], Inner, BuildRight, false
:- Filter isnotnull(int#77599)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77599] ParquetScan DataFilters: [isnotnull(int#77599)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=567553]
+- Filter isnotnull(int#77629)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77629] ParquetScan DataFilters: [isnotnull(int#77629)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxParquetDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#77598]
+- Filter (short#77598 = short#77628)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#77597,short#77598,int#77599,long#77600L,float#77601,double#77602,decimal#77603,string#77604,binary#77605,bool#77606,date#77607,timestamp#77608,array#77609,struct#77610,map#77611])
: +- RelationV2[byte#77597, short#77598, int#77599, long#77600L, float#77601, double#77602, decimal#77603, string#77604, binary#77605, bool#77606, date#77607, timestamp#77608, array#77609, struct#77610, map#77611] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#77627,short#77628,int#77629,long#77630L,float#77631,double#77632,decimal#77633,string#77634,binary#77635,bool#77636,date#77637,timestamp#77638,array#77639,struct#77640,map#77641])
+- RelationV2[byte#77627, short#77628, int#77629, long#77630L, float#77631, double#77632, decimal#77633, string#77634, binary#77635, bool#77636, date#77637, timestamp#77638, array#77639, struct#77640, map#77641] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [short#77598]
+- Join Inner, (short#77598 = short#77628)
:- Filter isnotnull(short#77598)
: +- RelationV2[short#77598] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(short#77628)
+- RelationV2[short#77628] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2578) ProjectExecTransformer [short#77598]
+- ^(2578) BroadcastHashJoinExecTransformer [short#77598], [short#77628], Inner, BuildRight, false
:- ^(2578) FilterExecTransformer isnotnull(short#77598)
: +- ^(2578) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77598] ParquetScan DataFilters: [isnotnull(short#77598)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77598)]
+- ^(2578) InputIteratorTransformer[short#77628]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=568677]
+- ^(2577) FilterExecTransformer isnotnull(short#77628)
+- ^(2577) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77628] ParquetScan DataFilters: [isnotnull(short#77628)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77628)]
+- == Initial Plan ==
Project [short#77598]
+- BroadcastHashJoin [short#77598], [short#77628], Inner, BuildRight, false
:- Filter isnotnull(short#77598)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77598] ParquetScan DataFilters: [isnotnull(short#77598)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=568468]
+- Filter isnotnull(short#77628)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77628] ParquetScan DataFilters: [isnotnull(short#77628)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxParquetDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#77597]
+- Filter (byte#77597 = byte#77627)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#77597,short#77598,int#77599,long#77600L,float#77601,double#77602,decimal#77603,string#77604,binary#77605,bool#77606,date#77607,timestamp#77608,array#77609,struct#77610,map#77611])
: +- RelationV2[byte#77597, short#77598, int#77599, long#77600L, float#77601, double#77602, decimal#77603, string#77604, binary#77605, bool#77606, date#77607, timestamp#77608, array#77609, struct#77610, map#77611] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#77627,short#77628,int#77629,long#77630L,float#77631,double#77632,decimal#77633,string#77634,binary#77635,bool#77636,date#77637,timestamp#77638,array#77639,struct#77640,map#77641])
+- RelationV2[byte#77627, short#77628, int#77629, long#77630L, float#77631, double#77632, decimal#77633, string#77634, binary#77635, bool#77636, date#77637, timestamp#77638, array#77639, struct#77640, map#77641] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [byte#77597]
+- Join Inner, (byte#77597 = byte#77627)
:- Filter isnotnull(byte#77597)
: +- RelationV2[byte#77597] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(byte#77627)
+- RelationV2[byte#77627] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2628) ProjectExecTransformer [byte#77597]
+- ^(2628) BroadcastHashJoinExecTransformer [byte#77597], [byte#77627], Inner, BuildRight, false
:- ^(2628) FilterExecTransformer isnotnull(byte#77597)
: +- ^(2628) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77597] ParquetScan DataFilters: [isnotnull(byte#77597)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77597)]
+- ^(2628) InputIteratorTransformer[byte#77627]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=573327]
+- ^(2627) FilterExecTransformer isnotnull(byte#77627)
+- ^(2627) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77627] ParquetScan DataFilters: [isnotnull(byte#77627)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77627)]
+- == Initial Plan ==
Project [byte#77597]
+- BroadcastHashJoin [byte#77597], [byte#77627], Inner, BuildRight, false
:- Filter isnotnull(byte#77597)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77597] ParquetScan DataFilters: [isnotnull(byte#77597)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=573118]
+- Filter isnotnull(byte#77627)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77627] ParquetScan DataFilters: [isnotnull(byte#77627)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxOrcDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L185
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#73704]
+- Filter (int#73704 = int#73734)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73702,short#73703,int#73704,long#73705L,float#73706,double#73707,decimal#73708,string#73709,binary#73710,bool#73711,date#73712,timestamp#73713,array#73714,struct#73715,map#73716])
: +- RelationV2[byte#73702, short#73703, int#73704, long#73705L, float#73706, double#73707, decimal#73708, string#73709, binary#73710, bool#73711, date#73712, timestamp#73713, array#73714, struct#73715, map#73716] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73732,short#73733,int#73734,long#73735L,float#73736,double#73737,decimal#73738,string#73739,binary#73740,bool#73741,date#73742,timestamp#73743,array#73744,struct#73745,map#73746])
+- RelationV2[byte#73732, short#73733, int#73734, long#73735L, float#73736, double#73737, decimal#73738, string#73739, binary#73740, bool#73741, date#73742, timestamp#73743, array#73744, struct#73745, map#73746] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [int#73704]
+- Join Inner, (int#73704 = int#73734)
:- Filter isnotnull(int#73704)
: +- RelationV2[int#73704] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(int#73734)
+- RelationV2[int#73734] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2377) ProjectExecTransformer [int#73704]
+- ^(2377) BroadcastHashJoinExecTransformer [int#73704], [int#73734], Inner, BuildRight, false
:- ^(2377) FilterExecTransformer isnotnull(int#73704)
: +- ^(2377) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#73704] OrcScan DataFilters: [isnotnull(int#73704)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73704)]
+- ^(2377) InputIteratorTransformer[int#73734]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=542836]
+- ^(2376) FilterExecTransformer isnotnull(int#73734)
+- ^(2376) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#73734] OrcScan DataFilters: [isnotnull(int#73734)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73734)]
+- == Initial Plan ==
Project [int#73704]
+- BroadcastHashJoin [int#73704], [int#73734], Inner, BuildRight, false
:- Filter isnotnull(int#73704)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#73704] OrcScan DataFilters: [isnotnull(int#73704)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=542627]
+- Filter isnotnull(int#73734)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#73734] OrcScan DataFilters: [isnotnull(int#73734)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxOrcDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#73703]
+- Filter (short#73703 = short#73733)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73702,short#73703,int#73704,long#73705L,float#73706,double#73707,decimal#73708,string#73709,binary#73710,bool#73711,date#73712,timestamp#73713,array#73714,struct#73715,map#73716])
: +- RelationV2[byte#73702, short#73703, int#73704, long#73705L, float#73706, double#73707, decimal#73708, string#73709, binary#73710, bool#73711, date#73712, timestamp#73713, array#73714, struct#73715, map#73716] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73732,short#73733,int#73734,long#73735L,float#73736,double#73737,decimal#73738,string#73739,binary#73740,bool#73741,date#73742,timestamp#73743,array#73744,struct#73745,map#73746])
+- RelationV2[byte#73732, short#73733, int#73734, long#73735L, float#73736, double#73737, decimal#73738, string#73739, binary#73740, bool#73741, date#73742, timestamp#73743, array#73744, struct#73745, map#73746] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [short#73703]
+- Join Inner, (short#73703 = short#73733)
:- Filter isnotnull(short#73703)
: +- RelationV2[short#73703] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(short#73733)
+- RelationV2[short#73733] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2386) ProjectExecTransformer [short#73703]
+- ^(2386) BroadcastHashJoinExecTransformer [short#73703], [short#73733], Inner, BuildRight, false
:- ^(2386) FilterExecTransformer isnotnull(short#73703)
: +- ^(2386) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73703] OrcScan DataFilters: [isnotnull(short#73703)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73703)]
+- ^(2386) InputIteratorTransformer[short#73733]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=543751]
+- ^(2385) FilterExecTransformer isnotnull(short#73733)
+- ^(2385) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73733] OrcScan DataFilters: [isnotnull(short#73733)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73733)]
+- == Initial Plan ==
Project [short#73703]
+- BroadcastHashJoin [short#73703], [short#73733], Inner, BuildRight, false
:- Filter isnotnull(short#73703)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73703] OrcScan DataFilters: [isnotnull(short#73703)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=543542]
+- Filter isnotnull(short#73733)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73733] OrcScan DataFilters: [isnotnull(short#73733)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#73702]
+- Filter (byte#73702 = byte#73732)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73702,short#73703,int#73704,long#73705L,float#73706,double#73707,decimal#73708,string#73709,binary#73710,bool#73711,date#73712,timestamp#73713,array#73714,struct#73715,map#73716])
: +- RelationV2[byte#73702, short#73703, int#73704, long#73705L, float#73706, double#73707, decimal#73708, string#73709, binary#73710, bool#73711, date#73712, timestamp#73713, array#73714, struct#73715, map#73716] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73732,short#73733,int#73734,long#73735L,float#73736,double#73737,decimal#73738,string#73739,binary#73740,bool#73741,date#73742,timestamp#73743,array#73744,struct#73745,map#73746])
+- RelationV2[byte#73732, short#73733, int#73734, long#73735L, float#73736, double#73737, decimal#73738, string#73739, binary#73740, bool#73741, date#73742, timestamp#73743, array#73744, struct#73745, map#73746] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [byte#73702]
+- Join Inner, (byte#73702 = byte#73732)
:- Filter isnotnull(byte#73702)
: +- RelationV2[byte#73702] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(byte#73732)
+- RelationV2[byte#73732] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2436) ProjectExecTransformer [byte#73702]
+- ^(2436) BroadcastHashJoinExecTransformer [byte#73702], [byte#73732], Inner, BuildRight, false
:- ^(2436) FilterExecTransformer isnotnull(byte#73702)
: +- ^(2436) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73702] OrcScan DataFilters: [isnotnull(byte#73702)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73702)]
+- ^(2436) InputIteratorTransformer[byte#73732]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=548401]
+- ^(2435) FilterExecTransformer isnotnull(byte#73732)
+- ^(2435) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73732] OrcScan DataFilters: [isnotnull(byte#73732)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73732)]
+- == Initial Plan ==
Project [byte#73702]
+- BroadcastHashJoin [byte#73702], [byte#73732], Inner, BuildRight, false
:- Filter isnotnull(byte#73702)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73702] OrcScan DataFilters: [isnotnull(byte#73702)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=548192]
+- Filter isnotnull(byte#73732)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73732] OrcScan DataFilters: [isnotnull(byte#73732)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxParquetDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#77367]
+- Filter (short#77367 = short#77397)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#77366,short#77367,int#77368,long#77369L,float#77370,double#77371,decimal#77372,string#77373,binary#77374,bool#77375,date#77376,timestamp#77377,array#77378,struct#77379,map#77380])
: +- RelationV2[byte#77366, short#77367, int#77368, long#77369L, float#77370, double#77371, decimal#77372, string#77373, binary#77374, bool#77375, date#77376, timestamp#77377, array#77378, struct#77379, map#77380] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#77396,short#77397,int#77398,long#77399L,float#77400,double#77401,decimal#77402,string#77403,binary#77404,bool#77405,date#77406,timestamp#77407,array#77408,struct#77409,map#77410])
+- RelationV2[byte#77396, short#77397, int#77398, long#77399L, float#77400, double#77401, decimal#77402, string#77403, binary#77404, bool#77405, date#77406, timestamp#77407, array#77408, struct#77409, map#77410] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [short#77367]
+- Join Inner, (short#77367 = short#77397)
:- Filter isnotnull(short#77367)
: +- RelationV2[short#77367] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(short#77397)
+- RelationV2[short#77397] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2574) ProjectExecTransformer [short#77367]
+- ^(2574) BroadcastHashJoinExecTransformer [short#77367], [short#77397], Inner, BuildRight, false
:- ^(2574) FilterExecTransformer isnotnull(short#77367)
: +- ^(2574) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77367] ParquetScan DataFilters: [isnotnull(short#77367)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77367)]
+- ^(2574) InputIteratorTransformer[short#77397]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=562268]
+- ^(2573) FilterExecTransformer isnotnull(short#77397)
+- ^(2573) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77397] ParquetScan DataFilters: [isnotnull(short#77397)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77397)]
+- == Initial Plan ==
Project [short#77367]
+- BroadcastHashJoin [short#77367], [short#77397], Inner, BuildRight, false
:- Filter isnotnull(short#77367)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77367] ParquetScan DataFilters: [isnotnull(short#77367)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=562059]
+- Filter isnotnull(short#77397)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77397] ParquetScan DataFilters: [isnotnull(short#77397)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L185
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#73137]
+- Filter (int#73137 = int#73167)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73135,short#73136,int#73137,long#73138L,float#73139,double#73140,decimal#73141,string#73142,binary#73143,bool#73144,date#73145,timestamp#73146,array#73147,struct#73148,map#73149])
: +- RelationV2[byte#73135, short#73136, int#73137, long#73138L, float#73139, double#73140, decimal#73141, string#73142, binary#73143, bool#73144, date#73145, timestamp#73146, array#73147, struct#73148, map#73149] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73165,short#73166,int#73167,long#73168L,float#73169,double#73170,decimal#73171,string#73172,binary#73173,bool#73174,date#73175,timestamp#73176,array#73177,struct#73178,map#73179])
+- RelationV2[byte#73165, short#73166, int#73167, long#73168L, float#73169, double#73170, decimal#73171, string#73172, binary#73173, bool#73174, date#73175, timestamp#73176, array#73177, struct#73178, map#73179] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [int#73137]
+- Join Inner, (int#73137 = int#73167)
:- Filter isnotnull(int#73137)
: +- RelationV2[int#73137] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(int#73167)
+- RelationV2[int#73167] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2584) ProjectExecTransformer [int#73137]
+- ^(2584) BroadcastHashJoinExecTransformer [int#73137], [int#73167], Inner, BuildRight, false
:- ^(2584) FilterExecTransformer isnotnull(int#73137)
: +- ^(2584) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[int#73137] OrcScan DataFilters: [isnotnull(int#73137)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73137)]
+- ^(2584) InputIteratorTransformer[int#73167]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=366602]
+- ^(2583) FilterExecTransformer isnotnull(int#73167)
+- ^(2583) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[int#73167] OrcScan DataFilters: [isnotnull(int#73167)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73167)]
+- == Initial Plan ==
Project [int#73137]
+- BroadcastHashJoin [int#73137], [int#73167], Inner, BuildRight, false
:- Filter isnotnull(int#73137)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[int#73137] OrcScan DataFilters: [isnotnull(int#73137)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=366393]
+- Filter isnotnull(int#73167)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[int#73167] OrcScan DataFilters: [isnotnull(int#73167)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxOrcDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#73136]
+- Filter (short#73136 = short#73166)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73135,short#73136,int#73137,long#73138L,float#73139,double#73140,decimal#73141,string#73142,binary#73143,bool#73144,date#73145,timestamp#73146,array#73147,struct#73148,map#73149])
: +- RelationV2[byte#73135, short#73136, int#73137, long#73138L, float#73139, double#73140, decimal#73141, string#73142, binary#73143, bool#73144, date#73145, timestamp#73146, array#73147, struct#73148, map#73149] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73165,short#73166,int#73167,long#73168L,float#73169,double#73170,decimal#73171,string#73172,binary#73173,bool#73174,date#73175,timestamp#73176,array#73177,struct#73178,map#73179])
+- RelationV2[byte#73165, short#73166, int#73167, long#73168L, float#73169, double#73170, decimal#73171, string#73172, binary#73173, bool#73174, date#73175, timestamp#73176, array#73177, struct#73178, map#73179] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [short#73136]
+- Join Inner, (short#73136 = short#73166)
:- Filter isnotnull(short#73136)
: +- RelationV2[short#73136] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(short#73166)
+- RelationV2[short#73166] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2593) ProjectExecTransformer [short#73136]
+- ^(2593) BroadcastHashJoinExecTransformer [short#73136], [short#73166], Inner, BuildRight, false
:- ^(2593) FilterExecTransformer isnotnull(short#73136)
: +- ^(2593) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[short#73136] OrcScan DataFilters: [isnotnull(short#73136)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73136)]
+- ^(2593) InputIteratorTransformer[short#73166]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=367517]
+- ^(2592) FilterExecTransformer isnotnull(short#73166)
+- ^(2592) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[short#73166] OrcScan DataFilters: [isnotnull(short#73166)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73166)]
+- == Initial Plan ==
Project [short#73136]
+- BroadcastHashJoin [short#73136], [short#73166], Inner, BuildRight, false
:- Filter isnotnull(short#73136)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[short#73136] OrcScan DataFilters: [isnotnull(short#73136)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=367308]
+- Filter isnotnull(short#73166)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[short#73166] OrcScan DataFilters: [isnotnull(short#73166)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L328
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#73135]
+- Filter (byte#73135 = byte#73165)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73135,short#73136,int#73137,long#73138L,float#73139,double#73140,decimal#73141,string#73142,binary#73143,bool#73144,date#73145,timestamp#73146,array#73147,struct#73148,map#73149])
: +- RelationV2[byte#73135, short#73136, int#73137, long#73138L, float#73139, double#73140, decimal#73141, string#73142, binary#73143, bool#73144, date#73145, timestamp#73146, array#73147, struct#73148, map#73149] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73165,short#73166,int#73167,long#73168L,float#73169,double#73170,decimal#73171,string#73172,binary#73173,bool#73174,date#73175,timestamp#73176,array#73177,struct#73178,map#73179])
+- RelationV2[byte#73165, short#73166, int#73167, long#73168L, float#73169, double#73170, decimal#73171, string#73172, binary#73173, bool#73174, date#73175, timestamp#73176, array#73177, struct#73178, map#73179] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [byte#73135]
+- Join Inner, (byte#73135 = byte#73165)
:- Filter isnotnull(byte#73135)
: +- RelationV2[byte#73135] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(byte#73165)
+- RelationV2[byte#73165] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2647) ProjectExecTransformer [byte#73135]
+- ^(2647) BroadcastHashJoinExecTransformer [byte#73135], [byte#73165], Inner, BuildRight, false
:- ^(2647) FilterExecTransformer isnotnull(byte#73135)
: +- ^(2647) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[byte#73135] OrcScan DataFilters: [isnotnull(byte#73135)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73135)]
+- ^(2647) InputIteratorTransformer[byte#73165]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=372555]
+- ^(2646) FilterExecTransformer isnotnull(byte#73165)
+- ^(2646) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[byte#73165] OrcScan DataFilters: [isnotnull(byte#73165)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73165)]
+- == Initial Plan ==
Project [byte#73135]
+- BroadcastHashJoin [byte#73135], [byte#73165], Inner, BuildRight, false
:- Filter isnotnull(byte#73135)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[byte#73135] OrcScan DataFilters: [isnotnull(byte#73135)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=372346]
+- Filter isnotnull(byte#73165)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[byte#73165] OrcScan DataFilters: [isnotnull(byte#73165)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxParquetDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#76906]
+- Filter (int#76906 = int#76936)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#76904,short#76905,int#76906,long#76907L,float#76908,double#76909,decimal#76910,string#76911,binary#76912,bool#76913,date#76914,timestamp#76915,array#76916,struct#76917,map#76918])
: +- RelationV2[byte#76904, short#76905, int#76906, long#76907L, float#76908, double#76909, decimal#76910, string#76911, binary#76912, bool#76913, date#76914, timestamp#76915, array#76916, struct#76917, map#76918] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#76934,short#76935,int#76936,long#76937L,float#76938,double#76939,decimal#76940,string#76941,binary#76942,bool#76943,date#76944,timestamp#76945,array#76946,struct#76947,map#76948])
+- RelationV2[byte#76934, short#76935, int#76936, long#76937L, float#76938, double#76939, decimal#76940, string#76941, binary#76942, bool#76943, date#76944, timestamp#76945, array#76946, struct#76947, map#76948] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [int#76906]
+- Join Inner, (int#76906 = int#76936)
:- Filter isnotnull(int#76906)
: +- RelationV2[int#76906] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1
+- Filter isnotnull(int#76936)
+- RelationV2[int#76936] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2772) ProjectExecTransformer [int#76906]
+- ^(2772) BroadcastHashJoinExecTransformer [int#76906], [int#76936], Inner, BuildRight, false
:- ^(2772) FilterExecTransformer isnotnull(int#76906)
: +- ^(2772) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[int#76906] ParquetScan DataFilters: [isnotnull(int#76906)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#76906)]
+- ^(2772) InputIteratorTransformer[int#76936]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=385119]
+- ^(2771) FilterExecTransformer isnotnull(int#76936)
+- ^(2771) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[int#76936] ParquetScan DataFilters: [isnotnull(int#76936)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#76936)]
+- == Initial Plan ==
Project [int#76906]
+- BroadcastHashJoin [int#76906], [int#76936], Inner, BuildRight, false
:- Filter isnotnull(int#76906)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[int#76906] ParquetScan DataFilters: [isnotnull(int#76906)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=384910]
+- Filter isnotnull(int#76936)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[int#76936] ParquetScan DataFilters: [isnotnull(int#76936)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxParquetDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#76905]
+- Filter (short#76905 = short#76935)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#76904,short#76905,int#76906,long#76907L,float#76908,double#76909,decimal#76910,string#76911,binary#76912,bool#76913,date#76914,timestamp#76915,array#76916,struct#76917,map#76918])
: +- RelationV2[byte#76904, short#76905, int#76906, long#76907L, float#76908, double#76909, decimal#76910, string#76911, binary#76912, bool#76913, date#76914, timestamp#76915, array#76916, struct#76917, map#76918] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#76934,short#76935,int#76936,long#76937L,float#76938,double#76939,decimal#76940,string#76941,binary#76942,bool#76943,date#76944,timestamp#76945,array#76946,struct#76947,map#76948])
+- RelationV2[byte#76934, short#76935, int#76936, long#76937L, float#76938, double#76939, decimal#76940, string#76941, binary#76942, bool#76943, date#76944, timestamp#76945, array#76946, struct#76947, map#76948] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [short#76905]
+- Join Inner, (short#76905 = short#76935)
:- Filter isnotnull(short#76905)
: +- RelationV2[short#76905] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1
+- Filter isnotnull(short#76935)
+- RelationV2[short#76935] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2781) ProjectExecTransformer [short#76905]
+- ^(2781) BroadcastHashJoinExecTransformer [short#76905], [short#76935], Inner, BuildRight, false
:- ^(2781) FilterExecTransformer isnotnull(short#76905)
: +- ^(2781) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[short#76905] ParquetScan DataFilters: [isnotnull(short#76905)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#76905)]
+- ^(2781) InputIteratorTransformer[short#76935]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=386034]
+- ^(2780) FilterExecTransformer isnotnull(short#76935)
+- ^(2780) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[short#76935] ParquetScan DataFilters: [isnotnull(short#76935)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#76935)]
+- == Initial Plan ==
Project [short#76905]
+- BroadcastHashJoin [short#76905], [short#76935], Inner, BuildRight, false
:- Filter isnotnull(short#76905)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[short#76905] ParquetScan DataFilters: [isnotnull(short#76905)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=385825]
+- Filter isnotnull(short#76935)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[short#76935] ParquetScan DataFilters: [isnotnull(short#76935)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxParquetDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#76904]
+- Filter (byte#76904 = byte#76934)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#76904,short#76905,int#76906,long#76907L,float#76908,double#76909,decimal#76910,string#76911,binary#76912,bool#76913,date#76914,timestamp#76915,array#76916,struct#76917,map#76918])
: +- RelationV2[byte#76904, short#76905, int#76906, long#76907L, float#76908, double#76909, decimal#76910, string#76911, binary#76912, bool#76913, date#76914, timestamp#76915, array#76916, struct#76917, map#76918] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#76934,short#76935,int#76936,long#76937L,float#76938,double#76939,decimal#76940,string#76941,binary#76942,bool#76943,date#76944,timestamp#76945,array#76946,struct#76947,map#76948])
+- RelationV2[byte#76934, short#76935, int#76936, long#76937L, float#76938, double#76939, decimal#76940, string#76941, binary#76942, bool#76943, date#76944, timestamp#76945, array#76946, struct#76947, map#76948] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [byte#76904]
+- Join Inner, (byte#76904 = byte#76934)
:- Filter isnotnull(byte#76904)
: +- RelationV2[byte#76904] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1
+- Filter isnotnull(byte#76934)
+- RelationV2[byte#76934] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2831) ProjectExecTransformer [byte#76904]
+- ^(2831) BroadcastHashJoinExecTransformer [byte#76904], [byte#76934], Inner, BuildRight, false
:- ^(2831) FilterExecTransformer isnotnull(byte#76904)
: +- ^(2831) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[byte#76904] ParquetScan DataFilters: [isnotnull(byte#76904)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#76904)]
+- ^(2831) InputIteratorTransformer[byte#76934]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=390684]
+- ^(2830) FilterExecTransformer isnotnull(byte#76934)
+- ^(2830) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[byte#76934] ParquetScan DataFilters: [isnotnull(byte#76934)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#76934)]
+- == Initial Plan ==
Project [byte#76904]
+- BroadcastHashJoin [byte#76904], [byte#76934], Inner, BuildRight, false
:- Filter isnotnull(byte#76904)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[byte#76904] ParquetScan DataFilters: [isnotnull(byte#76904)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=390475]
+- Filter isnotnull(byte#76934)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[byte#76934] ParquetScan DataFilters: [isnotnull(byte#76934)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|