[GLUTEN-7548][VL][test] Optimize BHJ in velox backend #15538
clickhouse_be_trigger.yml
on: pull_request_target
add-comment
23s
Annotations
29 errors
VeloxOrcDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L185
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#48956]
+- Filter (int#48956 = int#48986)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#48954,short#48955,int#48956,long#48957L,float#48958,double#48959,decimal#48960,string#48961,binary#48962,bool#48963,date#48964,timestamp#48965,array#48966,struct#48967,map#48968])
: +- RelationV2[byte#48954, short#48955, int#48956, long#48957L, float#48958, double#48959, decimal#48960, string#48961, binary#48962, bool#48963, date#48964, timestamp#48965, array#48966, struct#48967, map#48968] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#48984,short#48985,int#48986,long#48987L,float#48988,double#48989,decimal#48990,string#48991,binary#48992,bool#48993,date#48994,timestamp#48995,array#48996,struct#48997,map#48998])
+- RelationV2[byte#48984, short#48985, int#48986, long#48987L, float#48988, double#48989, decimal#48990, string#48991, binary#48992, bool#48993, date#48994, timestamp#48995, array#48996, struct#48997, map#48998] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [int#48956]
+- Join Inner, (int#48956 = int#48986)
:- Filter isnotnull(int#48956)
: +- RelationV2[int#48956] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(int#48986)
+- RelationV2[int#48986] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(1936) ProjectExecTransformer [int#48956]
+- ^(1936) BroadcastHashJoinExecTransformer [int#48956], [int#48986], Inner, BuildRight, false
:- ^(1936) FilterExecTransformer isnotnull(int#48956)
: +- ^(1936) BatchScanExecTransformer[int#48956] OrcScan DataFilters: [isnotnull(int#48956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#48956)]
+- ^(1936) InputIteratorTransformer[int#48986]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=511584]
+- ^(1935) FilterExecTransformer isnotnull(int#48986)
+- ^(1935) BatchScanExecTransformer[int#48986] OrcScan DataFilters: [isnotnull(int#48986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#48986)]
+- == Initial Plan ==
Project [int#48956]
+- BroadcastHashJoin [int#48956], [int#48986], Inner, BuildRight, false
:- Filter isnotnull(int#48956)
: +- BatchScan[int#48956] OrcScan DataFilters: [isnotnull(int#48956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=511375]
+- Filter isnotnull(int#48986)
+- BatchScan[int#48986] OrcScan DataFilters: [isnotnull(int#48986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxOrcDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#48955]
+- Filter (short#48955 = short#48985)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#48954,short#48955,int#48956,long#48957L,float#48958,double#48959,decimal#48960,string#48961,binary#48962,bool#48963,date#48964,timestamp#48965,array#48966,struct#48967,map#48968])
: +- RelationV2[byte#48954, short#48955, int#48956, long#48957L, float#48958, double#48959, decimal#48960, string#48961, binary#48962, bool#48963, date#48964, timestamp#48965, array#48966, struct#48967, map#48968] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#48984,short#48985,int#48986,long#48987L,float#48988,double#48989,decimal#48990,string#48991,binary#48992,bool#48993,date#48994,timestamp#48995,array#48996,struct#48997,map#48998])
+- RelationV2[byte#48984, short#48985, int#48986, long#48987L, float#48988, double#48989, decimal#48990, string#48991, binary#48992, bool#48993, date#48994, timestamp#48995, array#48996, struct#48997, map#48998] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [short#48955]
+- Join Inner, (short#48955 = short#48985)
:- Filter isnotnull(short#48955)
: +- RelationV2[short#48955] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(short#48985)
+- RelationV2[short#48985] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(1945) ProjectExecTransformer [short#48955]
+- ^(1945) BroadcastHashJoinExecTransformer [short#48955], [short#48985], Inner, BuildRight, false
:- ^(1945) FilterExecTransformer isnotnull(short#48955)
: +- ^(1945) BatchScanExecTransformer[short#48955] OrcScan DataFilters: [isnotnull(short#48955)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] NativeFilters: [isnotnull(short#48955)]
+- ^(1945) InputIteratorTransformer[short#48985]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=512499]
+- ^(1944) FilterExecTransformer isnotnull(short#48985)
+- ^(1944) BatchScanExecTransformer[short#48985] OrcScan DataFilters: [isnotnull(short#48985)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] NativeFilters: [isnotnull(short#48985)]
+- == Initial Plan ==
Project [short#48955]
+- BroadcastHashJoin [short#48955], [short#48985], Inner, BuildRight, false
:- Filter isnotnull(short#48955)
: +- BatchScan[short#48955] OrcScan DataFilters: [isnotnull(short#48955)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=512290]
+- Filter isnotnull(short#48985)
+- BatchScan[short#48985] OrcScan DataFilters: [isnotnull(short#48985)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#48954]
+- Filter (byte#48954 = byte#48984)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#48954,short#48955,int#48956,long#48957L,float#48958,double#48959,decimal#48960,string#48961,binary#48962,bool#48963,date#48964,timestamp#48965,array#48966,struct#48967,map#48968])
: +- RelationV2[byte#48954, short#48955, int#48956, long#48957L, float#48958, double#48959, decimal#48960, string#48961, binary#48962, bool#48963, date#48964, timestamp#48965, array#48966, struct#48967, map#48968] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#48984,short#48985,int#48986,long#48987L,float#48988,double#48989,decimal#48990,string#48991,binary#48992,bool#48993,date#48994,timestamp#48995,array#48996,struct#48997,map#48998])
+- RelationV2[byte#48984, short#48985, int#48986, long#48987L, float#48988, double#48989, decimal#48990, string#48991, binary#48992, bool#48993, date#48994, timestamp#48995, array#48996, struct#48997, map#48998] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [byte#48954]
+- Join Inner, (byte#48954 = byte#48984)
:- Filter isnotnull(byte#48954)
: +- RelationV2[byte#48954] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(byte#48984)
+- RelationV2[byte#48984] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(1983) ProjectExecTransformer [byte#48954]
+- ^(1983) BroadcastHashJoinExecTransformer [byte#48954], [byte#48984], Inner, BuildRight, false
:- ^(1983) FilterExecTransformer isnotnull(byte#48954)
: +- ^(1983) BatchScanExecTransformer[byte#48954] OrcScan DataFilters: [isnotnull(byte#48954)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] NativeFilters: [isnotnull(byte#48954)]
+- ^(1983) InputIteratorTransformer[byte#48984]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=516525]
+- ^(1982) FilterExecTransformer isnotnull(byte#48984)
+- ^(1982) BatchScanExecTransformer[byte#48984] OrcScan DataFilters: [isnotnull(byte#48984)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] NativeFilters: [isnotnull(byte#48984)]
+- == Initial Plan ==
Project [byte#48954]
+- BroadcastHashJoin [byte#48954], [byte#48984], Inner, BuildRight, false
:- Filter isnotnull(byte#48954)
: +- BatchScan[byte#48954] OrcScan DataFilters: [isnotnull(byte#48954)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=516316]
+- Filter isnotnull(byte#48984)
+- BatchScan[byte#48984] OrcScan DataFilters: [isnotnull(byte#48984)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxOrcDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L191
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#56434]
+- Filter (int#56434 = int#56464)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#56432,short#56433,int#56434,long#56435L,float#56436,double#56437,decimal#56438,string#56439,binary#56440,bool#56441,date#56442,timestamp#56443,array#56444,struct#56445,map#56446])
: +- RelationV2[byte#56432, short#56433, int#56434, long#56435L, float#56436, double#56437, decimal#56438, string#56439, binary#56440, bool#56441, date#56442, timestamp#56443, array#56444, struct#56445, map#56446] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#56462,short#56463,int#56464,long#56465L,float#56466,double#56467,decimal#56468,string#56469,binary#56470,bool#56471,date#56472,timestamp#56473,array#56474,struct#56475,map#56476])
+- RelationV2[byte#56462, short#56463, int#56464, long#56465L, float#56466, double#56467, decimal#56468, string#56469, binary#56470, bool#56471, date#56472, timestamp#56473, array#56474, struct#56475, map#56476] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [int#56434]
+- Join Inner, (int#56434 = int#56464)
:- Filter isnotnull(int#56434)
: +- RelationV2[int#56434] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(int#56464)
+- RelationV2[int#56464] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2056) ProjectExecTransformer [int#56434]
+- ^(2056) BroadcastHashJoinExecTransformer [int#56434], [int#56464], Inner, BuildRight, false
:- ^(2056) FilterExecTransformer isnotnull(int#56434)
: +- ^(2056) BatchScanExecTransformer[int#56434] OrcScan DataFilters: [isnotnull(int#56434)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#56434)]
+- ^(2056) InputIteratorTransformer[int#56464]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=523506]
+- ^(2055) FilterExecTransformer isnotnull(int#56464)
+- ^(2055) BatchScanExecTransformer[int#56464] OrcScan DataFilters: [isnotnull(int#56464)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#56464)]
+- == Initial Plan ==
Project [int#56434]
+- BroadcastHashJoin [int#56434], [int#56464], Inner, BuildRight, false
:- Filter isnotnull(int#56434)
: +- BatchScan[int#56434] OrcScan DataFilters: [isnotnull(int#56434)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=523297]
+- Filter isnotnull(int#56464)
+- BatchScan[int#56464] OrcScan DataFilters: [isnotnull(int#56464)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxOrcDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#56433]
+- Filter (short#56433 = short#56463)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#56432,short#56433,int#56434,long#56435L,float#56436,double#56437,decimal#56438,string#56439,binary#56440,bool#56441,date#56442,timestamp#56443,array#56444,struct#56445,map#56446])
: +- RelationV2[byte#56432, short#56433, int#56434, long#56435L, float#56436, double#56437, decimal#56438, string#56439, binary#56440, bool#56441, date#56442, timestamp#56443, array#56444, struct#56445, map#56446] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#56462,short#56463,int#56464,long#56465L,float#56466,double#56467,decimal#56468,string#56469,binary#56470,bool#56471,date#56472,timestamp#56473,array#56474,struct#56475,map#56476])
+- RelationV2[byte#56462, short#56463, int#56464, long#56465L, float#56466, double#56467, decimal#56468, string#56469, binary#56470, bool#56471, date#56472, timestamp#56473, array#56474, struct#56475, map#56476] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [short#56433]
+- Join Inner, (short#56433 = short#56463)
:- Filter isnotnull(short#56433)
: +- RelationV2[short#56433] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(short#56463)
+- RelationV2[short#56463] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2065) ProjectExecTransformer [short#56433]
+- ^(2065) BroadcastHashJoinExecTransformer [short#56433], [short#56463], Inner, BuildRight, false
:- ^(2065) FilterExecTransformer isnotnull(short#56433)
: +- ^(2065) BatchScanExecTransformer[short#56433] OrcScan DataFilters: [isnotnull(short#56433)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#56433)]
+- ^(2065) InputIteratorTransformer[short#56463]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=524421]
+- ^(2064) FilterExecTransformer isnotnull(short#56463)
+- ^(2064) BatchScanExecTransformer[short#56463] OrcScan DataFilters: [isnotnull(short#56463)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#56463)]
+- == Initial Plan ==
Project [short#56433]
+- BroadcastHashJoin [short#56433], [short#56463], Inner, BuildRight, false
:- Filter isnotnull(short#56433)
: +- BatchScan[short#56433] OrcScan DataFilters: [isnotnull(short#56433)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=524212]
+- Filter isnotnull(short#56463)
+- BatchScan[short#56463] OrcScan DataFilters: [isnotnull(short#56463)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxParquetDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L190
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#57849]
+- Filter (int#57849 = int#57879)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#57847,short#57848,int#57849,long#57850L,float#57851,double#57852,decimal#57853,string#57854,binary#57855,bool#57856,date#57857,timestamp#57858,array#57859,struct#57860,map#57861])
: +- RelationV2[byte#57847, short#57848, int#57849, long#57850L, float#57851, double#57852, decimal#57853, string#57854, binary#57855, bool#57856, date#57857, timestamp#57858, array#57859, struct#57860, map#57861] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#57877,short#57878,int#57879,long#57880L,float#57881,double#57882,decimal#57883,string#57884,binary#57885,bool#57886,date#57887,timestamp#57888,array#57889,struct#57890,map#57891])
+- RelationV2[byte#57877, short#57878, int#57879, long#57880L, float#57881, double#57882, decimal#57883, string#57884, binary#57885, bool#57886, date#57887, timestamp#57888, array#57889, struct#57890, map#57891] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [int#57849]
+- Join Inner, (int#57849 = int#57879)
:- Filter isnotnull(int#57849)
: +- RelationV2[int#57849] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(int#57879)
+- RelationV2[int#57879] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2234) ProjectExecTransformer [int#57849]
+- ^(2234) BroadcastHashJoinExecTransformer [int#57849], [int#57879], Inner, BuildRight, false
:- ^(2234) FilterExecTransformer isnotnull(int#57849)
: +- ^(2234) BatchScanExecTransformer[int#57849] ParquetScan DataFilters: [isnotnull(int#57849)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#57849)]
+- ^(2234) InputIteratorTransformer[int#57879]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=541428]
+- ^(2233) FilterExecTransformer isnotnull(int#57879)
+- ^(2233) BatchScanExecTransformer[int#57879] ParquetScan DataFilters: [isnotnull(int#57879)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#57879)]
+- == Initial Plan ==
Project [int#57849]
+- BroadcastHashJoin [int#57849], [int#57879], Inner, BuildRight, false
:- Filter isnotnull(int#57849)
: +- BatchScan[int#57849] ParquetScan DataFilters: [isnotnull(int#57849)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=541219]
+- Filter isnotnull(int#57879)
+- BatchScan[int#57879] ParquetScan DataFilters: [isnotnull(int#57879)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxParquetDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#57848]
+- Filter (short#57848 = short#57878)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#57847,short#57848,int#57849,long#57850L,float#57851,double#57852,decimal#57853,string#57854,binary#57855,bool#57856,date#57857,timestamp#57858,array#57859,struct#57860,map#57861])
: +- RelationV2[byte#57847, short#57848, int#57849, long#57850L, float#57851, double#57852, decimal#57853, string#57854, binary#57855, bool#57856, date#57857, timestamp#57858, array#57859, struct#57860, map#57861] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#57877,short#57878,int#57879,long#57880L,float#57881,double#57882,decimal#57883,string#57884,binary#57885,bool#57886,date#57887,timestamp#57888,array#57889,struct#57890,map#57891])
+- RelationV2[byte#57877, short#57878, int#57879, long#57880L, float#57881, double#57882, decimal#57883, string#57884, binary#57885, bool#57886, date#57887, timestamp#57888, array#57889, struct#57890, map#57891] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [short#57848]
+- Join Inner, (short#57848 = short#57878)
:- Filter isnotnull(short#57848)
: +- RelationV2[short#57848] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(short#57878)
+- RelationV2[short#57878] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2243) ProjectExecTransformer [short#57848]
+- ^(2243) BroadcastHashJoinExecTransformer [short#57848], [short#57878], Inner, BuildRight, false
:- ^(2243) FilterExecTransformer isnotnull(short#57848)
: +- ^(2243) BatchScanExecTransformer[short#57848] ParquetScan DataFilters: [isnotnull(short#57848)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#57848)]
+- ^(2243) InputIteratorTransformer[short#57878]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=542343]
+- ^(2242) FilterExecTransformer isnotnull(short#57878)
+- ^(2242) BatchScanExecTransformer[short#57878] ParquetScan DataFilters: [isnotnull(short#57878)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#57878)]
+- == Initial Plan ==
Project [short#57848]
+- BroadcastHashJoin [short#57848], [short#57878], Inner, BuildRight, false
:- Filter isnotnull(short#57848)
: +- BatchScan[short#57848] ParquetScan DataFilters: [isnotnull(short#57848)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=542134]
+- Filter isnotnull(short#57878)
+- BatchScan[short#57878] ParquetScan DataFilters: [isnotnull(short#57878)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#57572]
+- Filter (byte#57572 = byte#57602)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#57572,short#57573,int#57574,long#57575L,float#57576,double#57577,decimal#57578,string#57579,binary#57580,bool#57581,date#57582,timestamp#57583,array#57584,struct#57585,map#57586])
: +- RelationV2[byte#57572, short#57573, int#57574, long#57575L, float#57576, double#57577, decimal#57578, string#57579, binary#57580, bool#57581, date#57582, timestamp#57583, array#57584, struct#57585, map#57586] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#57602,short#57603,int#57604,long#57605L,float#57606,double#57607,decimal#57608,string#57609,binary#57610,bool#57611,date#57612,timestamp#57613,array#57614,struct#57615,map#57616])
+- RelationV2[byte#57602, short#57603, int#57604, long#57605L, float#57606, double#57607, decimal#57608, string#57609, binary#57610, bool#57611, date#57612, timestamp#57613, array#57614, struct#57615, map#57616] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [byte#57572]
+- Join Inner, (byte#57572 = byte#57602)
:- Filter isnotnull(byte#57572)
: +- RelationV2[byte#57572] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(byte#57602)
+- RelationV2[byte#57602] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2090) ProjectExecTransformer [byte#57572]
+- ^(2090) BroadcastHashJoinExecTransformer [byte#57572], [byte#57602], Inner, BuildRight, false
:- ^(2090) FilterExecTransformer isnotnull(byte#57572)
: +- ^(2090) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#57572] OrcScan DataFilters: [isnotnull(byte#57572)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#57572)]
+- ^(2090) InputIteratorTransformer[byte#57602]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=502873]
+- ^(2089) FilterExecTransformer isnotnull(byte#57602)
+- ^(2089) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#57602] OrcScan DataFilters: [isnotnull(byte#57602)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#57602)]
+- == Initial Plan ==
Project [byte#57572]
+- BroadcastHashJoin [byte#57572], [byte#57602], Inner, BuildRight, false
:- Filter isnotnull(byte#57572)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#57572] OrcScan DataFilters: [isnotnull(byte#57572)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=502664]
+- Filter isnotnull(byte#57602)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#57602] OrcScan DataFilters: [isnotnull(byte#57602)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxParquetDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#61244]
+- Filter (int#61244 = int#61274)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#61242,short#61243,int#61244,long#61245L,float#61246,double#61247,decimal#61248,string#61249,binary#61250,bool#61251,date#61252,timestamp#61253,array#61254,struct#61255,map#61256])
: +- RelationV2[byte#61242, short#61243, int#61244, long#61245L, float#61246, double#61247, decimal#61248, string#61249, binary#61250, bool#61251, date#61252, timestamp#61253, array#61254, struct#61255, map#61256] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#61272,short#61273,int#61274,long#61275L,float#61276,double#61277,decimal#61278,string#61279,binary#61280,bool#61281,date#61282,timestamp#61283,array#61284,struct#61285,map#61286])
+- RelationV2[byte#61272, short#61273, int#61274, long#61275L, float#61276, double#61277, decimal#61278, string#61279, binary#61280, bool#61281, date#61282, timestamp#61283, array#61284, struct#61285, map#61286] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [int#61244]
+- Join Inner, (int#61244 = int#61274)
:- Filter isnotnull(int#61244)
: +- RelationV2[int#61244] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(int#61274)
+- RelationV2[int#61274] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2213) ProjectExecTransformer [int#61244]
+- ^(2213) BroadcastHashJoinExecTransformer [int#61244], [int#61274], Inner, BuildRight, false
:- ^(2213) FilterExecTransformer isnotnull(int#61244)
: +- ^(2213) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#61244] ParquetScan DataFilters: [isnotnull(int#61244)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#61244)]
+- ^(2213) InputIteratorTransformer[int#61274]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=515309]
+- ^(2212) FilterExecTransformer isnotnull(int#61274)
+- ^(2212) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#61274] ParquetScan DataFilters: [isnotnull(int#61274)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#61274)]
+- == Initial Plan ==
Project [int#61244]
+- BroadcastHashJoin [int#61244], [int#61274], Inner, BuildRight, false
:- Filter isnotnull(int#61244)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#61244] ParquetScan DataFilters: [isnotnull(int#61244)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=515100]
+- Filter isnotnull(int#61274)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#61274] ParquetScan DataFilters: [isnotnull(int#61274)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxParquetDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L217
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#61243]
+- Filter (short#61243 = short#61273)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#61242,short#61243,int#61244,long#61245L,float#61246,double#61247,decimal#61248,string#61249,binary#61250,bool#61251,date#61252,timestamp#61253,array#61254,struct#61255,map#61256])
: +- RelationV2[byte#61242, short#61243, int#61244, long#61245L, float#61246, double#61247, decimal#61248, string#61249, binary#61250, bool#61251, date#61252, timestamp#61253, array#61254, struct#61255, map#61256] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#61272,short#61273,int#61274,long#61275L,float#61276,double#61277,decimal#61278,string#61279,binary#61280,bool#61281,date#61282,timestamp#61283,array#61284,struct#61285,map#61286])
+- RelationV2[byte#61272, short#61273, int#61274, long#61275L, float#61276, double#61277, decimal#61278, string#61279, binary#61280, bool#61281, date#61282, timestamp#61283, array#61284, struct#61285, map#61286] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [short#61243]
+- Join Inner, (short#61243 = short#61273)
:- Filter isnotnull(short#61243)
: +- RelationV2[short#61243] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(short#61273)
+- RelationV2[short#61273] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2226) ProjectExecTransformer [short#61243]
+- ^(2226) BroadcastHashJoinExecTransformer [short#61243], [short#61273], Inner, BuildRight, false
:- ^(2226) FilterExecTransformer isnotnull(short#61243)
: +- ^(2226) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#61243] ParquetScan DataFilters: [isnotnull(short#61243)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#61243)]
+- ^(2226) InputIteratorTransformer[short#61273]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=516612]
+- ^(2225) FilterExecTransformer isnotnull(short#61273)
+- ^(2225) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#61273] ParquetScan DataFilters: [isnotnull(short#61273)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#61273)]
+- == Initial Plan ==
Project [short#61243]
+- BroadcastHashJoin [short#61243], [short#61273], Inner, BuildRight, false
:- Filter isnotnull(short#61243)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#61243] ParquetScan DataFilters: [isnotnull(short#61243)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=516403]
+- Filter isnotnull(short#61273)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#61273] ParquetScan DataFilters: [isnotnull(short#61273)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxParquetDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L327
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#61242]
+- Filter (byte#61242 = byte#61272)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#61242,short#61243,int#61244,long#61245L,float#61246,double#61247,decimal#61248,string#61249,binary#61250,bool#61251,date#61252,timestamp#61253,array#61254,struct#61255,map#61256])
: +- RelationV2[byte#61242, short#61243, int#61244, long#61245L, float#61246, double#61247, decimal#61248, string#61249, binary#61250, bool#61251, date#61252, timestamp#61253, array#61254, struct#61255, map#61256] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#61272,short#61273,int#61274,long#61275L,float#61276,double#61277,decimal#61278,string#61279,binary#61280,bool#61281,date#61282,timestamp#61283,array#61284,struct#61285,map#61286])
+- RelationV2[byte#61272, short#61273, int#61274, long#61275L, float#61276, double#61277, decimal#61278, string#61279, binary#61280, bool#61281, date#61282, timestamp#61283, array#61284, struct#61285, map#61286] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [byte#61242]
+- Join Inner, (byte#61242 = byte#61272)
:- Filter isnotnull(byte#61242)
: +- RelationV2[byte#61242] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(byte#61272)
+- RelationV2[byte#61272] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2268) ProjectExecTransformer [byte#61242]
+- ^(2268) BroadcastHashJoinExecTransformer [byte#61242], [byte#61272], Inner, BuildRight, false
:- ^(2268) FilterExecTransformer isnotnull(byte#61242)
: +- ^(2268) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#61242] ParquetScan DataFilters: [isnotnull(byte#61242)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#61242)]
+- ^(2268) InputIteratorTransformer[byte#61272]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=521026]
+- ^(2267) FilterExecTransformer isnotnull(byte#61272)
+- ^(2267) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#61272] ParquetScan DataFilters: [isnotnull(byte#61272)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#61272)]
+- == Initial Plan ==
Project [byte#61242]
+- BroadcastHashJoin [byte#61242], [byte#61272], Inner, BuildRight, false
:- Filter isnotnull(byte#61242)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#61242] ParquetScan DataFilters: [isnotnull(byte#61242)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=520817]
+- Filter isnotnull(byte#61272)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#61272] ParquetScan DataFilters: [isnotnull(byte#61272)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxOrcDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#60984]
+- Filter (short#60984 = short#61014)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#60983,short#60984,int#60985,long#60986L,float#60987,double#60988,decimal#60989,string#60990,binary#60991,bool#60992,date#60993,timestamp#60994,array#60995,struct#60996,map#60997])
: +- RelationV2[byte#60983, short#60984, int#60985, long#60986L, float#60987, double#60988, decimal#60989, string#60990, binary#60991, bool#60992, date#60993, timestamp#60994, array#60995, struct#60996, map#60997] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#61013,short#61014,int#61015,long#61016L,float#61017,double#61018,decimal#61019,string#61020,binary#61021,bool#61022,date#61023,timestamp#61024,array#61025,struct#61026,map#61027])
+- RelationV2[byte#61013, short#61014, int#61015, long#61016L, float#61017, double#61018, decimal#61019, string#61020, binary#61021, bool#61022, date#61023, timestamp#61024, array#61025, struct#61026, map#61027] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [short#60984]
+- Join Inner, (short#60984 = short#61014)
:- Filter isnotnull(short#60984)
: +- RelationV2[short#60984] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(short#61014)
+- RelationV2[short#61014] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2166) ProjectExecTransformer [short#60984]
+- ^(2166) BroadcastHashJoinExecTransformer [short#60984], [short#61014], Inner, BuildRight, false
:- ^(2166) FilterExecTransformer isnotnull(short#60984)
: +- ^(2166) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#60984] OrcScan DataFilters: [isnotnull(short#60984)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#60984)]
+- ^(2166) InputIteratorTransformer[short#61014]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=542707]
+- ^(2165) FilterExecTransformer isnotnull(short#61014)
+- ^(2165) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#61014] OrcScan DataFilters: [isnotnull(short#61014)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#61014)]
+- == Initial Plan ==
Project [short#60984]
+- BroadcastHashJoin [short#60984], [short#61014], Inner, BuildRight, false
:- Filter isnotnull(short#60984)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#60984] OrcScan DataFilters: [isnotnull(short#60984)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=542498]
+- Filter isnotnull(short#61014)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#61014] OrcScan DataFilters: [isnotnull(short#61014)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L328
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#60983]
+- Filter (byte#60983 = byte#61013)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#60983,short#60984,int#60985,long#60986L,float#60987,double#60988,decimal#60989,string#60990,binary#60991,bool#60992,date#60993,timestamp#60994,array#60995,struct#60996,map#60997])
: +- RelationV2[byte#60983, short#60984, int#60985, long#60986L, float#60987, double#60988, decimal#60989, string#60990, binary#60991, bool#60992, date#60993, timestamp#60994, array#60995, struct#60996, map#60997] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#61013,short#61014,int#61015,long#61016L,float#61017,double#61018,decimal#61019,string#61020,binary#61021,bool#61022,date#61023,timestamp#61024,array#61025,struct#61026,map#61027])
+- RelationV2[byte#61013, short#61014, int#61015, long#61016L, float#61017, double#61018, decimal#61019, string#61020, binary#61021, bool#61022, date#61023, timestamp#61024, array#61025, struct#61026, map#61027] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [byte#60983]
+- Join Inner, (byte#60983 = byte#61013)
:- Filter isnotnull(byte#60983)
: +- RelationV2[byte#60983] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(byte#61013)
+- RelationV2[byte#61013] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2208) ProjectExecTransformer [byte#60983]
+- ^(2208) BroadcastHashJoinExecTransformer [byte#60983], [byte#61013], Inner, BuildRight, false
:- ^(2208) FilterExecTransformer isnotnull(byte#60983)
: +- ^(2208) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#60983] OrcScan DataFilters: [isnotnull(byte#60983)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#60983)]
+- ^(2208) InputIteratorTransformer[byte#61013]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=547121]
+- ^(2207) FilterExecTransformer isnotnull(byte#61013)
+- ^(2207) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#61013] OrcScan DataFilters: [isnotnull(byte#61013)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#61013)]
+- == Initial Plan ==
Project [byte#60983]
+- BroadcastHashJoin [byte#60983], [byte#61013], Inner, BuildRight, false
:- Filter isnotnull(byte#60983)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#60983] OrcScan DataFilters: [isnotnull(byte#60983)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=546912]
+- Filter isnotnull(byte#61013)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#61013] OrcScan DataFilters: [isnotnull(byte#61013)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxParquetDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#64655]
+- Filter (int#64655 = int#64685)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#64653,short#64654,int#64655,long#64656L,float#64657,double#64658,decimal#64659,string#64660,binary#64661,bool#64662,date#64663,timestamp#64664,array#64665,struct#64666,map#64667])
: +- RelationV2[byte#64653, short#64654, int#64655, long#64656L, float#64657, double#64658, decimal#64659, string#64660, binary#64661, bool#64662, date#64663, timestamp#64664, array#64665, struct#64666, map#64667] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#64683,short#64684,int#64685,long#64686L,float#64687,double#64688,decimal#64689,string#64690,binary#64691,bool#64692,date#64693,timestamp#64694,array#64695,struct#64696,map#64697])
+- RelationV2[byte#64683, short#64684, int#64685, long#64686L, float#64687, double#64688, decimal#64689, string#64690, binary#64691, bool#64692, date#64693, timestamp#64694, array#64695, struct#64696, map#64697] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [int#64655]
+- Join Inner, (int#64655 = int#64685)
:- Filter isnotnull(int#64655)
: +- RelationV2[int#64655] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(int#64685)
+- RelationV2[int#64685] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2331) ProjectExecTransformer [int#64655]
+- ^(2331) BroadcastHashJoinExecTransformer [int#64655], [int#64685], Inner, BuildRight, false
:- ^(2331) FilterExecTransformer isnotnull(int#64655)
: +- ^(2331) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#64655] ParquetScan DataFilters: [isnotnull(int#64655)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#64655)]
+- ^(2331) InputIteratorTransformer[int#64685]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=559557]
+- ^(2330) FilterExecTransformer isnotnull(int#64685)
+- ^(2330) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#64685] ParquetScan DataFilters: [isnotnull(int#64685)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#64685)]
+- == Initial Plan ==
Project [int#64655]
+- BroadcastHashJoin [int#64655], [int#64685], Inner, BuildRight, false
:- Filter isnotnull(int#64655)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#64655] ParquetScan DataFilters: [isnotnull(int#64655)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=559348]
+- Filter isnotnull(int#64685)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#64685] ParquetScan DataFilters: [isnotnull(int#64685)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxParquetDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#64653]
+- Filter (byte#64653 = byte#64683)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#64653,short#64654,int#64655,long#64656L,float#64657,double#64658,decimal#64659,string#64660,binary#64661,bool#64662,date#64663,timestamp#64664,array#64665,struct#64666,map#64667])
: +- RelationV2[byte#64653, short#64654, int#64655, long#64656L, float#64657, double#64658, decimal#64659, string#64660, binary#64661, bool#64662, date#64663, timestamp#64664, array#64665, struct#64666, map#64667] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#64683,short#64684,int#64685,long#64686L,float#64687,double#64688,decimal#64689,string#64690,binary#64691,bool#64692,date#64693,timestamp#64694,array#64695,struct#64696,map#64697])
+- RelationV2[byte#64683, short#64684, int#64685, long#64686L, float#64687, double#64688, decimal#64689, string#64690, binary#64691, bool#64692, date#64693, timestamp#64694, array#64695, struct#64696, map#64697] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [byte#64653]
+- Join Inner, (byte#64653 = byte#64683)
:- Filter isnotnull(byte#64653)
: +- RelationV2[byte#64653] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(byte#64683)
+- RelationV2[byte#64683] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2382) ProjectExecTransformer [byte#64653]
+- ^(2382) BroadcastHashJoinExecTransformer [byte#64653], [byte#64683], Inner, BuildRight, false
:- ^(2382) FilterExecTransformer isnotnull(byte#64653)
: +- ^(2382) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#64653] ParquetScan DataFilters: [isnotnull(byte#64653)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#64653)]
+- ^(2382) InputIteratorTransformer[byte#64683]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=564886]
+- ^(2381) FilterExecTransformer isnotnull(byte#64683)
+- ^(2381) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#64683] ParquetScan DataFilters: [isnotnull(byte#64683)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#64683)]
+- == Initial Plan ==
Project [byte#64653]
+- BroadcastHashJoin [byte#64653], [byte#64683], Inner, BuildRight, false
:- Filter isnotnull(byte#64653)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#64653] ParquetScan DataFilters: [isnotnull(byte#64653)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=564677]
+- Filter isnotnull(byte#64683)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#64683] ParquetScan DataFilters: [isnotnull(byte#64683)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxOrcDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L191
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#73755]
+- Filter (int#73755 = int#73785)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73753,short#73754,int#73755,long#73756L,float#73757,double#73758,decimal#73759,string#73760,binary#73761,bool#73762,date#73763,timestamp#73764,array#73765,struct#73766,map#73767])
: +- RelationV2[byte#73753, short#73754, int#73755, long#73756L, float#73757, double#73758, decimal#73759, string#73760, binary#73761, bool#73762, date#73763, timestamp#73764, array#73765, struct#73766, map#73767] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73783,short#73784,int#73785,long#73786L,float#73787,double#73788,decimal#73789,string#73790,binary#73791,bool#73792,date#73793,timestamp#73794,array#73795,struct#73796,map#73797])
+- RelationV2[byte#73783, short#73784, int#73785, long#73786L, float#73787, double#73788, decimal#73789, string#73790, binary#73791, bool#73792, date#73793, timestamp#73794, array#73795, struct#73796, map#73797] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [int#73755]
+- Join Inner, (int#73755 = int#73785)
:- Filter isnotnull(int#73755)
: +- RelationV2[int#73755] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(int#73785)
+- RelationV2[int#73785] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2382) ProjectExecTransformer [int#73755]
+- ^(2382) BroadcastHashJoinExecTransformer [int#73755], [int#73785], Inner, BuildRight, false
:- ^(2382) FilterExecTransformer isnotnull(int#73755)
: +- ^(2382) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#73755] OrcScan DataFilters: [isnotnull(int#73755)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73755)]
+- ^(2382) InputIteratorTransformer[int#73785]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=550361]
+- ^(2381) FilterExecTransformer isnotnull(int#73785)
+- ^(2381) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#73785] OrcScan DataFilters: [isnotnull(int#73785)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73785)]
+- == Initial Plan ==
Project [int#73755]
+- BroadcastHashJoin [int#73755], [int#73785], Inner, BuildRight, false
:- Filter isnotnull(int#73755)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#73755] OrcScan DataFilters: [isnotnull(int#73755)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=550152]
+- Filter isnotnull(int#73785)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#73785] OrcScan DataFilters: [isnotnull(int#73785)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxOrcDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L218
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#73754]
+- Filter (short#73754 = short#73784)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73753,short#73754,int#73755,long#73756L,float#73757,double#73758,decimal#73759,string#73760,binary#73761,bool#73762,date#73763,timestamp#73764,array#73765,struct#73766,map#73767])
: +- RelationV2[byte#73753, short#73754, int#73755, long#73756L, float#73757, double#73758, decimal#73759, string#73760, binary#73761, bool#73762, date#73763, timestamp#73764, array#73765, struct#73766, map#73767] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73783,short#73784,int#73785,long#73786L,float#73787,double#73788,decimal#73789,string#73790,binary#73791,bool#73792,date#73793,timestamp#73794,array#73795,struct#73796,map#73797])
+- RelationV2[byte#73783, short#73784, int#73785, long#73786L, float#73787, double#73788, decimal#73789, string#73790, binary#73791, bool#73792, date#73793, timestamp#73794, array#73795, struct#73796, map#73797] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [short#73754]
+- Join Inner, (short#73754 = short#73784)
:- Filter isnotnull(short#73754)
: +- RelationV2[short#73754] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(short#73784)
+- RelationV2[short#73784] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2395) ProjectExecTransformer [short#73754]
+- ^(2395) BroadcastHashJoinExecTransformer [short#73754], [short#73784], Inner, BuildRight, false
:- ^(2395) FilterExecTransformer isnotnull(short#73754)
: +- ^(2395) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73754] OrcScan DataFilters: [isnotnull(short#73754)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73754)]
+- ^(2395) InputIteratorTransformer[short#73784]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=551664]
+- ^(2394) FilterExecTransformer isnotnull(short#73784)
+- ^(2394) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73784] OrcScan DataFilters: [isnotnull(short#73784)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73784)]
+- == Initial Plan ==
Project [short#73754]
+- BroadcastHashJoin [short#73754], [short#73784], Inner, BuildRight, false
:- Filter isnotnull(short#73754)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73754] OrcScan DataFilters: [isnotnull(short#73754)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=551455]
+- Filter isnotnull(short#73784)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73784] OrcScan DataFilters: [isnotnull(short#73784)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L328
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#73753]
+- Filter (byte#73753 = byte#73783)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73753,short#73754,int#73755,long#73756L,float#73757,double#73758,decimal#73759,string#73760,binary#73761,bool#73762,date#73763,timestamp#73764,array#73765,struct#73766,map#73767])
: +- RelationV2[byte#73753, short#73754, int#73755, long#73756L, float#73757, double#73758, decimal#73759, string#73760, binary#73761, bool#73762, date#73763, timestamp#73764, array#73765, struct#73766, map#73767] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73783,short#73784,int#73785,long#73786L,float#73787,double#73788,decimal#73789,string#73790,binary#73791,bool#73792,date#73793,timestamp#73794,array#73795,struct#73796,map#73797])
+- RelationV2[byte#73783, short#73784, int#73785, long#73786L, float#73787, double#73788, decimal#73789, string#73790, binary#73791, bool#73792, date#73793, timestamp#73794, array#73795, struct#73796, map#73797] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [byte#73753]
+- Join Inner, (byte#73753 = byte#73783)
:- Filter isnotnull(byte#73753)
: +- RelationV2[byte#73753] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(byte#73783)
+- RelationV2[byte#73783] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2449) ProjectExecTransformer [byte#73753]
+- ^(2449) BroadcastHashJoinExecTransformer [byte#73753], [byte#73783], Inner, BuildRight, false
:- ^(2449) FilterExecTransformer isnotnull(byte#73753)
: +- ^(2449) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73753] OrcScan DataFilters: [isnotnull(byte#73753)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73753)]
+- ^(2449) InputIteratorTransformer[byte#73783]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=556688]
+- ^(2448) FilterExecTransformer isnotnull(byte#73783)
+- ^(2448) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73783] OrcScan DataFilters: [isnotnull(byte#73783)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73783)]
+- == Initial Plan ==
Project [byte#73753]
+- BroadcastHashJoin [byte#73753], [byte#73783], Inner, BuildRight, false
:- Filter isnotnull(byte#73753)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73753] OrcScan DataFilters: [isnotnull(byte#73753)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=556479]
+- Filter isnotnull(byte#73783)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73783] OrcScan DataFilters: [isnotnull(byte#73783)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxParquetDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L190
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#77734]
+- Filter (int#77734 = int#77764)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#77732,short#77733,int#77734,long#77735L,float#77736,double#77737,decimal#77738,string#77739,binary#77740,bool#77741,date#77742,timestamp#77743,array#77744,struct#77745,map#77746])
: +- RelationV2[byte#77732, short#77733, int#77734, long#77735L, float#77736, double#77737, decimal#77738, string#77739, binary#77740, bool#77741, date#77742, timestamp#77743, array#77744, struct#77745, map#77746] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#77762,short#77763,int#77764,long#77765L,float#77766,double#77767,decimal#77768,string#77769,binary#77770,bool#77771,date#77772,timestamp#77773,array#77774,struct#77775,map#77776])
+- RelationV2[byte#77762, short#77763, int#77764, long#77765L, float#77766, double#77767, decimal#77768, string#77769, binary#77770, bool#77771, date#77772, timestamp#77773, array#77774, struct#77775, map#77776] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [int#77734]
+- Join Inner, (int#77734 = int#77764)
:- Filter isnotnull(int#77734)
: +- RelationV2[int#77734] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(int#77764)
+- RelationV2[int#77764] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2578) ProjectExecTransformer [int#77734]
+- ^(2578) BroadcastHashJoinExecTransformer [int#77734], [int#77764], Inner, BuildRight, false
:- ^(2578) FilterExecTransformer isnotnull(int#77734)
: +- ^(2578) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77734] ParquetScan DataFilters: [isnotnull(int#77734)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77734)]
+- ^(2578) InputIteratorTransformer[int#77764]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=569640]
+- ^(2577) FilterExecTransformer isnotnull(int#77764)
+- ^(2577) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77764] ParquetScan DataFilters: [isnotnull(int#77764)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77764)]
+- == Initial Plan ==
Project [int#77734]
+- BroadcastHashJoin [int#77734], [int#77764], Inner, BuildRight, false
:- Filter isnotnull(int#77734)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77734] ParquetScan DataFilters: [isnotnull(int#77734)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=569431]
+- Filter isnotnull(int#77764)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77764] ParquetScan DataFilters: [isnotnull(int#77764)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxParquetDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#77733]
+- Filter (short#77733 = short#77763)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#77732,short#77733,int#77734,long#77735L,float#77736,double#77737,decimal#77738,string#77739,binary#77740,bool#77741,date#77742,timestamp#77743,array#77744,struct#77745,map#77746])
: +- RelationV2[byte#77732, short#77733, int#77734, long#77735L, float#77736, double#77737, decimal#77738, string#77739, binary#77740, bool#77741, date#77742, timestamp#77743, array#77744, struct#77745, map#77746] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#77762,short#77763,int#77764,long#77765L,float#77766,double#77767,decimal#77768,string#77769,binary#77770,bool#77771,date#77772,timestamp#77773,array#77774,struct#77775,map#77776])
+- RelationV2[byte#77762, short#77763, int#77764, long#77765L, float#77766, double#77767, decimal#77768, string#77769, binary#77770, bool#77771, date#77772, timestamp#77773, array#77774, struct#77775, map#77776] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [short#77733]
+- Join Inner, (short#77733 = short#77763)
:- Filter isnotnull(short#77733)
: +- RelationV2[short#77733] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(short#77763)
+- RelationV2[short#77763] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2587) ProjectExecTransformer [short#77733]
+- ^(2587) BroadcastHashJoinExecTransformer [short#77733], [short#77763], Inner, BuildRight, false
:- ^(2587) FilterExecTransformer isnotnull(short#77733)
: +- ^(2587) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77733] ParquetScan DataFilters: [isnotnull(short#77733)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77733)]
+- ^(2587) InputIteratorTransformer[short#77763]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=570555]
+- ^(2586) FilterExecTransformer isnotnull(short#77763)
+- ^(2586) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77763] ParquetScan DataFilters: [isnotnull(short#77763)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77763)]
+- == Initial Plan ==
Project [short#77733]
+- BroadcastHashJoin [short#77733], [short#77763], Inner, BuildRight, false
:- Filter isnotnull(short#77733)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77733] ParquetScan DataFilters: [isnotnull(short#77733)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=570346]
+- Filter isnotnull(short#77763)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77763] ParquetScan DataFilters: [isnotnull(short#77763)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxParquetDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#77732]
+- Filter (byte#77732 = byte#77762)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#77732,short#77733,int#77734,long#77735L,float#77736,double#77737,decimal#77738,string#77739,binary#77740,bool#77741,date#77742,timestamp#77743,array#77744,struct#77745,map#77746])
: +- RelationV2[byte#77732, short#77733, int#77734, long#77735L, float#77736, double#77737, decimal#77738, string#77739, binary#77740, bool#77741, date#77742, timestamp#77743, array#77744, struct#77745, map#77746] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#77762,short#77763,int#77764,long#77765L,float#77766,double#77767,decimal#77768,string#77769,binary#77770,bool#77771,date#77772,timestamp#77773,array#77774,struct#77775,map#77776])
+- RelationV2[byte#77762, short#77763, int#77764, long#77765L, float#77766, double#77767, decimal#77768, string#77769, binary#77770, bool#77771, date#77772, timestamp#77773, array#77774, struct#77775, map#77776] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [byte#77732]
+- Join Inner, (byte#77732 = byte#77762)
:- Filter isnotnull(byte#77732)
: +- RelationV2[byte#77732] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(byte#77762)
+- RelationV2[byte#77762] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2637) ProjectExecTransformer [byte#77732]
+- ^(2637) BroadcastHashJoinExecTransformer [byte#77732], [byte#77762], Inner, BuildRight, false
:- ^(2637) FilterExecTransformer isnotnull(byte#77732)
: +- ^(2637) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77732] ParquetScan DataFilters: [isnotnull(byte#77732)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77732)]
+- ^(2637) InputIteratorTransformer[byte#77762]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=575205]
+- ^(2636) FilterExecTransformer isnotnull(byte#77762)
+- ^(2636) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77762] ParquetScan DataFilters: [isnotnull(byte#77762)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77762)]
+- == Initial Plan ==
Project [byte#77732]
+- BroadcastHashJoin [byte#77732], [byte#77762], Inner, BuildRight, false
:- Filter isnotnull(byte#77732)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77732] ParquetScan DataFilters: [isnotnull(byte#77732)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=574996]
+- Filter isnotnull(byte#77762)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77762] ParquetScan DataFilters: [isnotnull(byte#77762)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxOrcDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#73734]
+- Filter (short#73734 = short#73764)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73733,short#73734,int#73735,long#73736L,float#73737,double#73738,decimal#73739,string#73740,binary#73741,bool#73742,date#73743,timestamp#73744,array#73745,struct#73746,map#73747])
: +- RelationV2[byte#73733, short#73734, int#73735, long#73736L, float#73737, double#73738, decimal#73739, string#73740, binary#73741, bool#73742, date#73743, timestamp#73744, array#73745, struct#73746, map#73747] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73763,short#73764,int#73765,long#73766L,float#73767,double#73768,decimal#73769,string#73770,binary#73771,bool#73772,date#73773,timestamp#73774,array#73775,struct#73776,map#73777])
+- RelationV2[byte#73763, short#73764, int#73765, long#73766L, float#73767, double#73768, decimal#73769, string#73770, binary#73771, bool#73772, date#73773, timestamp#73774, array#73775, struct#73776, map#73777] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [short#73734]
+- Join Inner, (short#73734 = short#73764)
:- Filter isnotnull(short#73734)
: +- RelationV2[short#73734] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(short#73764)
+- RelationV2[short#73764] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2390) ProjectExecTransformer [short#73734]
+- ^(2390) BroadcastHashJoinExecTransformer [short#73734], [short#73764], Inner, BuildRight, false
:- ^(2390) FilterExecTransformer isnotnull(short#73734)
: +- ^(2390) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73734] OrcScan DataFilters: [isnotnull(short#73734)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73734)]
+- ^(2390) InputIteratorTransformer[short#73764]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=529660]
+- ^(2389) FilterExecTransformer isnotnull(short#73764)
+- ^(2389) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73764] OrcScan DataFilters: [isnotnull(short#73764)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73764)]
+- == Initial Plan ==
Project [short#73734]
+- BroadcastHashJoin [short#73734], [short#73764], Inner, BuildRight, false
:- Filter isnotnull(short#73734)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73734] OrcScan DataFilters: [isnotnull(short#73734)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=529451]
+- Filter isnotnull(short#73764)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73764] OrcScan DataFilters: [isnotnull(short#73764)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#73733]
+- Filter (byte#73733 = byte#73763)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73733,short#73734,int#73735,long#73736L,float#73737,double#73738,decimal#73739,string#73740,binary#73741,bool#73742,date#73743,timestamp#73744,array#73745,struct#73746,map#73747])
: +- RelationV2[byte#73733, short#73734, int#73735, long#73736L, float#73737, double#73738, decimal#73739, string#73740, binary#73741, bool#73742, date#73743, timestamp#73744, array#73745, struct#73746, map#73747] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73763,short#73764,int#73765,long#73766L,float#73767,double#73768,decimal#73769,string#73770,binary#73771,bool#73772,date#73773,timestamp#73774,array#73775,struct#73776,map#73777])
+- RelationV2[byte#73763, short#73764, int#73765, long#73766L, float#73767, double#73768, decimal#73769, string#73770, binary#73771, bool#73772, date#73773, timestamp#73774, array#73775, struct#73776, map#73777] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [byte#73733]
+- Join Inner, (byte#73733 = byte#73763)
:- Filter isnotnull(byte#73733)
: +- RelationV2[byte#73733] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(byte#73763)
+- RelationV2[byte#73763] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2440) ProjectExecTransformer [byte#73733]
+- ^(2440) BroadcastHashJoinExecTransformer [byte#73733], [byte#73763], Inner, BuildRight, false
:- ^(2440) FilterExecTransformer isnotnull(byte#73733)
: +- ^(2440) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73733] OrcScan DataFilters: [isnotnull(byte#73733)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73733)]
+- ^(2440) InputIteratorTransformer[byte#73763]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=534310]
+- ^(2439) FilterExecTransformer isnotnull(byte#73763)
+- ^(2439) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73763] OrcScan DataFilters: [isnotnull(byte#73763)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73763)]
+- == Initial Plan ==
Project [byte#73733]
+- BroadcastHashJoin [byte#73733], [byte#73763], Inner, BuildRight, false
:- Filter isnotnull(byte#73733)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73733] OrcScan DataFilters: [isnotnull(byte#73733)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=534101]
+- Filter isnotnull(byte#73763)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73763] OrcScan DataFilters: [isnotnull(byte#73763)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxParquetDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#77504]
+- Filter (int#77504 = int#77534)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#77502,short#77503,int#77504,long#77505L,float#77506,double#77507,decimal#77508,string#77509,binary#77510,bool#77511,date#77512,timestamp#77513,array#77514,struct#77515,map#77516])
: +- RelationV2[byte#77502, short#77503, int#77504, long#77505L, float#77506, double#77507, decimal#77508, string#77509, binary#77510, bool#77511, date#77512, timestamp#77513, array#77514, struct#77515, map#77516] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#77532,short#77533,int#77534,long#77535L,float#77536,double#77537,decimal#77538,string#77539,binary#77540,bool#77541,date#77542,timestamp#77543,array#77544,struct#77545,map#77546])
+- RelationV2[byte#77532, short#77533, int#77534, long#77535L, float#77536, double#77537, decimal#77538, string#77539, binary#77540, bool#77541, date#77542, timestamp#77543, array#77544, struct#77545, map#77546] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [int#77504]
+- Join Inner, (int#77504 = int#77534)
:- Filter isnotnull(int#77504)
: +- RelationV2[int#77504] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(int#77534)
+- RelationV2[int#77534] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2565) ProjectExecTransformer [int#77504]
+- ^(2565) BroadcastHashJoinExecTransformer [int#77504], [int#77534], Inner, BuildRight, false
:- ^(2565) FilterExecTransformer isnotnull(int#77504)
: +- ^(2565) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77504] ParquetScan DataFilters: [isnotnull(int#77504)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77504)]
+- ^(2565) InputIteratorTransformer[int#77534]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=546874]
+- ^(2564) FilterExecTransformer isnotnull(int#77534)
+- ^(2564) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77534] ParquetScan DataFilters: [isnotnull(int#77534)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77534)]
+- == Initial Plan ==
Project [int#77504]
+- BroadcastHashJoin [int#77504], [int#77534], Inner, BuildRight, false
:- Filter isnotnull(int#77504)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77504] ParquetScan DataFilters: [isnotnull(int#77504)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=546665]
+- Filter isnotnull(int#77534)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77534] ParquetScan DataFilters: [isnotnull(int#77534)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxParquetDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#77502]
+- Filter (byte#77502 = byte#77532)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#77502,short#77503,int#77504,long#77505L,float#77506,double#77507,decimal#77508,string#77509,binary#77510,bool#77511,date#77512,timestamp#77513,array#77514,struct#77515,map#77516])
: +- RelationV2[byte#77502, short#77503, int#77504, long#77505L, float#77506, double#77507, decimal#77508, string#77509, binary#77510, bool#77511, date#77512, timestamp#77513, array#77514, struct#77515, map#77516] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#77532,short#77533,int#77534,long#77535L,float#77536,double#77537,decimal#77538,string#77539,binary#77540,bool#77541,date#77542,timestamp#77543,array#77544,struct#77545,map#77546])
+- RelationV2[byte#77532, short#77533, int#77534, long#77535L, float#77536, double#77537, decimal#77538, string#77539, binary#77540, bool#77541, date#77542, timestamp#77543, array#77544, struct#77545, map#77546] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [byte#77502]
+- Join Inner, (byte#77502 = byte#77532)
:- Filter isnotnull(byte#77502)
: +- RelationV2[byte#77502] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1
+- Filter isnotnull(byte#77532)
+- RelationV2[byte#77532] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2628) ProjectExecTransformer [byte#77502]
+- ^(2628) BroadcastHashJoinExecTransformer [byte#77502], [byte#77532], Inner, BuildRight, false
:- ^(2628) FilterExecTransformer isnotnull(byte#77502)
: +- ^(2628) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77502] ParquetScan DataFilters: [isnotnull(byte#77502)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77502)]
+- ^(2628) InputIteratorTransformer[byte#77532]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=552827]
+- ^(2627) FilterExecTransformer isnotnull(byte#77532)
+- ^(2627) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77532] ParquetScan DataFilters: [isnotnull(byte#77532)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77532)]
+- == Initial Plan ==
Project [byte#77502]
+- BroadcastHashJoin [byte#77502], [byte#77532], Inner, BuildRight, false
:- Filter isnotnull(byte#77502)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77502] ParquetScan DataFilters: [isnotnull(byte#77502)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=552618]
+- Filter isnotnull(byte#77532)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77532] ParquetScan DataFilters: [isnotnull(byte#77532)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxOrcDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L218
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#73209]
+- Filter (short#73209 = short#73239)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73208,short#73209,int#73210,long#73211L,float#73212,double#73213,decimal#73214,string#73215,binary#73216,bool#73217,date#73218,timestamp#73219,array#73220,struct#73221,map#73222])
: +- RelationV2[byte#73208, short#73209, int#73210, long#73211L, float#73212, double#73213, decimal#73214, string#73215, binary#73216, bool#73217, date#73218, timestamp#73219, array#73220, struct#73221, map#73222] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73238,short#73239,int#73240,long#73241L,float#73242,double#73243,decimal#73244,string#73245,binary#73246,bool#73247,date#73248,timestamp#73249,array#73250,struct#73251,map#73252])
+- RelationV2[byte#73238, short#73239, int#73240, long#73241L, float#73242, double#73243, decimal#73244, string#73245, binary#73246, bool#73247, date#73248, timestamp#73249, array#73250, struct#73251, map#73252] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [short#73209]
+- Join Inner, (short#73209 = short#73239)
:- Filter isnotnull(short#73209)
: +- RelationV2[short#73209] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(short#73239)
+- RelationV2[short#73239] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2602) ProjectExecTransformer [short#73209]
+- ^(2602) BroadcastHashJoinExecTransformer [short#73209], [short#73239], Inner, BuildRight, false
:- ^(2602) FilterExecTransformer isnotnull(short#73209)
: +- ^(2602) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[short#73209] OrcScan DataFilters: [isnotnull(short#73209)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73209)]
+- ^(2602) InputIteratorTransformer[short#73239]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=526457]
+- ^(2601) FilterExecTransformer isnotnull(short#73239)
+- ^(2601) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[short#73239] OrcScan DataFilters: [isnotnull(short#73239)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73239)]
+- == Initial Plan ==
Project [short#73209]
+- BroadcastHashJoin [short#73209], [short#73239], Inner, BuildRight, false
:- Filter isnotnull(short#73209)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[short#73209] OrcScan DataFilters: [isnotnull(short#73209)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=526248]
+- Filter isnotnull(short#73239)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[short#73239] OrcScan DataFilters: [isnotnull(short#73239)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|
VeloxOrcDataTypeValidationSuite.Byte type:
org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.byte]
+- 'Filter ('type1.byte = 'type2.byte)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
byte: tinyint
Project [byte#73208]
+- Filter (byte#73208 = byte#73238)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#73208,short#73209,int#73210,long#73211L,float#73212,double#73213,decimal#73214,string#73215,binary#73216,bool#73217,date#73218,timestamp#73219,array#73220,struct#73221,map#73222])
: +- RelationV2[byte#73208, short#73209, int#73210, long#73211L, float#73212, double#73213, decimal#73214, string#73215, binary#73216, bool#73217, date#73218, timestamp#73219, array#73220, struct#73221, map#73222] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc
+- SubqueryAlias type2
+- View (`type2`, [byte#73238,short#73239,int#73240,long#73241L,float#73242,double#73243,decimal#73244,string#73245,binary#73246,bool#73247,date#73248,timestamp#73249,array#73250,struct#73251,map#73252])
+- RelationV2[byte#73238, short#73239, int#73240, long#73241L, float#73242, double#73243, decimal#73244, string#73245, binary#73246, bool#73247, date#73248, timestamp#73249, array#73250, struct#73251, map#73252] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc
== Optimized Logical Plan ==
Project [byte#73208]
+- Join Inner, (byte#73208 = byte#73238)
:- Filter isnotnull(byte#73208)
: +- RelationV2[byte#73208] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc
+- Filter isnotnull(byte#73238)
+- RelationV2[byte#73238] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2652) ProjectExecTransformer [byte#73208]
+- ^(2652) BroadcastHashJoinExecTransformer [byte#73208], [byte#73238], Inner, BuildRight, false
:- ^(2652) FilterExecTransformer isnotnull(byte#73208)
: +- ^(2652) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[byte#73208] OrcScan DataFilters: [isnotnull(byte#73208)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73208)]
+- ^(2652) InputIteratorTransformer[byte#73238]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=531107]
+- ^(2651) FilterExecTransformer isnotnull(byte#73238)
+- ^(2651) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[byte#73238] OrcScan DataFilters: [isnotnull(byte#73238)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73238)]
+- == Initial Plan ==
Project [byte#73208]
+- BroadcastHashJoin [byte#73208], [byte#73238], Inner, BuildRight, false
:- Filter isnotnull(byte#73208)
: +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[byte#73208] OrcScan DataFilters: [isnotnull(byte#73208)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=530898]
+- Filter isnotnull(byte#73238)
+- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[byte#73238] OrcScan DataFilters: [isnotnull(byte#73238)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![1]
|
VeloxParquetDataTypeValidationSuite.Int type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L190
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.int]
+- 'Filter ('type1.int = 'type2.int)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
int: int
Project [int#77084]
+- Filter (int#77084 = int#77114)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#77082,short#77083,int#77084,long#77085L,float#77086,double#77087,decimal#77088,string#77089,binary#77090,bool#77091,date#77092,timestamp#77093,array#77094,struct#77095,map#77096])
: +- RelationV2[byte#77082, short#77083, int#77084, long#77085L, float#77086, double#77087, decimal#77088, string#77089, binary#77090, bool#77091, date#77092, timestamp#77093, array#77094, struct#77095, map#77096] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#77112,short#77113,int#77114,long#77115L,float#77116,double#77117,decimal#77118,string#77119,binary#77120,bool#77121,date#77122,timestamp#77123,array#77124,struct#77125,map#77126])
+- RelationV2[byte#77112, short#77113, int#77114, long#77115L, float#77116, double#77117, decimal#77118, string#77119, binary#77120, bool#77121, date#77122, timestamp#77123, array#77124, struct#77125, map#77126] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [int#77084]
+- Join Inner, (int#77084 = int#77114)
:- Filter isnotnull(int#77084)
: +- RelationV2[int#77084] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1
+- Filter isnotnull(int#77114)
+- RelationV2[int#77114] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2781) ProjectExecTransformer [int#77084]
+- ^(2781) BroadcastHashJoinExecTransformer [int#77084], [int#77114], Inner, BuildRight, false
:- ^(2781) FilterExecTransformer isnotnull(int#77084)
: +- ^(2781) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[int#77084] ParquetScan DataFilters: [isnotnull(int#77084)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77084)]
+- ^(2781) InputIteratorTransformer[int#77114]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=544059]
+- ^(2780) FilterExecTransformer isnotnull(int#77114)
+- ^(2780) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[int#77114] ParquetScan DataFilters: [isnotnull(int#77114)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77114)]
+- == Initial Plan ==
Project [int#77084]
+- BroadcastHashJoin [int#77084], [int#77114], Inner, BuildRight, false
:- Filter isnotnull(int#77084)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[int#77084] ParquetScan DataFilters: [isnotnull(int#77084)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=543850]
+- Filter isnotnull(int#77114)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[int#77114] ParquetScan DataFilters: [isnotnull(int#77114)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![3]
|
VeloxParquetDataTypeValidationSuite.Short type:
org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
'Project ['type1.short]
+- 'Filter ('type1.short = 'type2.short)
+- 'Join Inner
:- 'UnresolvedRelation [type1], [], false
+- 'UnresolvedRelation [type2], [], false
== Analyzed Logical Plan ==
short: smallint
Project [short#77083]
+- Filter (short#77083 = short#77113)
+- Join Inner
:- SubqueryAlias type1
: +- View (`type1`, [byte#77082,short#77083,int#77084,long#77085L,float#77086,double#77087,decimal#77088,string#77089,binary#77090,bool#77091,date#77092,timestamp#77093,array#77094,struct#77095,map#77096])
: +- RelationV2[byte#77082, short#77083, int#77084, long#77085L, float#77086, double#77087, decimal#77088, string#77089, binary#77090, bool#77091, date#77092, timestamp#77093, array#77094, struct#77095, map#77096] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1
+- SubqueryAlias type2
+- View (`type2`, [byte#77112,short#77113,int#77114,long#77115L,float#77116,double#77117,decimal#77118,string#77119,binary#77120,bool#77121,date#77122,timestamp#77123,array#77124,struct#77125,map#77126])
+- RelationV2[byte#77112, short#77113, int#77114, long#77115L, float#77116, double#77117, decimal#77118, string#77119, binary#77120, bool#77121, date#77122, timestamp#77123, array#77124, struct#77125, map#77126] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2
== Optimized Logical Plan ==
Project [short#77083]
+- Join Inner, (short#77083 = short#77113)
:- Filter isnotnull(short#77083)
: +- RelationV2[short#77083] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1
+- Filter isnotnull(short#77113)
+- RelationV2[short#77113] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
VeloxColumnarToRow
+- ^(2790) ProjectExecTransformer [short#77083]
+- ^(2790) BroadcastHashJoinExecTransformer [short#77083], [short#77113], Inner, BuildRight, false
:- ^(2790) FilterExecTransformer isnotnull(short#77083)
: +- ^(2790) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[short#77083] ParquetScan DataFilters: [isnotnull(short#77083)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77083)]
+- ^(2790) InputIteratorTransformer[short#77113]
+- BroadcastQueryStage 0
+- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=544974]
+- ^(2789) FilterExecTransformer isnotnull(short#77113)
+- ^(2789) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[short#77113] ParquetScan DataFilters: [isnotnull(short#77113)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77113)]
+- == Initial Plan ==
Project [short#77083]
+- BroadcastHashJoin [short#77083], [short#77113], Inner, BuildRight, false
:- Filter isnotnull(short#77083)
: +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[short#77083] ParquetScan DataFilters: [isnotnull(short#77083)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=544765]
+- Filter isnotnull(short#77113)
+- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[short#77113] ParquetScan DataFilters: [isnotnull(short#77113)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: []
== Results ==
== Results ==
!== Correct Answer - 1 == == Gluten Answer - 0 ==
struct<> struct<>
![2]
|