Skip to content

[GLUTEN-7548][VL][test] Optimize BHJ in velox backend #15538

[GLUTEN-7548][VL][test] Optimize BHJ in velox backend

[GLUTEN-7548][VL][test] Optimize BHJ in velox backend #15538

Triggered via pull request March 12, 2025 11:38
@JkSelfJkSelf
synchronize #8931
Status Success
Total duration 31s
Artifacts

clickhouse_be_trigger.yml

on: pull_request_target
add-comment
23s
add-comment
Fit to window
Zoom out
Zoom in

Annotations

29 errors
VeloxOrcDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L185
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#48956] +- Filter (int#48956 = int#48986) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#48954,short#48955,int#48956,long#48957L,float#48958,double#48959,decimal#48960,string#48961,binary#48962,bool#48963,date#48964,timestamp#48965,array#48966,struct#48967,map#48968]) : +- RelationV2[byte#48954, short#48955, int#48956, long#48957L, float#48958, double#48959, decimal#48960, string#48961, binary#48962, bool#48963, date#48964, timestamp#48965, array#48966, struct#48967, map#48968] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#48984,short#48985,int#48986,long#48987L,float#48988,double#48989,decimal#48990,string#48991,binary#48992,bool#48993,date#48994,timestamp#48995,array#48996,struct#48997,map#48998]) +- RelationV2[byte#48984, short#48985, int#48986, long#48987L, float#48988, double#48989, decimal#48990, string#48991, binary#48992, bool#48993, date#48994, timestamp#48995, array#48996, struct#48997, map#48998] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [int#48956] +- Join Inner, (int#48956 = int#48986) :- Filter isnotnull(int#48956) : +- RelationV2[int#48956] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(int#48986) +- RelationV2[int#48986] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(1936) ProjectExecTransformer [int#48956] +- ^(1936) BroadcastHashJoinExecTransformer [int#48956], [int#48986], Inner, BuildRight, false :- ^(1936) FilterExecTransformer isnotnull(int#48956) : +- ^(1936) BatchScanExecTransformer[int#48956] OrcScan DataFilters: [isnotnull(int#48956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#48956)] +- ^(1936) InputIteratorTransformer[int#48986] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=511584] +- ^(1935) FilterExecTransformer isnotnull(int#48986) +- ^(1935) BatchScanExecTransformer[int#48986] OrcScan DataFilters: [isnotnull(int#48986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] NativeFilters: [isnotnull(int#48986)] +- == Initial Plan == Project [int#48956] +- BroadcastHashJoin [int#48956], [int#48986], Inner, BuildRight, false :- Filter isnotnull(int#48956) : +- BatchScan[int#48956] OrcScan DataFilters: [isnotnull(int#48956)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=511375] +- Filter isnotnull(int#48986) +- BatchScan[int#48986] OrcScan DataFilters: [isnotnull(int#48986)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(int)], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxOrcDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#48955] +- Filter (short#48955 = short#48985) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#48954,short#48955,int#48956,long#48957L,float#48958,double#48959,decimal#48960,string#48961,binary#48962,bool#48963,date#48964,timestamp#48965,array#48966,struct#48967,map#48968]) : +- RelationV2[byte#48954, short#48955, int#48956, long#48957L, float#48958, double#48959, decimal#48960, string#48961, binary#48962, bool#48963, date#48964, timestamp#48965, array#48966, struct#48967, map#48968] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#48984,short#48985,int#48986,long#48987L,float#48988,double#48989,decimal#48990,string#48991,binary#48992,bool#48993,date#48994,timestamp#48995,array#48996,struct#48997,map#48998]) +- RelationV2[byte#48984, short#48985, int#48986, long#48987L, float#48988, double#48989, decimal#48990, string#48991, binary#48992, bool#48993, date#48994, timestamp#48995, array#48996, struct#48997, map#48998] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [short#48955] +- Join Inner, (short#48955 = short#48985) :- Filter isnotnull(short#48955) : +- RelationV2[short#48955] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(short#48985) +- RelationV2[short#48985] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(1945) ProjectExecTransformer [short#48955] +- ^(1945) BroadcastHashJoinExecTransformer [short#48955], [short#48985], Inner, BuildRight, false :- ^(1945) FilterExecTransformer isnotnull(short#48955) : +- ^(1945) BatchScanExecTransformer[short#48955] OrcScan DataFilters: [isnotnull(short#48955)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] NativeFilters: [isnotnull(short#48955)] +- ^(1945) InputIteratorTransformer[short#48985] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=512499] +- ^(1944) FilterExecTransformer isnotnull(short#48985) +- ^(1944) BatchScanExecTransformer[short#48985] OrcScan DataFilters: [isnotnull(short#48985)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] NativeFilters: [isnotnull(short#48985)] +- == Initial Plan == Project [short#48955] +- BroadcastHashJoin [short#48955], [short#48985], Inner, BuildRight, false :- Filter isnotnull(short#48955) : +- BatchScan[short#48955] OrcScan DataFilters: [isnotnull(short#48955)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=512290] +- Filter isnotnull(short#48985) +- BatchScan[short#48985] OrcScan DataFilters: [isnotnull(short#48985)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(short)], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#48954] +- Filter (byte#48954 = byte#48984) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#48954,short#48955,int#48956,long#48957L,float#48958,double#48959,decimal#48960,string#48961,binary#48962,bool#48963,date#48964,timestamp#48965,array#48966,struct#48967,map#48968]) : +- RelationV2[byte#48954, short#48955, int#48956, long#48957L, float#48958, double#48959, decimal#48960, string#48961, binary#48962, bool#48963, date#48964, timestamp#48965, array#48966, struct#48967, map#48968] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#48984,short#48985,int#48986,long#48987L,float#48988,double#48989,decimal#48990,string#48991,binary#48992,bool#48993,date#48994,timestamp#48995,array#48996,struct#48997,map#48998]) +- RelationV2[byte#48984, short#48985, int#48986, long#48987L, float#48988, double#48989, decimal#48990, string#48991, binary#48992, bool#48993, date#48994, timestamp#48995, array#48996, struct#48997, map#48998] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [byte#48954] +- Join Inner, (byte#48954 = byte#48984) :- Filter isnotnull(byte#48954) : +- RelationV2[byte#48954] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(byte#48984) +- RelationV2[byte#48984] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(1983) ProjectExecTransformer [byte#48954] +- ^(1983) BroadcastHashJoinExecTransformer [byte#48954], [byte#48984], Inner, BuildRight, false :- ^(1983) FilterExecTransformer isnotnull(byte#48954) : +- ^(1983) BatchScanExecTransformer[byte#48954] OrcScan DataFilters: [isnotnull(byte#48954)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] NativeFilters: [isnotnull(byte#48954)] +- ^(1983) InputIteratorTransformer[byte#48984] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=516525] +- ^(1982) FilterExecTransformer isnotnull(byte#48984) +- ^(1982) BatchScanExecTransformer[byte#48984] OrcScan DataFilters: [isnotnull(byte#48984)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] NativeFilters: [isnotnull(byte#48984)] +- == Initial Plan == Project [byte#48954] +- BroadcastHashJoin [byte#48954], [byte#48984], Inner, BuildRight, false :- Filter isnotnull(byte#48954) : +- BatchScan[byte#48954] OrcScan DataFilters: [isnotnull(byte#48954)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=516316] +- Filter isnotnull(byte#48984) +- BatchScan[byte#48984] OrcScan DataFilters: [isnotnull(byte#48984)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedFilters: [IsNotNull(byte)], ReadSchema: struct<byte:tinyint>, PushedFilters: [IsNotNull(byte)] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxOrcDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L191
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#56434] +- Filter (int#56434 = int#56464) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#56432,short#56433,int#56434,long#56435L,float#56436,double#56437,decimal#56438,string#56439,binary#56440,bool#56441,date#56442,timestamp#56443,array#56444,struct#56445,map#56446]) : +- RelationV2[byte#56432, short#56433, int#56434, long#56435L, float#56436, double#56437, decimal#56438, string#56439, binary#56440, bool#56441, date#56442, timestamp#56443, array#56444, struct#56445, map#56446] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#56462,short#56463,int#56464,long#56465L,float#56466,double#56467,decimal#56468,string#56469,binary#56470,bool#56471,date#56472,timestamp#56473,array#56474,struct#56475,map#56476]) +- RelationV2[byte#56462, short#56463, int#56464, long#56465L, float#56466, double#56467, decimal#56468, string#56469, binary#56470, bool#56471, date#56472, timestamp#56473, array#56474, struct#56475, map#56476] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [int#56434] +- Join Inner, (int#56434 = int#56464) :- Filter isnotnull(int#56434) : +- RelationV2[int#56434] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(int#56464) +- RelationV2[int#56464] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2056) ProjectExecTransformer [int#56434] +- ^(2056) BroadcastHashJoinExecTransformer [int#56434], [int#56464], Inner, BuildRight, false :- ^(2056) FilterExecTransformer isnotnull(int#56434) : +- ^(2056) BatchScanExecTransformer[int#56434] OrcScan DataFilters: [isnotnull(int#56434)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#56434)] +- ^(2056) InputIteratorTransformer[int#56464] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=523506] +- ^(2055) FilterExecTransformer isnotnull(int#56464) +- ^(2055) BatchScanExecTransformer[int#56464] OrcScan DataFilters: [isnotnull(int#56464)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#56464)] +- == Initial Plan == Project [int#56434] +- BroadcastHashJoin [int#56434], [int#56464], Inner, BuildRight, false :- Filter isnotnull(int#56434) : +- BatchScan[int#56434] OrcScan DataFilters: [isnotnull(int#56434)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=523297] +- Filter isnotnull(int#56464) +- BatchScan[int#56464] OrcScan DataFilters: [isnotnull(int#56464)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxOrcDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#56433] +- Filter (short#56433 = short#56463) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#56432,short#56433,int#56434,long#56435L,float#56436,double#56437,decimal#56438,string#56439,binary#56440,bool#56441,date#56442,timestamp#56443,array#56444,struct#56445,map#56446]) : +- RelationV2[byte#56432, short#56433, int#56434, long#56435L, float#56436, double#56437, decimal#56438, string#56439, binary#56440, bool#56441, date#56442, timestamp#56443, array#56444, struct#56445, map#56446] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#56462,short#56463,int#56464,long#56465L,float#56466,double#56467,decimal#56468,string#56469,binary#56470,bool#56471,date#56472,timestamp#56473,array#56474,struct#56475,map#56476]) +- RelationV2[byte#56462, short#56463, int#56464, long#56465L, float#56466, double#56467, decimal#56468, string#56469, binary#56470, bool#56471, date#56472, timestamp#56473, array#56474, struct#56475, map#56476] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [short#56433] +- Join Inner, (short#56433 = short#56463) :- Filter isnotnull(short#56433) : +- RelationV2[short#56433] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(short#56463) +- RelationV2[short#56463] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2065) ProjectExecTransformer [short#56433] +- ^(2065) BroadcastHashJoinExecTransformer [short#56433], [short#56463], Inner, BuildRight, false :- ^(2065) FilterExecTransformer isnotnull(short#56433) : +- ^(2065) BatchScanExecTransformer[short#56433] OrcScan DataFilters: [isnotnull(short#56433)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#56433)] +- ^(2065) InputIteratorTransformer[short#56463] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=524421] +- ^(2064) FilterExecTransformer isnotnull(short#56463) +- ^(2064) BatchScanExecTransformer[short#56463] OrcScan DataFilters: [isnotnull(short#56463)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#56463)] +- == Initial Plan == Project [short#56433] +- BroadcastHashJoin [short#56433], [short#56463], Inner, BuildRight, false :- Filter isnotnull(short#56433) : +- BatchScan[short#56433] OrcScan DataFilters: [isnotnull(short#56433)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=524212] +- Filter isnotnull(short#56463) +- BatchScan[short#56463] OrcScan DataFilters: [isnotnull(short#56463)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxParquetDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L190
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#57849] +- Filter (int#57849 = int#57879) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#57847,short#57848,int#57849,long#57850L,float#57851,double#57852,decimal#57853,string#57854,binary#57855,bool#57856,date#57857,timestamp#57858,array#57859,struct#57860,map#57861]) : +- RelationV2[byte#57847, short#57848, int#57849, long#57850L, float#57851, double#57852, decimal#57853, string#57854, binary#57855, bool#57856, date#57857, timestamp#57858, array#57859, struct#57860, map#57861] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#57877,short#57878,int#57879,long#57880L,float#57881,double#57882,decimal#57883,string#57884,binary#57885,bool#57886,date#57887,timestamp#57888,array#57889,struct#57890,map#57891]) +- RelationV2[byte#57877, short#57878, int#57879, long#57880L, float#57881, double#57882, decimal#57883, string#57884, binary#57885, bool#57886, date#57887, timestamp#57888, array#57889, struct#57890, map#57891] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [int#57849] +- Join Inner, (int#57849 = int#57879) :- Filter isnotnull(int#57849) : +- RelationV2[int#57849] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(int#57879) +- RelationV2[int#57879] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2234) ProjectExecTransformer [int#57849] +- ^(2234) BroadcastHashJoinExecTransformer [int#57849], [int#57879], Inner, BuildRight, false :- ^(2234) FilterExecTransformer isnotnull(int#57849) : +- ^(2234) BatchScanExecTransformer[int#57849] ParquetScan DataFilters: [isnotnull(int#57849)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#57849)] +- ^(2234) InputIteratorTransformer[int#57879] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=541428] +- ^(2233) FilterExecTransformer isnotnull(int#57879) +- ^(2233) BatchScanExecTransformer[int#57879] ParquetScan DataFilters: [isnotnull(int#57879)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(int#57879)] +- == Initial Plan == Project [int#57849] +- BroadcastHashJoin [int#57849], [int#57879], Inner, BuildRight, false :- Filter isnotnull(int#57849) : +- BatchScan[int#57849] ParquetScan DataFilters: [isnotnull(int#57849)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=541219] +- Filter isnotnull(int#57879) +- BatchScan[int#57879] ParquetScan DataFilters: [isnotnull(int#57879)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int>, PushedFilters: [IsNotNull(int)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxParquetDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#57848] +- Filter (short#57848 = short#57878) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#57847,short#57848,int#57849,long#57850L,float#57851,double#57852,decimal#57853,string#57854,binary#57855,bool#57856,date#57857,timestamp#57858,array#57859,struct#57860,map#57861]) : +- RelationV2[byte#57847, short#57848, int#57849, long#57850L, float#57851, double#57852, decimal#57853, string#57854, binary#57855, bool#57856, date#57857, timestamp#57858, array#57859, struct#57860, map#57861] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#57877,short#57878,int#57879,long#57880L,float#57881,double#57882,decimal#57883,string#57884,binary#57885,bool#57886,date#57887,timestamp#57888,array#57889,struct#57890,map#57891]) +- RelationV2[byte#57877, short#57878, int#57879, long#57880L, float#57881, double#57882, decimal#57883, string#57884, binary#57885, bool#57886, date#57887, timestamp#57888, array#57889, struct#57890, map#57891] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [short#57848] +- Join Inner, (short#57848 = short#57878) :- Filter isnotnull(short#57848) : +- RelationV2[short#57848] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(short#57878) +- RelationV2[short#57878] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2243) ProjectExecTransformer [short#57848] +- ^(2243) BroadcastHashJoinExecTransformer [short#57848], [short#57878], Inner, BuildRight, false :- ^(2243) FilterExecTransformer isnotnull(short#57848) : +- ^(2243) BatchScanExecTransformer[short#57848] ParquetScan DataFilters: [isnotnull(short#57848)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#57848)] +- ^(2243) InputIteratorTransformer[short#57878] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=542343] +- ^(2242) FilterExecTransformer isnotnull(short#57878) +- ^(2242) BatchScanExecTransformer[short#57878] ParquetScan DataFilters: [isnotnull(short#57878)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] NativeFilters: [isnotnull(short#57878)] +- == Initial Plan == Project [short#57848] +- BroadcastHashJoin [short#57848], [short#57878], Inner, BuildRight, false :- Filter isnotnull(short#57848) : +- BatchScan[short#57848] ParquetScan DataFilters: [isnotnull(short#57848)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=542134] +- Filter isnotnull(short#57878) +- BatchScan[short#57878] ParquetScan DataFilters: [isnotnull(short#57878)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint>, PushedFilters: [IsNotNull(short)], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#57572] +- Filter (byte#57572 = byte#57602) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#57572,short#57573,int#57574,long#57575L,float#57576,double#57577,decimal#57578,string#57579,binary#57580,bool#57581,date#57582,timestamp#57583,array#57584,struct#57585,map#57586]) : +- RelationV2[byte#57572, short#57573, int#57574, long#57575L, float#57576, double#57577, decimal#57578, string#57579, binary#57580, bool#57581, date#57582, timestamp#57583, array#57584, struct#57585, map#57586] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#57602,short#57603,int#57604,long#57605L,float#57606,double#57607,decimal#57608,string#57609,binary#57610,bool#57611,date#57612,timestamp#57613,array#57614,struct#57615,map#57616]) +- RelationV2[byte#57602, short#57603, int#57604, long#57605L, float#57606, double#57607, decimal#57608, string#57609, binary#57610, bool#57611, date#57612, timestamp#57613, array#57614, struct#57615, map#57616] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [byte#57572] +- Join Inner, (byte#57572 = byte#57602) :- Filter isnotnull(byte#57572) : +- RelationV2[byte#57572] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(byte#57602) +- RelationV2[byte#57602] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2090) ProjectExecTransformer [byte#57572] +- ^(2090) BroadcastHashJoinExecTransformer [byte#57572], [byte#57602], Inner, BuildRight, false :- ^(2090) FilterExecTransformer isnotnull(byte#57572) : +- ^(2090) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#57572] OrcScan DataFilters: [isnotnull(byte#57572)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#57572)] +- ^(2090) InputIteratorTransformer[byte#57602] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=502873] +- ^(2089) FilterExecTransformer isnotnull(byte#57602) +- ^(2089) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#57602] OrcScan DataFilters: [isnotnull(byte#57602)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#57602)] +- == Initial Plan == Project [byte#57572] +- BroadcastHashJoin [byte#57572], [byte#57602], Inner, BuildRight, false :- Filter isnotnull(byte#57572) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#57572] OrcScan DataFilters: [isnotnull(byte#57572)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=502664] +- Filter isnotnull(byte#57602) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#57602] OrcScan DataFilters: [isnotnull(byte#57602)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxParquetDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#61244] +- Filter (int#61244 = int#61274) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#61242,short#61243,int#61244,long#61245L,float#61246,double#61247,decimal#61248,string#61249,binary#61250,bool#61251,date#61252,timestamp#61253,array#61254,struct#61255,map#61256]) : +- RelationV2[byte#61242, short#61243, int#61244, long#61245L, float#61246, double#61247, decimal#61248, string#61249, binary#61250, bool#61251, date#61252, timestamp#61253, array#61254, struct#61255, map#61256] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#61272,short#61273,int#61274,long#61275L,float#61276,double#61277,decimal#61278,string#61279,binary#61280,bool#61281,date#61282,timestamp#61283,array#61284,struct#61285,map#61286]) +- RelationV2[byte#61272, short#61273, int#61274, long#61275L, float#61276, double#61277, decimal#61278, string#61279, binary#61280, bool#61281, date#61282, timestamp#61283, array#61284, struct#61285, map#61286] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [int#61244] +- Join Inner, (int#61244 = int#61274) :- Filter isnotnull(int#61244) : +- RelationV2[int#61244] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(int#61274) +- RelationV2[int#61274] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2213) ProjectExecTransformer [int#61244] +- ^(2213) BroadcastHashJoinExecTransformer [int#61244], [int#61274], Inner, BuildRight, false :- ^(2213) FilterExecTransformer isnotnull(int#61244) : +- ^(2213) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#61244] ParquetScan DataFilters: [isnotnull(int#61244)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#61244)] +- ^(2213) InputIteratorTransformer[int#61274] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=515309] +- ^(2212) FilterExecTransformer isnotnull(int#61274) +- ^(2212) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#61274] ParquetScan DataFilters: [isnotnull(int#61274)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#61274)] +- == Initial Plan == Project [int#61244] +- BroadcastHashJoin [int#61244], [int#61274], Inner, BuildRight, false :- Filter isnotnull(int#61244) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#61244] ParquetScan DataFilters: [isnotnull(int#61244)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=515100] +- Filter isnotnull(int#61274) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#61274] ParquetScan DataFilters: [isnotnull(int#61274)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxParquetDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L217
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#61243] +- Filter (short#61243 = short#61273) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#61242,short#61243,int#61244,long#61245L,float#61246,double#61247,decimal#61248,string#61249,binary#61250,bool#61251,date#61252,timestamp#61253,array#61254,struct#61255,map#61256]) : +- RelationV2[byte#61242, short#61243, int#61244, long#61245L, float#61246, double#61247, decimal#61248, string#61249, binary#61250, bool#61251, date#61252, timestamp#61253, array#61254, struct#61255, map#61256] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#61272,short#61273,int#61274,long#61275L,float#61276,double#61277,decimal#61278,string#61279,binary#61280,bool#61281,date#61282,timestamp#61283,array#61284,struct#61285,map#61286]) +- RelationV2[byte#61272, short#61273, int#61274, long#61275L, float#61276, double#61277, decimal#61278, string#61279, binary#61280, bool#61281, date#61282, timestamp#61283, array#61284, struct#61285, map#61286] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [short#61243] +- Join Inner, (short#61243 = short#61273) :- Filter isnotnull(short#61243) : +- RelationV2[short#61243] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(short#61273) +- RelationV2[short#61273] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2226) ProjectExecTransformer [short#61243] +- ^(2226) BroadcastHashJoinExecTransformer [short#61243], [short#61273], Inner, BuildRight, false :- ^(2226) FilterExecTransformer isnotnull(short#61243) : +- ^(2226) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#61243] ParquetScan DataFilters: [isnotnull(short#61243)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#61243)] +- ^(2226) InputIteratorTransformer[short#61273] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=516612] +- ^(2225) FilterExecTransformer isnotnull(short#61273) +- ^(2225) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#61273] ParquetScan DataFilters: [isnotnull(short#61273)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#61273)] +- == Initial Plan == Project [short#61243] +- BroadcastHashJoin [short#61243], [short#61273], Inner, BuildRight, false :- Filter isnotnull(short#61243) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#61243] ParquetScan DataFilters: [isnotnull(short#61243)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=516403] +- Filter isnotnull(short#61273) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#61273] ParquetScan DataFilters: [isnotnull(short#61273)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxParquetDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L327
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#61242] +- Filter (byte#61242 = byte#61272) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#61242,short#61243,int#61244,long#61245L,float#61246,double#61247,decimal#61248,string#61249,binary#61250,bool#61251,date#61252,timestamp#61253,array#61254,struct#61255,map#61256]) : +- RelationV2[byte#61242, short#61243, int#61244, long#61245L, float#61246, double#61247, decimal#61248, string#61249, binary#61250, bool#61251, date#61252, timestamp#61253, array#61254, struct#61255, map#61256] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#61272,short#61273,int#61274,long#61275L,float#61276,double#61277,decimal#61278,string#61279,binary#61280,bool#61281,date#61282,timestamp#61283,array#61284,struct#61285,map#61286]) +- RelationV2[byte#61272, short#61273, int#61274, long#61275L, float#61276, double#61277, decimal#61278, string#61279, binary#61280, bool#61281, date#61282, timestamp#61283, array#61284, struct#61285, map#61286] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [byte#61242] +- Join Inner, (byte#61242 = byte#61272) :- Filter isnotnull(byte#61242) : +- RelationV2[byte#61242] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(byte#61272) +- RelationV2[byte#61272] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2268) ProjectExecTransformer [byte#61242] +- ^(2268) BroadcastHashJoinExecTransformer [byte#61242], [byte#61272], Inner, BuildRight, false :- ^(2268) FilterExecTransformer isnotnull(byte#61242) : +- ^(2268) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#61242] ParquetScan DataFilters: [isnotnull(byte#61242)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#61242)] +- ^(2268) InputIteratorTransformer[byte#61272] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=521026] +- ^(2267) FilterExecTransformer isnotnull(byte#61272) +- ^(2267) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#61272] ParquetScan DataFilters: [isnotnull(byte#61272)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#61272)] +- == Initial Plan == Project [byte#61242] +- BroadcastHashJoin [byte#61242], [byte#61272], Inner, BuildRight, false :- Filter isnotnull(byte#61242) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#61242] ParquetScan DataFilters: [isnotnull(byte#61242)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=520817] +- Filter isnotnull(byte#61272) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#61272] ParquetScan DataFilters: [isnotnull(byte#61272)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxOrcDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#60984] +- Filter (short#60984 = short#61014) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#60983,short#60984,int#60985,long#60986L,float#60987,double#60988,decimal#60989,string#60990,binary#60991,bool#60992,date#60993,timestamp#60994,array#60995,struct#60996,map#60997]) : +- RelationV2[byte#60983, short#60984, int#60985, long#60986L, float#60987, double#60988, decimal#60989, string#60990, binary#60991, bool#60992, date#60993, timestamp#60994, array#60995, struct#60996, map#60997] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#61013,short#61014,int#61015,long#61016L,float#61017,double#61018,decimal#61019,string#61020,binary#61021,bool#61022,date#61023,timestamp#61024,array#61025,struct#61026,map#61027]) +- RelationV2[byte#61013, short#61014, int#61015, long#61016L, float#61017, double#61018, decimal#61019, string#61020, binary#61021, bool#61022, date#61023, timestamp#61024, array#61025, struct#61026, map#61027] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [short#60984] +- Join Inner, (short#60984 = short#61014) :- Filter isnotnull(short#60984) : +- RelationV2[short#60984] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(short#61014) +- RelationV2[short#61014] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2166) ProjectExecTransformer [short#60984] +- ^(2166) BroadcastHashJoinExecTransformer [short#60984], [short#61014], Inner, BuildRight, false :- ^(2166) FilterExecTransformer isnotnull(short#60984) : +- ^(2166) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#60984] OrcScan DataFilters: [isnotnull(short#60984)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#60984)] +- ^(2166) InputIteratorTransformer[short#61014] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=542707] +- ^(2165) FilterExecTransformer isnotnull(short#61014) +- ^(2165) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#61014] OrcScan DataFilters: [isnotnull(short#61014)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#61014)] +- == Initial Plan == Project [short#60984] +- BroadcastHashJoin [short#60984], [short#61014], Inner, BuildRight, false :- Filter isnotnull(short#60984) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#60984] OrcScan DataFilters: [isnotnull(short#60984)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=542498] +- Filter isnotnull(short#61014) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#61014] OrcScan DataFilters: [isnotnull(short#61014)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L328
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#60983] +- Filter (byte#60983 = byte#61013) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#60983,short#60984,int#60985,long#60986L,float#60987,double#60988,decimal#60989,string#60990,binary#60991,bool#60992,date#60993,timestamp#60994,array#60995,struct#60996,map#60997]) : +- RelationV2[byte#60983, short#60984, int#60985, long#60986L, float#60987, double#60988, decimal#60989, string#60990, binary#60991, bool#60992, date#60993, timestamp#60994, array#60995, struct#60996, map#60997] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#61013,short#61014,int#61015,long#61016L,float#61017,double#61018,decimal#61019,string#61020,binary#61021,bool#61022,date#61023,timestamp#61024,array#61025,struct#61026,map#61027]) +- RelationV2[byte#61013, short#61014, int#61015, long#61016L, float#61017, double#61018, decimal#61019, string#61020, binary#61021, bool#61022, date#61023, timestamp#61024, array#61025, struct#61026, map#61027] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [byte#60983] +- Join Inner, (byte#60983 = byte#61013) :- Filter isnotnull(byte#60983) : +- RelationV2[byte#60983] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(byte#61013) +- RelationV2[byte#61013] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2208) ProjectExecTransformer [byte#60983] +- ^(2208) BroadcastHashJoinExecTransformer [byte#60983], [byte#61013], Inner, BuildRight, false :- ^(2208) FilterExecTransformer isnotnull(byte#60983) : +- ^(2208) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#60983] OrcScan DataFilters: [isnotnull(byte#60983)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#60983)] +- ^(2208) InputIteratorTransformer[byte#61013] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=547121] +- ^(2207) FilterExecTransformer isnotnull(byte#61013) +- ^(2207) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#61013] OrcScan DataFilters: [isnotnull(byte#61013)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#61013)] +- == Initial Plan == Project [byte#60983] +- BroadcastHashJoin [byte#60983], [byte#61013], Inner, BuildRight, false :- Filter isnotnull(byte#60983) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#60983] OrcScan DataFilters: [isnotnull(byte#60983)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=546912] +- Filter isnotnull(byte#61013) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#61013] OrcScan DataFilters: [isnotnull(byte#61013)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxParquetDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#64655] +- Filter (int#64655 = int#64685) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#64653,short#64654,int#64655,long#64656L,float#64657,double#64658,decimal#64659,string#64660,binary#64661,bool#64662,date#64663,timestamp#64664,array#64665,struct#64666,map#64667]) : +- RelationV2[byte#64653, short#64654, int#64655, long#64656L, float#64657, double#64658, decimal#64659, string#64660, binary#64661, bool#64662, date#64663, timestamp#64664, array#64665, struct#64666, map#64667] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#64683,short#64684,int#64685,long#64686L,float#64687,double#64688,decimal#64689,string#64690,binary#64691,bool#64692,date#64693,timestamp#64694,array#64695,struct#64696,map#64697]) +- RelationV2[byte#64683, short#64684, int#64685, long#64686L, float#64687, double#64688, decimal#64689, string#64690, binary#64691, bool#64692, date#64693, timestamp#64694, array#64695, struct#64696, map#64697] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [int#64655] +- Join Inner, (int#64655 = int#64685) :- Filter isnotnull(int#64655) : +- RelationV2[int#64655] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(int#64685) +- RelationV2[int#64685] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2331) ProjectExecTransformer [int#64655] +- ^(2331) BroadcastHashJoinExecTransformer [int#64655], [int#64685], Inner, BuildRight, false :- ^(2331) FilterExecTransformer isnotnull(int#64655) : +- ^(2331) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#64655] ParquetScan DataFilters: [isnotnull(int#64655)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#64655)] +- ^(2331) InputIteratorTransformer[int#64685] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=559557] +- ^(2330) FilterExecTransformer isnotnull(int#64685) +- ^(2330) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#64685] ParquetScan DataFilters: [isnotnull(int#64685)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#64685)] +- == Initial Plan == Project [int#64655] +- BroadcastHashJoin [int#64655], [int#64685], Inner, BuildRight, false :- Filter isnotnull(int#64655) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#64655] ParquetScan DataFilters: [isnotnull(int#64655)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=559348] +- Filter isnotnull(int#64685) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#64685] ParquetScan DataFilters: [isnotnull(int#64685)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxParquetDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#64653] +- Filter (byte#64653 = byte#64683) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#64653,short#64654,int#64655,long#64656L,float#64657,double#64658,decimal#64659,string#64660,binary#64661,bool#64662,date#64663,timestamp#64664,array#64665,struct#64666,map#64667]) : +- RelationV2[byte#64653, short#64654, int#64655, long#64656L, float#64657, double#64658, decimal#64659, string#64660, binary#64661, bool#64662, date#64663, timestamp#64664, array#64665, struct#64666, map#64667] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#64683,short#64684,int#64685,long#64686L,float#64687,double#64688,decimal#64689,string#64690,binary#64691,bool#64692,date#64693,timestamp#64694,array#64695,struct#64696,map#64697]) +- RelationV2[byte#64683, short#64684, int#64685, long#64686L, float#64687, double#64688, decimal#64689, string#64690, binary#64691, bool#64692, date#64693, timestamp#64694, array#64695, struct#64696, map#64697] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [byte#64653] +- Join Inner, (byte#64653 = byte#64683) :- Filter isnotnull(byte#64653) : +- RelationV2[byte#64653] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(byte#64683) +- RelationV2[byte#64683] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2382) ProjectExecTransformer [byte#64653] +- ^(2382) BroadcastHashJoinExecTransformer [byte#64653], [byte#64683], Inner, BuildRight, false :- ^(2382) FilterExecTransformer isnotnull(byte#64653) : +- ^(2382) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#64653] ParquetScan DataFilters: [isnotnull(byte#64653)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#64653)] +- ^(2382) InputIteratorTransformer[byte#64683] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=564886] +- ^(2381) FilterExecTransformer isnotnull(byte#64683) +- ^(2381) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#64683] ParquetScan DataFilters: [isnotnull(byte#64683)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#64683)] +- == Initial Plan == Project [byte#64653] +- BroadcastHashJoin [byte#64653], [byte#64683], Inner, BuildRight, false :- Filter isnotnull(byte#64653) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#64653] ParquetScan DataFilters: [isnotnull(byte#64653)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=564677] +- Filter isnotnull(byte#64683) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#64683] ParquetScan DataFilters: [isnotnull(byte#64683)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxOrcDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L191
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#73755] +- Filter (int#73755 = int#73785) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73753,short#73754,int#73755,long#73756L,float#73757,double#73758,decimal#73759,string#73760,binary#73761,bool#73762,date#73763,timestamp#73764,array#73765,struct#73766,map#73767]) : +- RelationV2[byte#73753, short#73754, int#73755, long#73756L, float#73757, double#73758, decimal#73759, string#73760, binary#73761, bool#73762, date#73763, timestamp#73764, array#73765, struct#73766, map#73767] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73783,short#73784,int#73785,long#73786L,float#73787,double#73788,decimal#73789,string#73790,binary#73791,bool#73792,date#73793,timestamp#73794,array#73795,struct#73796,map#73797]) +- RelationV2[byte#73783, short#73784, int#73785, long#73786L, float#73787, double#73788, decimal#73789, string#73790, binary#73791, bool#73792, date#73793, timestamp#73794, array#73795, struct#73796, map#73797] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [int#73755] +- Join Inner, (int#73755 = int#73785) :- Filter isnotnull(int#73755) : +- RelationV2[int#73755] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(int#73785) +- RelationV2[int#73785] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2382) ProjectExecTransformer [int#73755] +- ^(2382) BroadcastHashJoinExecTransformer [int#73755], [int#73785], Inner, BuildRight, false :- ^(2382) FilterExecTransformer isnotnull(int#73755) : +- ^(2382) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#73755] OrcScan DataFilters: [isnotnull(int#73755)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73755)] +- ^(2382) InputIteratorTransformer[int#73785] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=550361] +- ^(2381) FilterExecTransformer isnotnull(int#73785) +- ^(2381) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#73785] OrcScan DataFilters: [isnotnull(int#73785)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#73785)] +- == Initial Plan == Project [int#73755] +- BroadcastHashJoin [int#73755], [int#73785], Inner, BuildRight, false :- Filter isnotnull(int#73755) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[int#73755] OrcScan DataFilters: [isnotnull(int#73755)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=550152] +- Filter isnotnull(int#73785) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[int#73785] OrcScan DataFilters: [isnotnull(int#73785)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxOrcDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L218
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#73754] +- Filter (short#73754 = short#73784) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73753,short#73754,int#73755,long#73756L,float#73757,double#73758,decimal#73759,string#73760,binary#73761,bool#73762,date#73763,timestamp#73764,array#73765,struct#73766,map#73767]) : +- RelationV2[byte#73753, short#73754, int#73755, long#73756L, float#73757, double#73758, decimal#73759, string#73760, binary#73761, bool#73762, date#73763, timestamp#73764, array#73765, struct#73766, map#73767] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73783,short#73784,int#73785,long#73786L,float#73787,double#73788,decimal#73789,string#73790,binary#73791,bool#73792,date#73793,timestamp#73794,array#73795,struct#73796,map#73797]) +- RelationV2[byte#73783, short#73784, int#73785, long#73786L, float#73787, double#73788, decimal#73789, string#73790, binary#73791, bool#73792, date#73793, timestamp#73794, array#73795, struct#73796, map#73797] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [short#73754] +- Join Inner, (short#73754 = short#73784) :- Filter isnotnull(short#73754) : +- RelationV2[short#73754] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(short#73784) +- RelationV2[short#73784] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2395) ProjectExecTransformer [short#73754] +- ^(2395) BroadcastHashJoinExecTransformer [short#73754], [short#73784], Inner, BuildRight, false :- ^(2395) FilterExecTransformer isnotnull(short#73754) : +- ^(2395) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73754] OrcScan DataFilters: [isnotnull(short#73754)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73754)] +- ^(2395) InputIteratorTransformer[short#73784] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=551664] +- ^(2394) FilterExecTransformer isnotnull(short#73784) +- ^(2394) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73784] OrcScan DataFilters: [isnotnull(short#73784)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73784)] +- == Initial Plan == Project [short#73754] +- BroadcastHashJoin [short#73754], [short#73784], Inner, BuildRight, false :- Filter isnotnull(short#73754) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73754] OrcScan DataFilters: [isnotnull(short#73754)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=551455] +- Filter isnotnull(short#73784) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73784] OrcScan DataFilters: [isnotnull(short#73784)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L328
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#73753] +- Filter (byte#73753 = byte#73783) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73753,short#73754,int#73755,long#73756L,float#73757,double#73758,decimal#73759,string#73760,binary#73761,bool#73762,date#73763,timestamp#73764,array#73765,struct#73766,map#73767]) : +- RelationV2[byte#73753, short#73754, int#73755, long#73756L, float#73757, double#73758, decimal#73759, string#73760, binary#73761, bool#73762, date#73763, timestamp#73764, array#73765, struct#73766, map#73767] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73783,short#73784,int#73785,long#73786L,float#73787,double#73788,decimal#73789,string#73790,binary#73791,bool#73792,date#73793,timestamp#73794,array#73795,struct#73796,map#73797]) +- RelationV2[byte#73783, short#73784, int#73785, long#73786L, float#73787, double#73788, decimal#73789, string#73790, binary#73791, bool#73792, date#73793, timestamp#73794, array#73795, struct#73796, map#73797] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [byte#73753] +- Join Inner, (byte#73753 = byte#73783) :- Filter isnotnull(byte#73753) : +- RelationV2[byte#73753] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(byte#73783) +- RelationV2[byte#73783] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2449) ProjectExecTransformer [byte#73753] +- ^(2449) BroadcastHashJoinExecTransformer [byte#73753], [byte#73783], Inner, BuildRight, false :- ^(2449) FilterExecTransformer isnotnull(byte#73753) : +- ^(2449) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73753] OrcScan DataFilters: [isnotnull(byte#73753)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73753)] +- ^(2449) InputIteratorTransformer[byte#73783] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=556688] +- ^(2448) FilterExecTransformer isnotnull(byte#73783) +- ^(2448) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73783] OrcScan DataFilters: [isnotnull(byte#73783)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73783)] +- == Initial Plan == Project [byte#73753] +- BroadcastHashJoin [byte#73753], [byte#73783], Inner, BuildRight, false :- Filter isnotnull(byte#73753) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73753] OrcScan DataFilters: [isnotnull(byte#73753)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=556479] +- Filter isnotnull(byte#73783) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73783] OrcScan DataFilters: [isnotnull(byte#73783)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxParquetDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L190
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#77734] +- Filter (int#77734 = int#77764) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#77732,short#77733,int#77734,long#77735L,float#77736,double#77737,decimal#77738,string#77739,binary#77740,bool#77741,date#77742,timestamp#77743,array#77744,struct#77745,map#77746]) : +- RelationV2[byte#77732, short#77733, int#77734, long#77735L, float#77736, double#77737, decimal#77738, string#77739, binary#77740, bool#77741, date#77742, timestamp#77743, array#77744, struct#77745, map#77746] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#77762,short#77763,int#77764,long#77765L,float#77766,double#77767,decimal#77768,string#77769,binary#77770,bool#77771,date#77772,timestamp#77773,array#77774,struct#77775,map#77776]) +- RelationV2[byte#77762, short#77763, int#77764, long#77765L, float#77766, double#77767, decimal#77768, string#77769, binary#77770, bool#77771, date#77772, timestamp#77773, array#77774, struct#77775, map#77776] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [int#77734] +- Join Inner, (int#77734 = int#77764) :- Filter isnotnull(int#77734) : +- RelationV2[int#77734] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(int#77764) +- RelationV2[int#77764] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2578) ProjectExecTransformer [int#77734] +- ^(2578) BroadcastHashJoinExecTransformer [int#77734], [int#77764], Inner, BuildRight, false :- ^(2578) FilterExecTransformer isnotnull(int#77734) : +- ^(2578) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77734] ParquetScan DataFilters: [isnotnull(int#77734)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77734)] +- ^(2578) InputIteratorTransformer[int#77764] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=569640] +- ^(2577) FilterExecTransformer isnotnull(int#77764) +- ^(2577) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77764] ParquetScan DataFilters: [isnotnull(int#77764)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77764)] +- == Initial Plan == Project [int#77734] +- BroadcastHashJoin [int#77734], [int#77764], Inner, BuildRight, false :- Filter isnotnull(int#77734) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77734] ParquetScan DataFilters: [isnotnull(int#77734)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=569431] +- Filter isnotnull(int#77764) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77764] ParquetScan DataFilters: [isnotnull(int#77764)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxParquetDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#77733] +- Filter (short#77733 = short#77763) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#77732,short#77733,int#77734,long#77735L,float#77736,double#77737,decimal#77738,string#77739,binary#77740,bool#77741,date#77742,timestamp#77743,array#77744,struct#77745,map#77746]) : +- RelationV2[byte#77732, short#77733, int#77734, long#77735L, float#77736, double#77737, decimal#77738, string#77739, binary#77740, bool#77741, date#77742, timestamp#77743, array#77744, struct#77745, map#77746] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#77762,short#77763,int#77764,long#77765L,float#77766,double#77767,decimal#77768,string#77769,binary#77770,bool#77771,date#77772,timestamp#77773,array#77774,struct#77775,map#77776]) +- RelationV2[byte#77762, short#77763, int#77764, long#77765L, float#77766, double#77767, decimal#77768, string#77769, binary#77770, bool#77771, date#77772, timestamp#77773, array#77774, struct#77775, map#77776] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [short#77733] +- Join Inner, (short#77733 = short#77763) :- Filter isnotnull(short#77733) : +- RelationV2[short#77733] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(short#77763) +- RelationV2[short#77763] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2587) ProjectExecTransformer [short#77733] +- ^(2587) BroadcastHashJoinExecTransformer [short#77733], [short#77763], Inner, BuildRight, false :- ^(2587) FilterExecTransformer isnotnull(short#77733) : +- ^(2587) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77733] ParquetScan DataFilters: [isnotnull(short#77733)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77733)] +- ^(2587) InputIteratorTransformer[short#77763] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=570555] +- ^(2586) FilterExecTransformer isnotnull(short#77763) +- ^(2586) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77763] ParquetScan DataFilters: [isnotnull(short#77763)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77763)] +- == Initial Plan == Project [short#77733] +- BroadcastHashJoin [short#77733], [short#77763], Inner, BuildRight, false :- Filter isnotnull(short#77733) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[short#77733] ParquetScan DataFilters: [isnotnull(short#77733)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=570346] +- Filter isnotnull(short#77763) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[short#77763] ParquetScan DataFilters: [isnotnull(short#77763)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxParquetDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#77732] +- Filter (byte#77732 = byte#77762) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#77732,short#77733,int#77734,long#77735L,float#77736,double#77737,decimal#77738,string#77739,binary#77740,bool#77741,date#77742,timestamp#77743,array#77744,struct#77745,map#77746]) : +- RelationV2[byte#77732, short#77733, int#77734, long#77735L, float#77736, double#77737, decimal#77738, string#77739, binary#77740, bool#77741, date#77742, timestamp#77743, array#77744, struct#77745, map#77746] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#77762,short#77763,int#77764,long#77765L,float#77766,double#77767,decimal#77768,string#77769,binary#77770,bool#77771,date#77772,timestamp#77773,array#77774,struct#77775,map#77776]) +- RelationV2[byte#77762, short#77763, int#77764, long#77765L, float#77766, double#77767, decimal#77768, string#77769, binary#77770, bool#77771, date#77772, timestamp#77773, array#77774, struct#77775, map#77776] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [byte#77732] +- Join Inner, (byte#77732 = byte#77762) :- Filter isnotnull(byte#77732) : +- RelationV2[byte#77732] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(byte#77762) +- RelationV2[byte#77762] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2637) ProjectExecTransformer [byte#77732] +- ^(2637) BroadcastHashJoinExecTransformer [byte#77732], [byte#77762], Inner, BuildRight, false :- ^(2637) FilterExecTransformer isnotnull(byte#77732) : +- ^(2637) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77732] ParquetScan DataFilters: [isnotnull(byte#77732)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77732)] +- ^(2637) InputIteratorTransformer[byte#77762] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=575205] +- ^(2636) FilterExecTransformer isnotnull(byte#77762) +- ^(2636) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77762] ParquetScan DataFilters: [isnotnull(byte#77762)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77762)] +- == Initial Plan == Project [byte#77732] +- BroadcastHashJoin [byte#77732], [byte#77762], Inner, BuildRight, false :- Filter isnotnull(byte#77732) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77732] ParquetScan DataFilters: [isnotnull(byte#77732)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=574996] +- Filter isnotnull(byte#77762) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77762] ParquetScan DataFilters: [isnotnull(byte#77762)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxOrcDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L212
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#73734] +- Filter (short#73734 = short#73764) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73733,short#73734,int#73735,long#73736L,float#73737,double#73738,decimal#73739,string#73740,binary#73741,bool#73742,date#73743,timestamp#73744,array#73745,struct#73746,map#73747]) : +- RelationV2[byte#73733, short#73734, int#73735, long#73736L, float#73737, double#73738, decimal#73739, string#73740, binary#73741, bool#73742, date#73743, timestamp#73744, array#73745, struct#73746, map#73747] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73763,short#73764,int#73765,long#73766L,float#73767,double#73768,decimal#73769,string#73770,binary#73771,bool#73772,date#73773,timestamp#73774,array#73775,struct#73776,map#73777]) +- RelationV2[byte#73763, short#73764, int#73765, long#73766L, float#73767, double#73768, decimal#73769, string#73770, binary#73771, bool#73772, date#73773, timestamp#73774, array#73775, struct#73776, map#73777] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [short#73734] +- Join Inner, (short#73734 = short#73764) :- Filter isnotnull(short#73734) : +- RelationV2[short#73734] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(short#73764) +- RelationV2[short#73764] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2390) ProjectExecTransformer [short#73734] +- ^(2390) BroadcastHashJoinExecTransformer [short#73734], [short#73764], Inner, BuildRight, false :- ^(2390) FilterExecTransformer isnotnull(short#73734) : +- ^(2390) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73734] OrcScan DataFilters: [isnotnull(short#73734)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73734)] +- ^(2390) InputIteratorTransformer[short#73764] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=529660] +- ^(2389) FilterExecTransformer isnotnull(short#73764) +- ^(2389) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73764] OrcScan DataFilters: [isnotnull(short#73764)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73764)] +- == Initial Plan == Project [short#73734] +- BroadcastHashJoin [short#73734], [short#73764], Inner, BuildRight, false :- Filter isnotnull(short#73734) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[short#73734] OrcScan DataFilters: [isnotnull(short#73734)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=529451] +- Filter isnotnull(short#73764) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[short#73764] OrcScan DataFilters: [isnotnull(short#73764)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#73733] +- Filter (byte#73733 = byte#73763) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73733,short#73734,int#73735,long#73736L,float#73737,double#73738,decimal#73739,string#73740,binary#73741,bool#73742,date#73743,timestamp#73744,array#73745,struct#73746,map#73747]) : +- RelationV2[byte#73733, short#73734, int#73735, long#73736L, float#73737, double#73738, decimal#73739, string#73740, binary#73741, bool#73742, date#73743, timestamp#73744, array#73745, struct#73746, map#73747] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73763,short#73764,int#73765,long#73766L,float#73767,double#73768,decimal#73769,string#73770,binary#73771,bool#73772,date#73773,timestamp#73774,array#73775,struct#73776,map#73777]) +- RelationV2[byte#73763, short#73764, int#73765, long#73766L, float#73767, double#73768, decimal#73769, string#73770, binary#73771, bool#73772, date#73773, timestamp#73774, array#73775, struct#73776, map#73777] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [byte#73733] +- Join Inner, (byte#73733 = byte#73763) :- Filter isnotnull(byte#73733) : +- RelationV2[byte#73733] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(byte#73763) +- RelationV2[byte#73763] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2440) ProjectExecTransformer [byte#73733] +- ^(2440) BroadcastHashJoinExecTransformer [byte#73733], [byte#73763], Inner, BuildRight, false :- ^(2440) FilterExecTransformer isnotnull(byte#73733) : +- ^(2440) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73733] OrcScan DataFilters: [isnotnull(byte#73733)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73733)] +- ^(2440) InputIteratorTransformer[byte#73763] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=534310] +- ^(2439) FilterExecTransformer isnotnull(byte#73763) +- ^(2439) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73763] OrcScan DataFilters: [isnotnull(byte#73763)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73763)] +- == Initial Plan == Project [byte#73733] +- BroadcastHashJoin [byte#73733], [byte#73763], Inner, BuildRight, false :- Filter isnotnull(byte#73733) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc[byte#73733] OrcScan DataFilters: [isnotnull(byte#73733)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=534101] +- Filter isnotnull(byte#73763) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2_orc[byte#73763] OrcScan DataFilters: [isnotnull(byte#73763)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxParquetDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L184
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#77504] +- Filter (int#77504 = int#77534) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#77502,short#77503,int#77504,long#77505L,float#77506,double#77507,decimal#77508,string#77509,binary#77510,bool#77511,date#77512,timestamp#77513,array#77514,struct#77515,map#77516]) : +- RelationV2[byte#77502, short#77503, int#77504, long#77505L, float#77506, double#77507, decimal#77508, string#77509, binary#77510, bool#77511, date#77512, timestamp#77513, array#77514, struct#77515, map#77516] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#77532,short#77533,int#77534,long#77535L,float#77536,double#77537,decimal#77538,string#77539,binary#77540,bool#77541,date#77542,timestamp#77543,array#77544,struct#77545,map#77546]) +- RelationV2[byte#77532, short#77533, int#77534, long#77535L, float#77536, double#77537, decimal#77538, string#77539, binary#77540, bool#77541, date#77542, timestamp#77543, array#77544, struct#77545, map#77546] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [int#77504] +- Join Inner, (int#77504 = int#77534) :- Filter isnotnull(int#77504) : +- RelationV2[int#77504] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(int#77534) +- RelationV2[int#77534] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2565) ProjectExecTransformer [int#77504] +- ^(2565) BroadcastHashJoinExecTransformer [int#77504], [int#77534], Inner, BuildRight, false :- ^(2565) FilterExecTransformer isnotnull(int#77504) : +- ^(2565) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77504] ParquetScan DataFilters: [isnotnull(int#77504)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77504)] +- ^(2565) InputIteratorTransformer[int#77534] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=546874] +- ^(2564) FilterExecTransformer isnotnull(int#77534) +- ^(2564) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77534] ParquetScan DataFilters: [isnotnull(int#77534)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77534)] +- == Initial Plan == Project [int#77504] +- BroadcastHashJoin [int#77504], [int#77534], Inner, BuildRight, false :- Filter isnotnull(int#77504) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[int#77504] ParquetScan DataFilters: [isnotnull(int#77504)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=546665] +- Filter isnotnull(int#77534) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[int#77534] ParquetScan DataFilters: [isnotnull(int#77534)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxParquetDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L321
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#77502] +- Filter (byte#77502 = byte#77532) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#77502,short#77503,int#77504,long#77505L,float#77506,double#77507,decimal#77508,string#77509,binary#77510,bool#77511,date#77512,timestamp#77513,array#77514,struct#77515,map#77516]) : +- RelationV2[byte#77502, short#77503, int#77504, long#77505L, float#77506, double#77507, decimal#77508, string#77509, binary#77510, bool#77511, date#77512, timestamp#77513, array#77514, struct#77515, map#77516] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#77532,short#77533,int#77534,long#77535L,float#77536,double#77537,decimal#77538,string#77539,binary#77540,bool#77541,date#77542,timestamp#77543,array#77544,struct#77545,map#77546]) +- RelationV2[byte#77532, short#77533, int#77534, long#77535L, float#77536, double#77537, decimal#77538, string#77539, binary#77540, bool#77541, date#77542, timestamp#77543, array#77544, struct#77545, map#77546] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [byte#77502] +- Join Inner, (byte#77502 = byte#77532) :- Filter isnotnull(byte#77502) : +- RelationV2[byte#77502] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1 +- Filter isnotnull(byte#77532) +- RelationV2[byte#77532] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2628) ProjectExecTransformer [byte#77502] +- ^(2628) BroadcastHashJoinExecTransformer [byte#77502], [byte#77532], Inner, BuildRight, false :- ^(2628) FilterExecTransformer isnotnull(byte#77502) : +- ^(2628) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77502] ParquetScan DataFilters: [isnotnull(byte#77502)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77502)] +- ^(2628) InputIteratorTransformer[byte#77532] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=552827] +- ^(2627) FilterExecTransformer isnotnull(byte#77532) +- ^(2627) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77532] ParquetScan DataFilters: [isnotnull(byte#77532)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#77532)] +- == Initial Plan == Project [byte#77502] +- BroadcastHashJoin [byte#77502], [byte#77532], Inner, BuildRight, false :- Filter isnotnull(byte#77502) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1[byte#77502] ParquetScan DataFilters: [isnotnull(byte#77502)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=552618] +- Filter isnotnull(byte#77532) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type2[byte#77532] ParquetScan DataFilters: [isnotnull(byte#77532)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxOrcDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L218
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#73209] +- Filter (short#73209 = short#73239) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73208,short#73209,int#73210,long#73211L,float#73212,double#73213,decimal#73214,string#73215,binary#73216,bool#73217,date#73218,timestamp#73219,array#73220,struct#73221,map#73222]) : +- RelationV2[byte#73208, short#73209, int#73210, long#73211L, float#73212, double#73213, decimal#73214, string#73215, binary#73216, bool#73217, date#73218, timestamp#73219, array#73220, struct#73221, map#73222] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73238,short#73239,int#73240,long#73241L,float#73242,double#73243,decimal#73244,string#73245,binary#73246,bool#73247,date#73248,timestamp#73249,array#73250,struct#73251,map#73252]) +- RelationV2[byte#73238, short#73239, int#73240, long#73241L, float#73242, double#73243, decimal#73244, string#73245, binary#73246, bool#73247, date#73248, timestamp#73249, array#73250, struct#73251, map#73252] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [short#73209] +- Join Inner, (short#73209 = short#73239) :- Filter isnotnull(short#73209) : +- RelationV2[short#73209] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(short#73239) +- RelationV2[short#73239] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2602) ProjectExecTransformer [short#73209] +- ^(2602) BroadcastHashJoinExecTransformer [short#73209], [short#73239], Inner, BuildRight, false :- ^(2602) FilterExecTransformer isnotnull(short#73209) : +- ^(2602) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[short#73209] OrcScan DataFilters: [isnotnull(short#73209)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73209)] +- ^(2602) InputIteratorTransformer[short#73239] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=526457] +- ^(2601) FilterExecTransformer isnotnull(short#73239) +- ^(2601) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[short#73239] OrcScan DataFilters: [isnotnull(short#73239)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#73239)] +- == Initial Plan == Project [short#73209] +- BroadcastHashJoin [short#73209], [short#73239], Inner, BuildRight, false :- Filter isnotnull(short#73209) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[short#73209] OrcScan DataFilters: [isnotnull(short#73209)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=526248] +- Filter isnotnull(short#73239) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[short#73239] OrcScan DataFilters: [isnotnull(short#73239)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]
VeloxOrcDataTypeValidationSuite.Byte type: org/apache/gluten/execution/VeloxOrcDataTypeValidationSuite#L322
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.byte] +- 'Filter ('type1.byte = 'type2.byte) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == byte: tinyint Project [byte#73208] +- Filter (byte#73208 = byte#73238) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#73208,short#73209,int#73210,long#73211L,float#73212,double#73213,decimal#73214,string#73215,binary#73216,bool#73217,date#73218,timestamp#73219,array#73220,struct#73221,map#73222]) : +- RelationV2[byte#73208, short#73209, int#73210, long#73211L, float#73212, double#73213, decimal#73214, string#73215, binary#73216, bool#73217, date#73218, timestamp#73219, array#73220, struct#73221, map#73222] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc +- SubqueryAlias type2 +- View (`type2`, [byte#73238,short#73239,int#73240,long#73241L,float#73242,double#73243,decimal#73244,string#73245,binary#73246,bool#73247,date#73248,timestamp#73249,array#73250,struct#73251,map#73252]) +- RelationV2[byte#73238, short#73239, int#73240, long#73241L, float#73242, double#73243, decimal#73244, string#73245, binary#73246, bool#73247, date#73248, timestamp#73249, array#73250, struct#73251, map#73252] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc == Optimized Logical Plan == Project [byte#73208] +- Join Inner, (byte#73208 = byte#73238) :- Filter isnotnull(byte#73208) : +- RelationV2[byte#73208] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc +- Filter isnotnull(byte#73238) +- RelationV2[byte#73238] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2652) ProjectExecTransformer [byte#73208] +- ^(2652) BroadcastHashJoinExecTransformer [byte#73208], [byte#73238], Inner, BuildRight, false :- ^(2652) FilterExecTransformer isnotnull(byte#73208) : +- ^(2652) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[byte#73208] OrcScan DataFilters: [isnotnull(byte#73208)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73208)] +- ^(2652) InputIteratorTransformer[byte#73238] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=531107] +- ^(2651) FilterExecTransformer isnotnull(byte#73238) +- ^(2651) BatchScanTransformer orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[byte#73238] OrcScan DataFilters: [isnotnull(byte#73238)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] NativeFilters: [isnotnull(byte#73238)] +- == Initial Plan == Project [byte#73208] +- BroadcastHashJoin [byte#73208], [byte#73238], Inner, BuildRight, false :- Filter isnotnull(byte#73208) : +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1_orc[byte#73208] OrcScan DataFilters: [isnotnull(byte#73208)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, tinyint, false] as bigint)),false), [plan_id=530898] +- Filter isnotnull(byte#73238) +- BatchScan orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2_orc[byte#73238] OrcScan DataFilters: [isnotnull(byte#73238)], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(byte)], PushedGroupBy: [], ReadSchema: struct<byte:tinyint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![1]
VeloxParquetDataTypeValidationSuite.Int type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L190
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.int] +- 'Filter ('type1.int = 'type2.int) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == int: int Project [int#77084] +- Filter (int#77084 = int#77114) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#77082,short#77083,int#77084,long#77085L,float#77086,double#77087,decimal#77088,string#77089,binary#77090,bool#77091,date#77092,timestamp#77093,array#77094,struct#77095,map#77096]) : +- RelationV2[byte#77082, short#77083, int#77084, long#77085L, float#77086, double#77087, decimal#77088, string#77089, binary#77090, bool#77091, date#77092, timestamp#77093, array#77094, struct#77095, map#77096] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#77112,short#77113,int#77114,long#77115L,float#77116,double#77117,decimal#77118,string#77119,binary#77120,bool#77121,date#77122,timestamp#77123,array#77124,struct#77125,map#77126]) +- RelationV2[byte#77112, short#77113, int#77114, long#77115L, float#77116, double#77117, decimal#77118, string#77119, binary#77120, bool#77121, date#77122, timestamp#77123, array#77124, struct#77125, map#77126] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [int#77084] +- Join Inner, (int#77084 = int#77114) :- Filter isnotnull(int#77084) : +- RelationV2[int#77084] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1 +- Filter isnotnull(int#77114) +- RelationV2[int#77114] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2781) ProjectExecTransformer [int#77084] +- ^(2781) BroadcastHashJoinExecTransformer [int#77084], [int#77114], Inner, BuildRight, false :- ^(2781) FilterExecTransformer isnotnull(int#77084) : +- ^(2781) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[int#77084] ParquetScan DataFilters: [isnotnull(int#77084)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77084)] +- ^(2781) InputIteratorTransformer[int#77114] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=544059] +- ^(2780) FilterExecTransformer isnotnull(int#77114) +- ^(2780) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[int#77114] ParquetScan DataFilters: [isnotnull(int#77114)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] NativeFilters: [isnotnull(int#77114)] +- == Initial Plan == Project [int#77084] +- BroadcastHashJoin [int#77084], [int#77114], Inner, BuildRight, false :- Filter isnotnull(int#77084) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[int#77084] ParquetScan DataFilters: [isnotnull(int#77084)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=543850] +- Filter isnotnull(int#77114) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[int#77114] ParquetScan DataFilters: [isnotnull(int#77114)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(int)], PushedGroupBy: [], ReadSchema: struct<int:int> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![3]
VeloxParquetDataTypeValidationSuite.Short type: org/apache/gluten/execution/VeloxParquetDataTypeValidationSuite#L211
Results do not match for query: Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] Timezone Env: == Parsed Logical Plan == 'Project ['type1.short] +- 'Filter ('type1.short = 'type2.short) +- 'Join Inner :- 'UnresolvedRelation [type1], [], false +- 'UnresolvedRelation [type2], [], false == Analyzed Logical Plan == short: smallint Project [short#77083] +- Filter (short#77083 = short#77113) +- Join Inner :- SubqueryAlias type1 : +- View (`type1`, [byte#77082,short#77083,int#77084,long#77085L,float#77086,double#77087,decimal#77088,string#77089,binary#77090,bool#77091,date#77092,timestamp#77093,array#77094,struct#77095,map#77096]) : +- RelationV2[byte#77082, short#77083, int#77084, long#77085L, float#77086, double#77087, decimal#77088, string#77089, binary#77090, bool#77091, date#77092, timestamp#77093, array#77094, struct#77095, map#77096] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1 +- SubqueryAlias type2 +- View (`type2`, [byte#77112,short#77113,int#77114,long#77115L,float#77116,double#77117,decimal#77118,string#77119,binary#77120,bool#77121,date#77122,timestamp#77123,array#77124,struct#77125,map#77126]) +- RelationV2[byte#77112, short#77113, int#77114, long#77115L, float#77116, double#77117, decimal#77118, string#77119, binary#77120, bool#77121, date#77122, timestamp#77123, array#77124, struct#77125, map#77126] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2 == Optimized Logical Plan == Project [short#77083] +- Join Inner, (short#77083 = short#77113) :- Filter isnotnull(short#77083) : +- RelationV2[short#77083] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1 +- Filter isnotnull(short#77113) +- RelationV2[short#77113] parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2 == Physical Plan == AdaptiveSparkPlan isFinalPlan=true +- == Final Plan == VeloxColumnarToRow +- ^(2790) ProjectExecTransformer [short#77083] +- ^(2790) BroadcastHashJoinExecTransformer [short#77083], [short#77113], Inner, BuildRight, false :- ^(2790) FilterExecTransformer isnotnull(short#77083) : +- ^(2790) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[short#77083] ParquetScan DataFilters: [isnotnull(short#77083)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77083)] +- ^(2790) InputIteratorTransformer[short#77113] +- BroadcastQueryStage 0 +- ColumnarBroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=544974] +- ^(2789) FilterExecTransformer isnotnull(short#77113) +- ^(2789) BatchScanTransformer parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[short#77113] ParquetScan DataFilters: [isnotnull(short#77113)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] NativeFilters: [isnotnull(short#77113)] +- == Initial Plan == Project [short#77083] +- BroadcastHashJoin [short#77083], [short#77113], Inner, BuildRight, false :- Filter isnotnull(short#77083) : +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type1[short#77083] ParquetScan DataFilters: [isnotnull(short#77083)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, smallint, false] as bigint)),false), [plan_id=544765] +- Filter isnotnull(short#77113) +- BatchScan parquet file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.13/test-classes/data-type-validation-data/type2[short#77113] ParquetScan DataFilters: [isnotnull(short#77113)], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [IsNotNull(short)], PushedGroupBy: [], ReadSchema: struct<short:smallint> RuntimeFilters: [] == Results == == Results == !== Correct Answer - 1 == == Gluten Answer - 0 == struct<> struct<> ![2]