You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In Spark, if the configuration setting spark.sql.legacy.statisticalAggregate is set to true, statistical aggregation function will return Double.NaN instead of NULL when DivideByZero occurs during expression evaluation. For example, stddev(2.0) will return NaN in Spark but return NULL in Velox. Statistical aggregation function includes stddev, stddev_samp, variance, var_samp, skewness, kurtosis, covar_samp, corr.
Bug description
In Spark, if the configuration setting
spark.sql.legacy.statisticalAggregate
is set totrue
, statistical aggregation function will returnDouble.NaN
instead ofNULL
when DivideByZero occurs during expression evaluation. For example, stddev(2.0) will returnNaN
in Spark but returnNULL
in Velox. Statistical aggregation function includes stddev, stddev_samp, variance, var_samp, skewness, kurtosis, covar_samp, corr.Spark PR: apache/spark#29983
System information
/
Relevant logs
The text was updated successfully, but these errors were encountered: