feat: Replace tz implementation from date with LLVM implementation [1/4] (#12422)

Summary:
Pull Request resolved: #12422

We currently use (slightly modified versions of) the tz.h/tz.cpp (along with ios.h/tz_private.h) files
from https://github.com/HowardHinnant/date to handle time zone conversions.  A version of this
API was voted into the C++20 standard. Since then further support/development of the library has
been limited.

In particular it has a known limitation when daylight savings time rules used to determine transition
dates/times into the distant future are not supported in the library. This means Velox only supports
datetimes in time zones with DST up to the year 2037.

LLVM has an implementation of the new C++20 standard that is heavily based on that in the date
repo. It is not finished yet, and still considered "experimental" but it already provides some notable
improvements, e.g. support of DST into the distant future.

In order to take advantage of this and prepare us to adopt the C++20 library once it's available,
I've replace the tz.h/tz.cpp files with the set of files from LLVM that implement the API in
std::chrono in C++20.

This first PR simply pulls in the files from LLVM, removes some libcpp specific macros, replaces C++20-isms with C++17 equivalents, and updates namespaces and includes.  The only meaningful code change in these files is I ported the logic to find the necessary zoneinfo files/directories on Mac from the date library as LLVM only supports finding them in Linux systems.

Beyond that I updated includes/namspaces in the rest of the velox codebase to use the new files.  I commented out/modified some code (mostly tests) that are broken by the change, in case I include a comment indicating the follow up PR that addresses the issue and restores the code.  I will land these as a single stack, I did this to simplify the review process and ensure each PR individually builds and passes tests.

Note that we still use the date.h and ios_week.h files from https://github.com/HowardHinnant/date
for two reasons:
1) These were not fully adopted by the C++20 standard and we use some of the types that did not
make it into the standard.
2) There's some discussion of supporting a range of years beyond what std::year supports to
match Presto Java's behavior, which will require forking these anyway.

Differential Revision: D69997637

Author: Kevin Wilfong

velox/CMakeLists.txt

@@ -23,6 +23,7 @@ add_subdirectory(vector)
 add_subdirectory(row)
 add_subdirectory(flag_definitions)
 add_subdirectory(external/date)
+add_subdirectory(external/tzdb)
 add_subdirectory(external/md5)
 add_subdirectory(external/hdfs)
 #

velox/docs/develop/timestamp.rst

@@ -73,9 +73,9 @@ used to efficiently represent timezones, preventing the use of inefficient
 timezone string names like ``America/Los_Angeles``. Considering there are about
 2k valid timezone definitions, 12 bits are enough to represent timezone IDs. 
 
-Timezone IDs in Velox are based on the id map used by Presto and are 
-`available here <https://github.com/prestodb/presto/blob/master/presto-common/src/main/resources/com/facebook/presto/common/type/zone-index.properties>`_. 
-They are automatically generated using `this script <https://github.com/facebookincubator/velox/blob/main/velox/type/tz/gen_timezone_database.py>`_. 
+Timezone IDs in Velox are based on the id map used by Presto and are
+`available here <https://github.com/prestodb/presto/blob/master/presto-common/src/main/resources/com/facebook/presto/common/type/zone-index.properties>`_.
+They are automatically generated using `this script <https://github.com/facebookincubator/velox/blob/main/velox/type/tz/gen_timezone_database.py>`_.
 While timezone IDs are an implementation detail and ideally should not leak
 outside of Velox execution, they are exposed if data containing
 TimestampWithTimezones are serialized, for example.
@@ -98,7 +98,7 @@ ID).
 
 However, unpacking/converting a TimestampWithTimezone into an absolute time
 definition requires a
-`timezone conversion <https://github.com/facebookincubator/velox/blob/main/velox/functions/prestosql/DateTimeFunctions.h#L74-L84>`_. 
+`timezone conversion <https://github.com/facebookincubator/velox/blob/main/velox/functions/prestosql/DateTimeFunctions.h#L74-L84>`_.
 
 Conversions Across Timezones
 ----------------------------
@@ -119,7 +119,7 @@ for Linux. 
 In Velox, Timezone conversions are done using std::chrono. Starting in C++20,
 std::chrono `supports conversion of timestamp across timezones <https://en.cppreference.com/w/cpp/chrono/time_zone>`_.
 To support older versions of the C++ standard, in Velox we vendor an
-implementation of this API at `velox/external/date/ <https://github.com/facebookincubator/velox/tree/main/velox/external/date>`_.
+implementation of this API at `velox/external/tzdb/ <https://github.com/facebookincubator/velox/tree/main/velox/external/tzdb>`_.
 This class handles timezone conversions by leveraging APIs provided by the
 operating system, based on the tzdata database installed locally. If systems
 happen to have inconsistent or older versions of the tzdata database, Velox’s
@@ -153,23 +153,23 @@ on the string on not:
 ::
 
   SELECT typeof(TIMESTAMP '1970-01-01 00:00:00'); -- timestamp
-  SELECT typeof(TIMESTAMP '1970-01-01 00:00:00 UTC'); -- timestamp with time zone 
+  SELECT typeof(TIMESTAMP '1970-01-01 00:00:00 UTC'); -- timestamp with time zone
 
 Converting a TimestampWithTimezone into a Timestamp works by dropping the
 timezone information and returning only the timestamp portion:
 
 ::
 
-  SELECT cast(TIMESTAMP '1970-01-01 00:00:00 UTC' as timestamp); -- 1970-01-01 00:00:00.000 
-  SELECT cast(TIMESTAMP '1970-01-01 00:00:00 America/New_York' as timestamp); -- 1970-01-01 00:00:00.000 
+  SELECT cast(TIMESTAMP '1970-01-01 00:00:00 UTC' as timestamp); -- 1970-01-01 00:00:00.000
+  SELECT cast(TIMESTAMP '1970-01-01 00:00:00 America/New_York' as timestamp); -- 1970-01-01 00:00:00.000
 
 To convert a Timestamp into a TimestampWithTimezone, one needs to specify a
 timezone. In Presto, the session timezone is used by default:
 
 ::
 
   SELECT current_timezone(); -- America/Los_Angeles
-  SELECT cast(TIMESTAMP '1970-01-01 00:00:00' as timestamp with time zone); -- 1970-01-01 00:00:00.000 America/Los_Angeles 
+  SELECT cast(TIMESTAMP '1970-01-01 00:00:00' as timestamp with time zone); -- 1970-01-01 00:00:00.000 America/Los_Angeles
 
 Conversion across TimestampWithTimezone can be done using the AT TIME ZONE
 construct. 
@@ -180,15 +180,15 @@ the clock/calendar read at the target timezone (Los Angeles)?
 
 ::
 
-  SELECT TIMESTAMP '1970-01-01 00:00:00 UTC' AT TIME ZONE 'America/Los_Angeles'; -- 1969-12-31 16:00:00.000 America/Los_Angeles 
-  SELECT TIMESTAMP '1970-01-01 00:00:00 UTC' AT TIME ZONE 'UTC'; -- 1970-01-01 00:00:00.000 UTC 
+  SELECT TIMESTAMP '1970-01-01 00:00:00 UTC' AT TIME ZONE 'America/Los_Angeles'; -- 1969-12-31 16:00:00.000 America/Los_Angeles
+  SELECT TIMESTAMP '1970-01-01 00:00:00 UTC' AT TIME ZONE 'UTC'; -- 1970-01-01 00:00:00.000 UTC
 
 Strings can be converted into Timestamp and TimestampWithTimezone:
 
 ::
 
-  SELECT cast('1970-01-01 00:00:00' as timestamp); -- 1970-01-01 00:00:00.000 
-  SELECT cast('1970-01-01 00:00:00 America/Los_Angeles' as timestamp with time zone); -- 1970-01-01 00:00:00.000 America/Los_Angeles 
+  SELECT cast('1970-01-01 00:00:00' as timestamp); -- 1970-01-01 00:00:00.000
+  SELECT cast('1970-01-01 00:00:00 America/Los_Angeles' as timestamp with time zone); -- 1970-01-01 00:00:00.000 America/Los_Angeles
 
 One can also convert a TimestampWithTimezone into a unix epoch/time. The
 semantic of this operation is: at the absolute point in time described by the
@@ -197,16 +197,16 @@ that unix epoch is the number of seconds since ``1970-01-01 00:00:00`` in UTC:
 
 ::
 
-  SELECT to_unixtime(TIMESTAMP '1970-01-01 00:00:00 UTC'); -- 0.0 
-  SELECT to_unixtime(TIMESTAMP '1970-01-01 00:00:00 America/Los_Angeles'); -- 28800.0 
+  SELECT to_unixtime(TIMESTAMP '1970-01-01 00:00:00 UTC'); -- 0.0
+  SELECT to_unixtime(TIMESTAMP '1970-01-01 00:00:00 America/Los_Angeles'); -- 28800.0
 
 The opposite conversion can be achieved using ``from_unixtime()``. The function
 may take an optional second parameter to specify the timezone, having the same
 semantic as AT TIME ZONE described above:
 
 ::
 
-  SELECT from_unixtime(0); -- 1970-01-01 00:00:00.000 
+  SELECT from_unixtime(0); -- 1970-01-01 00:00:00.000
   SELECT from_unixtime(0, 'UTC'); -- 1970-01-01 00:00:00.000 UTC 
   SELECT from_unixtime(0, 'America/Los_Angeles'); -- 1969-12-31 16:00:00.000 America/Los_Angeles
 
@@ -225,8 +225,8 @@ timestamps have a different semantic:
 ::
 
   SET SESSION legacy_timestamp = true;
-  SELECT cast(TIMESTAMP '1970-01-01 00:00:00 UTC' as timestamp); -- 1969-12-31 16:00:00.000 
-  SELECT cast('1970-01-01 00:00:00 UTC' as timestamp); -- 1969-12-31 16:00:00.000 
+  SELECT cast(TIMESTAMP '1970-01-01 00:00:00 UTC' as timestamp); -- 1969-12-31 16:00:00.000
+  SELECT cast('1970-01-01 00:00:00 UTC' as timestamp); -- 1969-12-31 16:00:00.000
 
 To support the two timestamp semantics, the
 ``core::QueryConfig::kAdjustTimestampToTimezone`` query flag was added to Velox.

velox/dwio/parquet/tests/reader/ParquetTableScanTest.cpp

@@ -25,7 +25,6 @@
 #include "velox/exec/tests/utils/HiveConnectorTestBase.h" // @manual
 #include "velox/exec/tests/utils/PlanBuilder.h"
 #include "velox/exec/tests/utils/TempDirectoryPath.h"
-#include "velox/external/date/tz.h"
 #include "velox/type/tests/SubfieldFiltersBuilder.h"
 #include "velox/type/tz/TimeZoneMap.h"
 

velox/expression/CastExpr-inl.h

@@ -19,7 +19,6 @@
 #include "velox/common/base/Exceptions.h"
 #include "velox/core/CoreTypeSystem.h"
 #include "velox/expression/StringWriter.h"
-#include "velox/external/date/tz.h"
 #include "velox/type/Type.h"
 #include "velox/vector/SelectivityVector.h"
 

velox/expression/tests/CastExprTest.cpp

@@ -656,14 +656,17 @@ TEST_F(CastExprTest, stringToTimestamp) {
       (evaluateOnce<Timestamp, std::string>(
           "try_cast(c0 as timestamp)", "201915-04-23 11:46:00.000")),
       "Timepoint is outside of supported year range");
-  VELOX_ASSERT_THROW(
-      (evaluateOnce<Timestamp, std::string>(
-          "cast(c0 as timestamp)", "2045-12-31 18:00:00")),
-      "Unable to convert timezone 'America/Los_Angeles' past 2037-11-01 09:00:00");
-  VELOX_ASSERT_THROW(
-      (evaluateOnce<Timestamp, std::string>(
-          "try_cast(c0 as timestamp)", "2045-12-31 18:00:00")),
-      "Unable to convert timezone 'America/Los_Angeles' past 2037-11-01 09:00:00");
+  // TODO: Address in https://github.com/facebookincubator/velox/pull/12471
+  //   VELOX_ASSERT_THROW(
+  //       (evaluateOnce<Timestamp, std::string>(
+  //           "cast(c0 as timestamp)", "2045-12-31 18:00:00")),
+  //       "Unable to convert timezone 'America/Los_Angeles' past 2037-11-01
+  //       09:00:00");
+  //   VELOX_ASSERT_THROW(
+  //       (evaluateOnce<Timestamp, std::string>(
+  //           "try_cast(c0 as timestamp)", "2045-12-31 18:00:00")),
+  //       "Unable to convert timezone 'America/Los_Angeles' past 2037-11-01
+  //       09:00:00");
   // Only one white space is allowed before the offset string.
   VELOX_ASSERT_THROW(
       (evaluateOnce<Timestamp, std::string>(
@@ -788,23 +791,27 @@ TEST_F(CastExprTest, timestampToString) {
           std::nullopt,
       });
 
-  // Ensure external/date throws since it doesn't know how to convert large
-  // timestamps.
-  auto mustThrow = [&]() {
-    return testCast<Timestamp, std::string>(
-        "string", {Timestamp(253405036800, 0)}, {"10000-02-01 08:00:00.000"});
-  };
-  VELOX_ASSERT_THROW(
-      mustThrow(), "Unable to convert timezone 'America/Los_Angeles' past");
-
-  // try_cast should also throw since it's runtime error.
-  auto tryCastMustThrow = [&]() {
-    return testTryCast<Timestamp, std::string>(
-        "string", {Timestamp(253405036800, 0)}, {"10000-02-01 08:00:00.000"});
-  };
-  VELOX_ASSERT_THROW(
-      tryCastMustThrow(),
-      "Unable to convert timezone 'America/Los_Angeles' past");
+  // TODO: Address in https://github.com/facebookincubator/velox/pull/12471
+  //   // Ensure external/date throws since it doesn't know how to convert large
+  //   // timestamps.
+  //   auto mustThrow = [&]() {
+  //     return testCast<Timestamp, std::string>(
+  //         "string", {Timestamp(253405036800, 0)}, {"10000-02-01
+  //         08:00:00.000"});
+  //   };
+  //   VELOX_ASSERT_THROW(
+  //       mustThrow(), "Unable to convert timezone 'America/Los_Angeles'
+  //       past");
+
+  //   // try_cast should also throw since it's runtime error.
+  //   auto tryCastMustThrow = [&]() {
+  //     return testTryCast<Timestamp, std::string>(
+  //         "string", {Timestamp(253405036800, 0)}, {"10000-02-01
+  //         08:00:00.000"});
+  //   };
+  //   VELOX_ASSERT_THROW(
+  //       tryCastMustThrow(),
+  //       "Unable to convert timezone 'America/Los_Angeles' past");
 }
 
 TEST_F(CastExprTest, dateToTimestamp) {
@@ -859,23 +866,25 @@ TEST_F(CastExprTest, timestampToDate) {
       TIMESTAMP(),
       DATE());
 
-  // Ensure external/date throws since it doesn't know how to convert large
-  // timestamps.
-  auto mustThrow = [&]() {
-    return testCast<Timestamp, int32_t>(
-        "date", {Timestamp(253405036800, 0)}, {0});
-  };
-  VELOX_ASSERT_THROW(
-      mustThrow(), "Unable to convert timezone 'America/Los_Angeles' past");
-
-  // try_cast should also throw since it's runtime error.
-  auto tryCastMustThrow = [&]() {
-    return testTryCast<Timestamp, int32_t>(
-        "date", {Timestamp(253405036800, 0)}, {0});
-  };
-  VELOX_ASSERT_THROW(
-      tryCastMustThrow(),
-      "Unable to convert timezone 'America/Los_Angeles' past");
+  // TODO: Address in https://github.com/facebookincubator/velox/pull/12471
+  //   // Ensure external/date throws since it doesn't know how to convert large
+  //   // timestamps.
+  //   auto mustThrow = [&]() {
+  //     return testCast<Timestamp, int32_t>(
+  //         "date", {Timestamp(253405036800, 0)}, {0});
+  //   };
+  //   VELOX_ASSERT_THROW(
+  //       mustThrow(), "Unable to convert timezone 'America/Los_Angeles'
+  //       past");
+
+  //   // try_cast should also throw since it's runtime error.
+  //   auto tryCastMustThrow = [&]() {
+  //     return testTryCast<Timestamp, int32_t>(
+  //         "date", {Timestamp(253405036800, 0)}, {0});
+  //   };
+  //   VELOX_ASSERT_THROW(
+  //       tryCastMustThrow(),
+  //       "Unable to convert timezone 'America/Los_Angeles' past");
 }
 
 TEST_F(CastExprTest, timestampInvalid) {

velox/external/date/CMakeLists.txt

@@ -10,7 +10,5 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-
-velox_add_library(velox_external_date tz.cpp)
-velox_include_directories(velox_external_date SYSTEM PUBLIC velox/external)
-velox_compile_definitions(velox_external_date PRIVATE USE_OS_TZDB)
+velox_add_library(velox_external_date INTERFACE)
+velox_include_directories(velox_external_date INTERFACE velox/external/date)