Can no longer pass pd.Timestamp objects into places/expressions that take datetime64

[agossard]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

This code worked in 1.5, but crashes in 1.6

pl.datetime_range(pd.to_datetime('12/31/2020'),
                              pd.to_datetime('12/31/2022'),
                              interval="1d",
                              eager=True)

TypeError: cannot create expression literal for value of type Timestamp: Timestamp('2020-12-31 00:00:00+0000', tz='UTC')
Hint: Pass `allow_object=True` to accept any value and create a literal of type Object.

This fixes it:

pl.datetime_range(pd.to_datetime('12/31/2020').to_datetime64(),
                  pd.to_datetime('12/31/2022').to_datetime64(),
                  interval="1d",
                  eager=True)

Understand the documentation does not explicitly say that you can pass it a pandas Timestamp, but once again, seemed a reasonable user assumption and used to work.

Also, this crashes now with similar error.

df = pl.DataFrame({"dates": pd.date_range(pd.to_datetime('12/31/2020'), pd.to_datetime('12/31/2021'), freq='B')})
df.filter(pl.col('dt') < pd.to_datetime('12/31/2020'))

Log output

No response

Issue description

Cannot pass pd.Timestamp to functions that expect datetime.

Expected behavior

Would be nice to automatically cast the Timestamp to datetime64 on entry to polars

Installed versions

didn't get a chance to do this... it's on my work computer

[MarcoGorelli]

thanks @agossard for the report, much appreciated 🙏

This regressed in #16018 (@mcrumiller fancy taking a look? maybe something similar to #18346 can solve this?)

[mcrumiller]

I'll definitely take a look.

[mcrumiller]

Yep, also for pd.Timedelta/np.datetime64/np.timedelta64/. Working on it.

[mcrumiller]

@MarcoGorelli, should we allow pl.lit(datetime(...), pl.Time)? i.e. extract time of day from a datetime object?

[mcrumiller]

I suppose that should be an enhancement, I'll leave that out of this.

[btschroer]

Can confirm, the following used to work in 1.5:

import polars as pl
import pandas as pd

t0 = pd.Timestamp("2001-01-03")
frame = pl.from_dict({
    "t": ["2001-01-01", "2001-01-02", "2001-01-03", "2001-01-04"]
}).with_columns(
    pl.col("t").str.to_datetime()
)
frame.filter(pl.col("t") >= t0)

[MarcoGorelli]

thanks @btschroer !

I think pd.Timestamp should be supported in so far as it's a subclass of datetime.datetime

np.datetime64 I'm less sure about tbh, not sure it's worth the added complexity