Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow pl.Duration in write_csv #19971

Open
gab23r opened this issue Nov 25, 2024 · 3 comments
Open

Allow pl.Duration in write_csv #19971

gab23r opened this issue Nov 25, 2024 · 3 comments
Labels
enhancement New feature or an improvement of an existing feature

Comments

@gab23r
Copy link
Contributor

gab23r commented Nov 25, 2024

Description

Today it is not possible to write csv files with pl.Duration

pl.Series([1, 2, 3]).cast(pl.Duration('us')).to_frame().write_csv("a.csv")

# ComputeError: datatype duration[μs] cannot be written to CSV
# Consider using JSON or a binary format.
@gab23r gab23r added the enhancement New feature or an improvement of an existing feature label Nov 25, 2024
@gab23r
Copy link
Contributor Author

gab23r commented Nov 25, 2024

@alexander-beedie, pinging you as maybe It was on your roadmap after the great work you've done here #19697

@Julian-J-S
Copy link
Contributor

Julian-J-S commented Nov 25, 2024

Imo this is difficult.
What is your expected result?

date/datetime and duration are physically stored as int.
date/datetime have very common and standardized "text representation" which is YYYY-MM-DD

duration does not really have it (yes, there is an iso format but you almost never see it and almost no tools have support for parsing this).
Most of this times you want to cast the Duration to your desired int value (total_seconds, total_minutes, total_hours, total_days) or keep the underlying physical representation.

@alexander-beedie
Copy link
Collaborator

alexander-beedie commented Nov 25, 2024

The best thing might be to improve the error message, so the caller sees that there are now some more specific options (such as doing a cs.duration().to_string() before export to return ISO format) 🤔

I haven't looked at much of the wider tooling around this, so not sure what the degree of support is for parsing ISO durations is like (we don't yet, for example). I don't think the degree of support is near-zero though, as I've definitely seen a few libraries that handle it quite comfortably.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature
Projects
None yet
Development

No branches or pull requests

3 participants