Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ddof parameter on pl.corr is nonsensical #19420

Closed
orlp opened this issue Oct 24, 2024 · 6 comments · Fixed by #20197
Closed

ddof parameter on pl.corr is nonsensical #19420

orlp opened this issue Oct 24, 2024 · 6 comments · Fixed by #20197
Labels
A-api Area: changes to the public API accepted Ready for implementation enhancement New feature or an improvement of an existing feature good first issue Good for newcomers P-low Priority: low
Milestone

Comments

@orlp
Copy link
Collaborator

orlp commented Oct 24, 2024

I only realized this after implementing it recently, but the ddof parameter is not used for pl.corr at all. It completely cancels out of the formula. To confirm this in np.corrcoef you'll see:

ddof: _NoValue, optional
    Has no effect, do not use.

We should just remove the parameter in the Rust code and on the Python side deprecate the parameter, removing it in 2.0.

@orlp orlp added good first issue Good for newcomers enhancement New feature or an improvement of an existing feature accepted Ready for implementation P-low Priority: low labels Oct 24, 2024
@github-project-automation github-project-automation bot moved this to Ready in Backlog Oct 24, 2024
@stinodego stinodego added the A-api Area: changes to the public API label Oct 24, 2024
@stinodego stinodego added this to the 2.0.0 milestone Oct 24, 2024
@orlp
Copy link
Collaborator Author

orlp commented Oct 25, 2024

The same holds for pl.rolling_corr.

@zachlefevre
Copy link

I am interested in doing this work as my first contribution to Polars.

@zachlefevre
Copy link

I'm still working on getting my local dev machine setup -- made more difficult since I use Guix.

@hshreekar
Copy link

Hi @orlp, so on the rust side we would need to remove the finalize call or do we need to change the implementation and calculate it using some other formula ?
this is the function pearson_corr in polars/crates/polars-ops/src/chunked_array/cov.rs. I assume this is the implementation and i would need to modify it for every usage of this function and also for Spearman, and the other method in the Enum ?
Screenshot 2024-11-24 at 2 50 31 PM

@orlp
Copy link
Collaborator Author

orlp commented Nov 25, 2024

@hshreekar Don't remove the finalize call, remove the ddof parameter from it. In general from both Spearman and Pearson, not for covariance.

@flowlight0
Copy link
Contributor

Hello! Since other potential contributors looks inactive, let me make my first contributions to Polars :) (#20197)

@github-project-automation github-project-automation bot moved this from Ready to Done in Backlog Dec 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-api Area: changes to the public API accepted Ready for implementation enhancement New feature or an improvement of an existing feature good first issue Good for newcomers P-low Priority: low
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

5 participants