Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement parallel API requests for Socrata #703

Open
wrridgeway opened this issue Jan 9, 2025 · 1 comment
Open

Implement parallel API requests for Socrata #703

wrridgeway opened this issue Jan 9, 2025 · 1 comment

Comments

@wrridgeway
Copy link
Member

Let's see if we can use something like this to speed up the script.

@wrridgeway
Copy link
Member Author

wrridgeway commented Feb 25, 2025

I spent a good chunk of the day trying to get this to work. requests-futures is simple to implement, but socrata's API throws a lot of errors while using it. resolving those errors slows things down to the point where there's really no performance gain:

{'content': {'code': 'conflict_operation',
             'error': True,
             'message': 'Another process is writing to the dataset; please try '
                        'again later'}}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

1 participant