Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DB can get missing entries when scrape gets interrupted (e.g. network error, http response != 200 etc) #11

Open
Tomen opened this issue Mar 22, 2022 · 0 comments
Open

DB can get missing entries when scrape gets interrupted (e.g. network error, http response != 200 etc) #11

Tomen opened this issue Mar 22, 2022 · 0 comments
Assignees
@notstano
Labels
bug Something isn't working
Milestone
1.1 Make Subscrap...

Comments

@Tomen
Copy link
Contributor

Tomen commented Mar 22, 2022

Error handling is not very sophisticated, so if the scrape is interrupted by an uncaught exception, the delta might be incomplete and subsequent runs might miss some data. To remedy the issue, the delta must be deleted and the scraper run again.
@Tomen Tomen added the bug Something isn't working label Mar 22, 2022
@Tomen Tomen moved this to Todo in Subscrape Grant 1 May 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@notstano notstano

Labels
bug Something isn't working
Projects
Subscrape Grant 1
Status: Todo
Milestone
1.1 Make Subscrape onboarding-friendly
Development

No branches or pull requests
2 participants
@Tomen @notstano