Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve offchain #1860

Merged
merged 5 commits into from
Oct 4, 2024
Merged

Improve offchain #1860

merged 5 commits into from
Oct 4, 2024

Conversation

kderme
Copy link
Contributor

@kderme kderme commented Sep 17, 2024

Description

Add your description here, if it fixes a particular issue please provide a link to the issue.

Checklist

  • Commit sequence broadly makes sense
  • Commits have useful messages
  • New tests are added if needed and existing tests are updated
  • Any changes are noted in the changelog
  • Code is formatted with fourmolu on version 0.10.1.0 (which can be run with scripts/fourmolize.sh)
  • Self-reviewed the diff

Migrations

  • The pr causes a breaking change of type a,b or c
  • If there is a breaking change, the pr includes a database migration and/or a fix process for old values, so that upgrade is possible
  • Resyncing and running the migrations provided will result in the same database semantically

If there is a breaking change, especially a big one, please add a justification here. Please elaborate
more what the migration achieves, what it cannot achieve or why a migration is not possible.

@kderme kderme requested a review from a team as a code owner September 17, 2024 13:31
@kderme
Copy link
Contributor Author

kderme commented Sep 17, 2024

Needs testing

httpGetOffChainVoteData gateways vurl metaHash anchorType = do
case useIpfsGatewayMaybe vurl gateways of
Nothing -> httpGetOffChainVoteDataSingle vurl metaHash anchorType
Just [] -> left $ OCFErrNoIpfsGateway (OffChainVoteUrl vurl)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just make this a NonEmpty?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be empty

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be reading this wrong, but it looks to me like having empty gateways is an error here. If that's correct, can we make gateways a nonempty? That would push the check up to a config check and make everywhere else a safer call

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an error that doesn't bubble up. It ends up in the off_chain_vote_fetch_error table.
The user has to specify a list of gateaways, if he wants to support ipfs links

@@ -103,6 +104,7 @@ syncNodeConfig loggingCfg =
<*> triggerHardFork
<*> triggerHardFork
<*> syncInsertOptions
<*> pure []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would probably be useful to generate ipfs_gateway values

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any suggestions for a generator to be used here? Not bvery familiar with these tests

Cmdv
Cmdv previously approved these changes Sep 26, 2024
Copy link
Contributor

@Cmdv Cmdv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM just a minor hlint related comment

@@ -112,6 +112,7 @@ classifyFetchError tf fe =
OCFErrIOException {} -> tf {tfIOException = tfIOException tf + 1}
OCFErrTimeout {} -> tf {tfTimeout = tfTimeout tf + 1}
OCFErrConnectionFailure {} -> tf {tfConnectionFailure = tfConnectionFailure tf + 1}
_ -> tf
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hlint always complains about _ -> that's the only reason I've been doing _otherwise 🤷

@kderme
Copy link
Contributor Author

kderme commented Oct 1, 2024

After testing this on mainnet

image = Just (Image {content = "PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4KPHN2ZyB3aWR0aD0iMTc4MCIgaGVpZ2h0PSIxNzgwIiB2aWV3Qm94PSItODkwIC04OTAgMTc4MCAxNzgwIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgo8c3R5bGU+LmJ7ZmlsbDojMTYyNjMzfS5ze3N0cm9rZTojNDQ2OTg3fS5je2ZpbGw6IzQ0Njk4N30uaHtzdHJva2U6I2JjZGFmOH1wYXRoe2ZpbGw6bm9uZTtzdHJva2Utd2lkdGg6MjQ7c3Ryb2tlLWxpbmVjYXA6cm91bmQ7c3Ryb2tlLWxpbmVqb2luOnJvdW5kfTwvc3R5bGU+CjxyZWN0IGNsYXNzPSJiIiB4PSItODkwIiB5PSItODkwIiB3aWR0aD0iMTc4MCIgaGVpZ2h0PSIxNzgwIi8+CjxwYXRoIGNsYXNzPSJzIiBkPSJtLTcwMCAwYTcwMCA3MDAgMCAwIDEgMTQwMCAwIDY1MCA2NTAgMCAwIDEtMTMwMCAwIDYwMCA2MDAgMCAwIDEgMTIwMCAwIDU1MCA1NTAgMCAwIDEtMTEwMCAwIDUwMCA1MDAgMCAwIDEgMWUzIDAgNDUwIDQ1MCAwIDAgMS05MDAgMCA0MDAgNDAwIDAgMCAxIDgwMCAwIDM1MCAzNTAgMCAwIDEtNzAwIDAgMzAwIDMwMCAwIDAgMSA2MDAgMCAyNTAgMjUwIDAgMCAxLTUwMCAwIDIwMCAyMDAgMCAwIDEgNDAwIDAgMTUwIDE1MCAwIDAgMS0zMDAgMCAxMDAgMTAwIDAgMCAxIDIwMCAwIDUwIDUwIDAgMCAxLTEwMCAwIi8+CjxjaXJjbGUgY2xhc3M9ImMiIHI9IjI0Ii8+CjxwYXRoIGNsYXNzPSJoIiBkPSJtMC03MDAtMzA0IDEzMzEgODUxLTEwNjctMTIyOSA1OTJoMTM2NGwtMTIyOS01OTIgODUxIDEwNjd6Ii8+Cjwvc3ZnPgo=", msha256 = Nothing}
 1641 |               38 | Error Offchain Voting Anchor: HTTP Response error from https://blinklabs.io/metadata.jsonld: expected JSON, but got : "application/ld+json"                                                                                             | 2024-09-09 00:42:58.410516 |           7
 2075 |               38 | Hash mismatch when fetching metadata from https://blinklabs.io/metadata.jsonld. Expected "6edc9815082fff7601794c12813586dd40167ce8a90bcfb7ae960258757d5818" but got "3fc767b5bcdc1b429553ae6841bb1ee67ae486c91bdbb25b6229d5372dc0e8e8". | 2024-09-30 12:25:51.568471 |           8
 3072 |               90 | Error Offchain Voting Anchor: List of errors for each ipfs gateway: Hash mismatch when fetching metadata from https://ipfs.io/ipfs/QmWjcHsrq9kKHZZ7aPPFjqN6wLuxH9d8bcqssmrE7H4cvb. Expected "2f98f57c4149fdfed2b73cbd821226fe417ef5ed49d8f836a37b31edf14dea47" but got "a5f2785526d91ec3f7cdd642b679ee668f4483d0f1af045b3bf2750f1fcc3dcb". | 2024-10-02 00:09:30.802303 |           0

@kderme kderme force-pushed the kderme/improve-offchain branch 2 times, most recently from 774396d to 27f8621 Compare October 4, 2024 09:33
@kderme kderme force-pushed the kderme/improve-offchain branch from 27f8621 to 19cc86e Compare October 4, 2024 09:43
@kderme kderme merged commit e8a00e5 into master Oct 4, 2024
13 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants