Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RD-14664: SQL parser slow against large queries #499

Merged
merged 2 commits into from
Sep 2, 2024

Conversation

bgaidioz
Copy link
Collaborator

@bgaidioz bgaidioz commented Aug 30, 2024

The slow part was identified as parsing lists (the comma separated ones at least), which would lead to a nested cascade of lists of two elements: (1, (2, (3, (4, ...)))) which eventually makes the parser very slow, instead of a single list of items 1,2,3,4,....

I attach two queries, the one used as a template we advertise in the web app, because it's also slow to parse, although it takes less than a second, and the large one that takes minutes. With the patch, both parse much faster (far less than a second).

@bgaidioz bgaidioz marked this pull request as ready for review August 30, 2024 15:07
@bgaidioz bgaidioz requested a review from alexzerntev September 2, 2024 09:24
Copy link
Contributor

@alexzerntev alexzerntev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only case that this won't work is ,2,3,4, this shouldn't parse, but will. But we have a safety net, the pssql call which will output an error. Will fix the parser in a next PR

@bgaidioz bgaidioz merged commit 17fddf0 into main Sep 2, 2024
7 checks passed
@bgaidioz bgaidioz deleted the RD-14664-sql-parser-slow-when-parsing-large-queries branch September 2, 2024 09:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants