Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 [fs-bq-import-collection] Multi-Threaded import Script does not Support Cross-Project loading #2298

Open
justin-plancraft opened this issue Feb 20, 2025 · 2 comments · May be fixed by #2299
Open

🐛 [fs-bq-import-collection] Multi-Threaded import Script does not Support Cross-Project loading #2298

justin-plancraft opened this issue Feb 20, 2025 · 2 comments · May be fixed by #2299
Labels
type: bug Something isn't working

Comments

@justin-plancraft
Copy link

  • Extension name: fs-bq-import-collection
  • Extension version: 0.1.23

Steps to reproduce:

When trying to Backfill via the Import Script in a Cross-Project setup (reading from project-dev-01 and writing to bigquery in project-prod-01 for example) - and running the import script multi-threaded, the worker.ts does not pass the neccessary bqProjectId to the BigQuery Writer, therefore always failing because the project-dev-01 does not contain the BigQuery Dataset and Table (and should not contain it).

Expected result

Should work

Actual result

Fails because view not found
@justin-plancraft justin-plancraft added the type: bug Something isn't working label Feb 20, 2025
justin-plancraft added a commit to justin-plancraft/extensions that referenced this issue Feb 20, 2025
@justin-plancraft
fix: Pass through BigQuery Config in multithreaded worker (closes fir…
f4dcff1 
…ebase#2298)
@justin-plancraft justin-plancraft linked a pull request Feb 20, 2025 that will close this issue
Open
@CorieW
Copy link
Contributor

CorieW commented Feb 20, 2025

Hi @justin-plancraft

Thanks for reporting this issue! We’ve received it and are reviewing it. We’ll provide updates as soon as possible.

@justin-plancraft
Copy link
Author

Awesome, thanks so much for the prompt response :) Hope my fix is helpful, I got it working locally by using these changes directly and editing inline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: Pass through BigQuery Config in multithreaded worker (closes #2298) justin-plancraft/extensions
2 participants
@CorieW @justin-plancraft