Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-33] Add macro/run-operation for uploading files from local filesystem to S3 #64

Closed
NiallRees opened this issue Jan 11, 2022 · 3 comments
Labels
good_first_issue Good for newcomers Stale type:enhancement New feature or request

Comments

@NiallRees
Copy link
Contributor

NiallRees commented Jan 11, 2022

Describe the feature

Prompted by brooklyn-data/dbt_artifacts#60.

Access to a macro such as:

{% do adapter.upload_file(file_path, destination) %}

would enable dbt_artifacts to become compatible with Redshift. The user would need to create their own S3 bucket, and grant any policies needed.

The adapter already has boto3 as a dependency, so no additional packages should be needed.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Additional context

Please include any other relevant context here.

Who will this benefit?

What kind of use case will this feature be useful for? Please be specific and provide examples, this will help us prioritize properly.

Are you interested in contributing this feature?

Let us know if you want to write some code, and how we can help.

@NiallRees NiallRees added type:enhancement New feature or request triage:product labels Jan 11, 2022
@github-actions github-actions bot changed the title Add macro/run-operation for uploading files from local filesystem to S3 [CT-33] Add macro/run-operation for uploading files from local filesystem to S3 Jan 11, 2022
@McKnight-42
Copy link
Contributor

@NiallRees Thank you for also trying to implement this functionality in Redshift love the idea of having uniformity for how we extend our adapters as much as possible, I will open this up as a good first issue to work on as we did for the big-query

@McKnight-42 McKnight-42 added good_first_issue Good for newcomers and removed triage:product labels Jan 25, 2022
@McKnight-42
Copy link
Contributor

boto3 upload_file would need to add a new method in impl.py to be able to reference within the adapter.sql suggested macro.

@github-actions
Copy link
Contributor

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good_first_issue Good for newcomers Stale type:enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants