Skip to content

Latest commit

 

History

History
23 lines (16 loc) · 494 Bytes

README.md

File metadata and controls

23 lines (16 loc) · 494 Bytes

Spark_Multi_Cloud_Storage_Utils

Spark Read data from Multi Cloud Utils(GCP,Azure and AWS)

  1. In this repo Read data from Spark with AWS S3,GCS Storage and Azure Data Lake FS
  2. In this repo Write data into Spark with AWS S3,GCS Storage and Azure Data Lake FS

Pre-Requests

  1. AWS Account
  2. GCP Account
  3. Azure Account

Input Source

  1. AWS S3
  2. Google GCS Storage
  3. Azure Storage (Data lake Gen2)

Output Source

  1. AWS S3
  2. Google GCS Storage
  3. Azure Storage (Data lake Gen2)