Spark ETL Chapter 4 with Cloud data lakes (AWS S3 bucket)

Previous blog/Context: In an earlier blog, we discussed Spark ETL with Cloud data lakes (Azure blob and Azure Data Lake services). Please find below blog post for more details. https://developershome.blog/2023/03/08/spark-etl-chapter-3-with-cloud-data-lakes-azure-blob-azure-adls/ Introduction: In this blog, we will discuss Spark ETL with Cloud data lakes and we will be doing Spark ETL with AWS S3 bucket. We... Continue Reading →

March 12, 2023 1

Data Engineering Problem 8 (Top distance travelled by rider)

Please find earlier blogs to have understanding of our Data Engineering Learning plan and system setup for Data Engineering. Today we are solving and learning one more Data Engineering problem and learning new concepts. For earlier problem solution and key learning points follow below. https://developershome.blog/category/data-engineering/problem-solving/ Problem Statement Find the top 10 users that have traveled... Continue Reading →

February 23, 2023 0

Data Engineering Problem 5 (City names starting with vowels)

Please find earlier blogs to have understanding of our Data Engineering Learning plan and system setup for Data Engineering. Today we are solving and learning one more Data Engineering problem and learning new concepts. For earlier problem solution and key learning points follow below. https://developershome.blog/category/data-engineering/problem-solving/ Problem Statement Query the list of CITY names starting with... Continue Reading →

February 14, 2023 0

Up ↑