Introduction: In this blog we will discuss Spark ETL (Extract, transform and load) or ELT (Extract, load and transform). In Spark to connect different data sources we need to install libraries, we will discuss how to install all the required libraries and how to connect with different data sources and extract, transform and load data.... Continue Reading →
Data Engineering Problem 8 (Top distance travelled by rider)
Please find earlier blogs to have understanding of our Data Engineering Learning plan and system setup for Data Engineering. Today we are solving and learning one more Data Engineering problem and learning new concepts. For earlier problem solution and key learning points follow below. https://developershome.blog/category/data-engineering/problem-solving/ Problem Statement Find the top 10 users that have traveled... Continue Reading →
Data Engineering Problem 7 (Ebay Returning active users)
Please find earlier blogs to have understanding of our Data Engineering Learning plan and system setup for Data Engineering. Today we are solving and learning one more Data Engineering problem and learning new concepts. For earlier problem solution and key learning points follow below. https://developershome.blog/category/data-engineering/problem-solving/ Problem Statement Write a query that'll identify returning active users.... Continue Reading →