Apache Spark
Course Overview
Apache Spark is an open-source, distributed computing system that is designed for fast, flexible, and expressive data processing. Spark is a general-purpose data processing engine that can handle a wide variety of data types and processing workloads, including batch processing, stream processing, machine learning, and graph processing. Training in Apache Spark typically covers the fundamental concepts and architecture of Spark, as well as how to develop and deploy Spark-based applications. It may also cover topics such as Spark streaming, Spark SQL, and Spark machine learning.
At the end of the training, participants will be able to:
- 1.Understand the architecture of Spark and explain its business use cases
- 2.Distribute, store, and process data using RDDs in a Hadoop cluster
- 3.Use Spark SQL for querying DBs
- 4.Write, configure, and deploy Spark applications on a cluster
- 5.Use the Spark shell for interactive data analysis
- 6.Process and query structured data using Spark SQL
Pre-requisite
Knowledge of Apache Hadoop ecosystem, SQL, Linux CLI and Scala is required.
Duarion
2 days
Course Outline
- What is Apache Spark?
- Starting the Spark Shell
- Using the Spark Shell
- Getting Started with Datasets and DataFrames
- DataFrame Operations
- Working with DataFrames and Schemas
- Creating DataFrames from Data Sources
- Saving DataFrames to Data Sources
- DataFrame Schemas
- Eager and Lazy Execution
- Analyzing Data with DataFrame Queries
- Querying DataFrames Using
- Column Expressions
- Grouping and Aggregation Queries
- Joining DataFrames
- RDD Overview
- RDD Data Sources
- Creating and Saving RDDs
- RDD Operations
- Writing and Passing
- Transformation Functions
- Transformation Execution
- Converting Between RDDs and DataFrames
- Querying Tables in Spark Using SQL
- Querying Files and Views
- The Catalog API
- Comparing Spark SQL, Apache Impala, and Apache Hive-on-Spark
- Apache Spark Applications
- Writing a Spark Application
- Building and Running an Application
- Application Deployment Mode
- The Spark Application Web UI
- Configuring Application Properties
- Review: Apache Spark on a Cluster
- RDD Partitions
- Example: Partitioning in Queries
- Stages and Tasks
- Job Execution Planning
- Example: Catalyst Execution Plan
- Example: RDD Execution Plan
- Data Processing
- Common Apache Spark Use Cases
- Iterative Algorithms in Apache Spark
- Machine Learning
- Example: k-means
Reviews
"The corporate training provided by Greater Insights was truly exceptional. The trainers were highly knowledgeable and engaging, making the sessions both informative and enjoyable. I gained valuable insights and practical skills that I immediately applied in my work. I highly recommend Greater Insights for their professionalism and expertise."
Rahul NiraniaSamsung 
"As a participant of the corporate training program organized by Greater Insights, I was thoroughly impressed with the level of customization they offered. They took the time to understand our specific needs and tailored the training accordingly. The trainers were fantastic, and the interactive sessions fostered an environment of active learning. I am grateful for the valuable knowledge and skills I acquired."
Bhawna TiwariSamsung 
"I cannot express enough gratitude to Greater Insights for the outstanding corporate training they provided. The trainers were not only experts in their respective fields but also incredible communicators. They created a dynamic and collaborative learning environment, which allowed us to learn from one another. The training surpassed my expectations, and I would eagerly participate in any future programs they offer."
Ashutosh SinghSamsung 
"Attending the corporate training sessions organized by Greater Insights was a game-changer for me. The trainers were not only experienced professionals but also inspiring mentors. They equipped me with practical tools and strategies that have significantly enhanced my productivity and efficiency at work. I wholeheartedly recommend Greater Insights to anyone looking to excel in their professional endeavors."
Somashekar MuniyappaInfogain 
"The corporate training program offered by Greater Insights was an enlightening experience. The trainers had a deep understanding of the subject matter and were able to break down complex concepts into easily digestible information. The interactive activities and case studies made the training engaging and relevant to our day-to-day work challenges. This training has undoubtedly boosted my confidence and competence."
Amresh DiwanInfogain 
"I had the privilege of participating in a corporate training program organized by Greater Insights, and I must say it was a transformative experience. The trainers' expertise and passion for their subjects were evident in every session. The training materials provided were comprehensive and well-structured, enabling us to grasp the content effectively. I am grateful for the valuable skills I acquired, which have greatly contributed to my professional growth."
Namratha BabuTorry Harris 
"The corporate training sessions conducted by Greater Insights were simply outstanding. The trainers went above and beyond to ensure that we understood the concepts thoroughly. The interactive nature of the sessions encouraged active participation and enhanced our learning experience. I am truly grateful for the practical strategies and techniques I learned, which have had a positive impact on my work performance."
Rahul BhashyamTorry Harris 
"Participating in the corporate training program organized by Greater Insights was an enlightening experience. The trainers were not only knowledgeable but also skilled at delivering the content in an engaging manner. The training materials were well-designed, and the real-life examples provided valuable insights. I am confident that the knowledge and skills I gained will significantly contribute to my professional growth."
Deepti JainTorry Harris 
"I can confidently say that the corporate training provided by Greater Insights was top-notch. The trainers were highly experienced and had a knack for simplifying complex concepts. The training sessions were interactive and encouraged open discussions, which fostered a collaborative learning environment. The practical skills I acquired during the training have proven to be invaluable in my day-to-day work."
HARSHAL TRIVEDIL&T TS 
"Greater Insights exceeded my expectations with their corporate training program. The trainers were not only experts in their fields but also exceptional communicators. They effortlessly connected with the participants and ensured everyone's active involvement. The training content was comprehensive and provided a solid foundation for professional growth. I am grateful for the opportunity to learn from such seasoned professionals."
Namrata PillayL&T TS
"The corporate training facilitated by Greater Insights was exceptional in every aspect. The trainers were engaging and created a positive and inclusive learning environment. The training materials were well-structured and easy to follow, making the learning process enjoyable. The practical exercises and case studies allowed me to apply the knowledge immediately, resulting in improved job performance. I highly recommend Greater Insights for their commitment to excellence."
Sandhya M SanuHoneywell
"I recently attended a corporate training program organized by Greater Insights, and I am incredibly impressed. The trainers were not only knowledgeable but also highly skilled at delivering the content in a relatable manner. The training sessions were interactive and encouraged open dialogue, which made the learning experience engaging and dynamic. The practical skills I gained have already made a significant impact on my professional growth."
Neha SahayHoneywell
"I had the privilege of participating in a corporate training program conducted by Greater Insights, and it was truly transformative. The trainers were passionate and dedicated professionals who took the time to address our individual needs. The training sessions were interactive and encouraged active participation, allowing us to learn from one another's experiences. I am grateful for the knowledge and skills I acquired, which have undoubtedly propelled my career forward."
Abishek KalliparambilAllianz 
"The corporate training program offered by Greater Insights was a remarkable experience. The trainers had a wealth of knowledge and were adept at imparting it effectively. The training materials were comprehensive and provided valuable resources for further exploration. The interactive activities and practical exercises helped me internalize the concepts and apply them in real-world scenarios. This training has been instrumental in my professional development."
Kishor KumarITC Infotech 
"I cannot thank Greater Insights enough for the incredible corporate training they provided. The trainers were not only subject matter experts but also skilled at creating a positive and inclusive learning environment. The training sessions were well-structured, and the trainers ensured that we understood the content thoroughly. I am grateful for the valuable insights and skills I gained, which have already made a noticeable difference in my professional life."
Anitha PrashanthIndegene 
