Apache PIG & HIVE
Course Overview
Master core concepts on hadoop distributed file system and Understand apache pig and advanced apache hive programming concepts as you learn with our certified experts. Learn how to use Hcatalog, joining datasets in apache hive and HDFS Commands.
Gain practical experience to import and export RDBMS data into HDFS, analyze clickstream data. Data using quantiles. With our cloudlabs get hands-on experience to run a YARN application, apache hive programming, analyzing big data with apache hive, join datasets with apache pig and starting an HDP cluster.
At the end of the training, participants will be able to:
- Explain Hadoop and the Hadoop Distributed File System (HDFS)
- Interpret Common HDFS Commands Types
- Export Table
- Distinguish between Relational Databases and Hadoop
- Explain Purpose of NameNodes, DataNode, MapReduce and Reduce Phases
- Differentiate Pig Latin Relation Names and Field Names
- Explain programming concepts using PIG and HIVE.
- Perform Inner, Outer and Replicated Join
- Demonstrate the Use of HCatLoader and HCatStorer with Apache Pig
- Explain Lifecycle of YARN Applications
- Common use cases of Spark
- Load Data and Perform a Word Count
- Perform SQL Queries
- Perform DataFrame Operations
- Submit an Apache Oozie Workflow
Pre-requisite
- Should be familiar with programming principles and have experience in software development.
- SQL knowledge is also helpful.
- No prior Hadoop knowledge is required
Duration
2 days
Course Outline
- List the Three “V”s of Big Data
- List the Six Key Hadoop Data Types
- Describe Hadoop, YARN and Use Cases for Hadoop
- Describe Hadoop Ecosystem Tools and Frameworks
- Describe the Differences Between Relational Databases and Hadoop
- Describe What is New in Hadoop 2.x
- Describe the Hadoop Distributed File System (HDFS)
- Describe the Differences Between HDFS and an RDBMS
- Describe the Purpose of NameNodes and DataNodes
- List Common HDFS Commands
- Describe HDFS File Permissions
- List Options for Data Input
- Describe WebHDFS
- Describe the Purpose of Sqoop and Flume
- Describe How to Export to a Table
- Describe the Purpose of MapReduce
- Define Key/Value Pairs in MapReduce
- Describe the Map and Reduce Phases
- Describe Hadoop Streaming
- Starting an HDP Cluster
- Demonstration: Understanding Block Storage (Lab)
- Using HDFS Commands (Lab)
- Importing RDBMS Data into HDFS (Lab)
- Exporting HDFS Data to an RDBMS (Lab)
- Importing Log Data into HDFS Using Flume (Lab)
- Demonstration: Understanding MapReduce (Lab)
- Running a MapReduce Job (Lab)
- Describe the Purpose of Apache Pig
- Describe the Purpose of Pig Latin
- Demonstrate the Use of the Grunt Shell
- List Pig Latin Relation Names and Field Names
- List Pig Data Types
- Define a Schema
- Describe the Purpose of the GROUP Operator
- Describe Common Pig Operators ( ORDER BY, CASE, DISTINCT, PARALLEL, FLATTEN, FOREACH)
- Perform an Inner, Outer and Replicated Join
- Describe the Purpose of the DataFu Library
- Demonstration: Understanding Apache Pig (Lab)
- Getting Starting with Apache Pig (Lab)
- Exploring Data with Apache Pig (Lab)
- Splitting a Dataset (Lab)
- Joining Datasets with Apache Pig (Lab)
- Preparing Data for Apache Hive (Lab)
- Demonstration: Computing Page Rank (Lab)
- Analyzing Clickstream Data (Lab)
- Analyzing Stock Market Data Using Quantiles (Lab)
- Describe the Purpose of Apache Hive
- Describe the Differences Between Apache Hive and SQL
- Describe the Apache Hive Architecture
- Demonstrate How to Submit Hive Queries
- Describe How to Define Tables
- Describe How to Load Date Into Hive
- Define Hive Partitions, Buckets and Skew
- Describe How to Sort Data
- List Hive Join Strategies
- Describe the Purpose of HCatalog
- Describe the HCatalog Ecosystem
- Define a New Schema
- Demonstrate the Use of HCatLoader and HCatStorer with Apache Pig
- Perform a Multi-table/File Insert
- Describe the Purpose of Views
- Describe the Purpose of the OVER Clause
- Describe the Purpose of Windows
- List Hive Analytics Functions
- List Hive File Formats
- Describe the Purpose of Hive SerDe
- Understanding Hive Tables (Lab)
- Understanding Partition and Skew (Lab)
- Analyzing Big Data with Apache Hive (Lab)
- Demonstration: Computing NGrams (Lab)
- Joining Datasets in Apache Hive (Lab)
- Computing NGrams of Emails in Avro Format (Lab)
- Using HCatalog with Apache Pig (Lab)
- Describe the Purpose HDFS Federation
- Describe the Purpose of HDFS High Availability (HA)
- Describe the Purpose of the Quorum Journal Manager
- Demonstrate How to Configure Automatic Failover
- Describe the Purpose of YARN
- List the Components of YARN
- Describe the Lifecycle of a YARN Application
- Describe the Purpose of a Cluster View
- Describe the Purpose of Apache Slider
- Describe the Origin and Purpose of Apache Spark
- List Common Spark Use Cases
- Describe the Differences Between Apache Spark and MapReduce
- Demonstrate the Use of the Spark Shell
- Describe the Purpose of an Resilient Distributed Dateset (RDD)
- Demonstrate How to Load Data and Perform a Word Count
- Define Lazy Evaluation
- Describe How to Load Multiple Types of Data
- Demonstrate How to Perform SQL Queries
- Demonstrate How to Perform DataFrame Operations
- Describe the Purpose of the Optimization Engine
- Describe the Purpose of Apache Oozie
- Describe Apache Pig Actions
- Describe Apache Hive Actions
- Describe MapReduce Actions
- Describe How to Submit an Apache Oozie Workflow
- Define an Oozie Coordinator Job
- Advanced Apache Hive Programming (Lab)
- Running a YARN Application (Lab)
- Getting Started with Apache Spark (Lab)
- Exploring Apache Spark SQL (Lab)
- Defining an Apache Oozie Workflow (Lab)
Reviews
"The corporate training provided by Greater Insights was truly exceptional. The trainers were highly knowledgeable and engaging, making the sessions both informative and enjoyable. I gained valuable insights and practical skills that I immediately applied in my work. I highly recommend Greater Insights for their professionalism and expertise."
Rahul NiraniaSamsung 
"As a participant of the corporate training program organized by Greater Insights, I was thoroughly impressed with the level of customization they offered. They took the time to understand our specific needs and tailored the training accordingly. The trainers were fantastic, and the interactive sessions fostered an environment of active learning. I am grateful for the valuable knowledge and skills I acquired."
Bhawna TiwariSamsung 
"I cannot express enough gratitude to Greater Insights for the outstanding corporate training they provided. The trainers were not only experts in their respective fields but also incredible communicators. They created a dynamic and collaborative learning environment, which allowed us to learn from one another. The training surpassed my expectations, and I would eagerly participate in any future programs they offer."
Ashutosh SinghSamsung 
"Attending the corporate training sessions organized by Greater Insights was a game-changer for me. The trainers were not only experienced professionals but also inspiring mentors. They equipped me with practical tools and strategies that have significantly enhanced my productivity and efficiency at work. I wholeheartedly recommend Greater Insights to anyone looking to excel in their professional endeavors."
Somashekar MuniyappaInfogain 
"The corporate training program offered by Greater Insights was an enlightening experience. The trainers had a deep understanding of the subject matter and were able to break down complex concepts into easily digestible information. The interactive activities and case studies made the training engaging and relevant to our day-to-day work challenges. This training has undoubtedly boosted my confidence and competence."
Amresh DiwanInfogain 
"I had the privilege of participating in a corporate training program organized by Greater Insights, and I must say it was a transformative experience. The trainers' expertise and passion for their subjects were evident in every session. The training materials provided were comprehensive and well-structured, enabling us to grasp the content effectively. I am grateful for the valuable skills I acquired, which have greatly contributed to my professional growth."
Namratha BabuTorry Harris 
"The corporate training sessions conducted by Greater Insights were simply outstanding. The trainers went above and beyond to ensure that we understood the concepts thoroughly. The interactive nature of the sessions encouraged active participation and enhanced our learning experience. I am truly grateful for the practical strategies and techniques I learned, which have had a positive impact on my work performance."
Rahul BhashyamTorry Harris 
"Participating in the corporate training program organized by Greater Insights was an enlightening experience. The trainers were not only knowledgeable but also skilled at delivering the content in an engaging manner. The training materials were well-designed, and the real-life examples provided valuable insights. I am confident that the knowledge and skills I gained will significantly contribute to my professional growth."
Deepti JainTorry Harris 
"I can confidently say that the corporate training provided by Greater Insights was top-notch. The trainers were highly experienced and had a knack for simplifying complex concepts. The training sessions were interactive and encouraged open discussions, which fostered a collaborative learning environment. The practical skills I acquired during the training have proven to be invaluable in my day-to-day work."
HARSHAL TRIVEDIL&T TS 
"Greater Insights exceeded my expectations with their corporate training program. The trainers were not only experts in their fields but also exceptional communicators. They effortlessly connected with the participants and ensured everyone's active involvement. The training content was comprehensive and provided a solid foundation for professional growth. I am grateful for the opportunity to learn from such seasoned professionals."
Namrata PillayL&T TS
"The corporate training facilitated by Greater Insights was exceptional in every aspect. The trainers were engaging and created a positive and inclusive learning environment. The training materials were well-structured and easy to follow, making the learning process enjoyable. The practical exercises and case studies allowed me to apply the knowledge immediately, resulting in improved job performance. I highly recommend Greater Insights for their commitment to excellence."
Sandhya M SanuHoneywell
"I recently attended a corporate training program organized by Greater Insights, and I am incredibly impressed. The trainers were not only knowledgeable but also highly skilled at delivering the content in a relatable manner. The training sessions were interactive and encouraged open dialogue, which made the learning experience engaging and dynamic. The practical skills I gained have already made a significant impact on my professional growth."
Neha SahayHoneywell
"I had the privilege of participating in a corporate training program conducted by Greater Insights, and it was truly transformative. The trainers were passionate and dedicated professionals who took the time to address our individual needs. The training sessions were interactive and encouraged active participation, allowing us to learn from one another's experiences. I am grateful for the knowledge and skills I acquired, which have undoubtedly propelled my career forward."
Abishek KalliparambilAllianz 
"The corporate training program offered by Greater Insights was a remarkable experience. The trainers had a wealth of knowledge and were adept at imparting it effectively. The training materials were comprehensive and provided valuable resources for further exploration. The interactive activities and practical exercises helped me internalize the concepts and apply them in real-world scenarios. This training has been instrumental in my professional development."
Kishor KumarITC Infotech 
"I cannot thank Greater Insights enough for the incredible corporate training they provided. The trainers were not only subject matter experts but also skilled at creating a positive and inclusive learning environment. The training sessions were well-structured, and the trainers ensured that we understood the content thoroughly. I am grateful for the valuable insights and skills I gained, which have already made a noticeable difference in my professional life."
Anitha PrashanthIndegene 
