Apache Hadoop
CloudLabs
Projects
Assignment
24x7 Support
Lifetime Access
.
Course Overview
Starting from the very basic, this advanced course on Apache Hadoop Administration teaches you the functions and major goals of HDFS, explains how YARN works, and then covers Hadoop cluster planning, installation and administration. This course also teaches Resource management, logging and monitoring
At the end of the training, participants will be able to:
Understand Hadoop Distributed File System Understand YARN Learn Hadoop Cluster Planning, Installation and Administration
Pre-requisite
- Basic Networking Concepts and operating system knowledge
Duration
3 days
Course Outline
- Describe the function of HDFS daemons
- Describe the normal operation of an Apache Hadoop cluster, both in data storage and in data processing
- Identify current features of computing systems that motivate a system like Apache Hadoop
- Classify major goals of HDFS Design
- Given a scenario, identify appropriate use case for HDFS Federation
- Identify components and daemon of an HDFS HA-Quorum cluster
- Analyze the role of HDFS security (Kerberos)
- Determine the best data serialization choice for a given scenario
- Describe file read and write paths
- Identify the commands to manipulate files in the Hadoop File System Shell
- Understand how to deploy core ecosystem components, including Spark, Impala, and Hive
- Understand how to deploy MapReduce v2 (MRv2 / YARN), including all YARN daemons
- Understand basic design strategy for YARN and Hadoop
- Determine how YARN handles resource allocations
- Identify the workflow of job running on YARN
- Determine which files you must change and how in order to migrate a cluster from MapReduce version 1 (MRv1) to MapReduce version 2 (MRv2) running on YARN
- Principal points to consider in choosing the hardware and operating systems to host an Apache Hadoop cluster
- Analyze the choices in selecting an OS
- Understand kernel tuning and disk swapping
- Given a scenario and workload pattern, identify a hardware configuration appropriate to the scenario
- Given a scenario, determine the ecosystem components your cluster needs to run in order to fulfill the SLA
- Cluster sizing: given a scenario and frequency of execution, identify the specifics for the workload, including CPU, memory, storage, disk I/O
- Disk Sizing and Configuration, including JBOD vs RAID, SANs, virtualization, and disk sizing requirements in a cluster
- Network Topologies: understand network usage in Hadoop (for both HDFS and MapReduce) and propose or identify key network design components for a given scenario
- Given a scenario, identify how the cluster will handle disk and machine failures
- Analyze a logging configuration and logging configuration file format
- Understand the basics of Hadoop metrics and cluster health monitoring
- Identify the function and purpose of available tools for cluster monitoring
- Be able to install all the ecosystem components in CDH 5, including (but not limited to): Impala,
- Flume, Oozie, Hue, Cloudera Manager, Sqoop, Hive, and Pig
- Identify the function and purpose of available tools for managing the Apache Hadoop file system
- Understand the overall design goals of each of Hadoop schedulers
- Given a scenario, determine how the FIFO Scheduler allocates cluster resources
- Given a scenario, determine how the Fair Scheduler allocates cluster resources under YARN
- Given a scenario, determine how the Capacity Scheduler allocates cluster resources
- Understand the functions and features of Hadoop’s metric collection abilities
- Analyze the NameNode and JobTracker Web UIs
- Understand how to monitor cluster daemons
- Identify and monitor CPU usage on master nodes
- Describe how to monitor swap and memory allocation on all nodes
- Identify how to view and manage Hadoop’s log files
- Interpret a log file
Reviews
"The corporate training provided by Greater Insights was truly exceptional. The trainers were highly knowledgeable and engaging, making the sessions both informative and enjoyable. I gained valuable insights and practical skills that I immediately applied in my work. I highly recommend Greater Insights for their professionalism and expertise."
Rahul NiraniaSamsung
"As a participant of the corporate training program organized by Greater Insights, I was thoroughly impressed with the level of customization they offered. They took the time to understand our specific needs and tailored the training accordingly. The trainers were fantastic, and the interactive sessions fostered an environment of active learning. I am grateful for the valuable knowledge and skills I acquired."
Bhawna TiwariSamsung
"I cannot express enough gratitude to Greater Insights for the outstanding corporate training they provided. The trainers were not only experts in their respective fields but also incredible communicators. They created a dynamic and collaborative learning environment, which allowed us to learn from one another. The training surpassed my expectations, and I would eagerly participate in any future programs they offer."
Ashutosh SinghSamsung
"Attending the corporate training sessions organized by Greater Insights was a game-changer for me. The trainers were not only experienced professionals but also inspiring mentors. They equipped me with practical tools and strategies that have significantly enhanced my productivity and efficiency at work. I wholeheartedly recommend Greater Insights to anyone looking to excel in their professional endeavors."
Somashekar MuniyappaInfogain
"The corporate training program offered by Greater Insights was an enlightening experience. The trainers had a deep understanding of the subject matter and were able to break down complex concepts into easily digestible information. The interactive activities and case studies made the training engaging and relevant to our day-to-day work challenges. This training has undoubtedly boosted my confidence and competence."
Amresh DiwanInfogain
"I had the privilege of participating in a corporate training program organized by Greater Insights, and I must say it was a transformative experience. The trainers' expertise and passion for their subjects were evident in every session. The training materials provided were comprehensive and well-structured, enabling us to grasp the content effectively. I am grateful for the valuable skills I acquired, which have greatly contributed to my professional growth."
Namratha BabuTorry Harris
"The corporate training sessions conducted by Greater Insights were simply outstanding. The trainers went above and beyond to ensure that we understood the concepts thoroughly. The interactive nature of the sessions encouraged active participation and enhanced our learning experience. I am truly grateful for the practical strategies and techniques I learned, which have had a positive impact on my work performance."
Rahul BhashyamTorry Harris
"Participating in the corporate training program organized by Greater Insights was an enlightening experience. The trainers were not only knowledgeable but also skilled at delivering the content in an engaging manner. The training materials were well-designed, and the real-life examples provided valuable insights. I am confident that the knowledge and skills I gained will significantly contribute to my professional growth."
Deepti JainTorry Harris
"I can confidently say that the corporate training provided by Greater Insights was top-notch. The trainers were highly experienced and had a knack for simplifying complex concepts. The training sessions were interactive and encouraged open discussions, which fostered a collaborative learning environment. The practical skills I acquired during the training have proven to be invaluable in my day-to-day work."
HARSHAL TRIVEDIL&T TS
"Greater Insights exceeded my expectations with their corporate training program. The trainers were not only experts in their fields but also exceptional communicators. They effortlessly connected with the participants and ensured everyone's active involvement. The training content was comprehensive and provided a solid foundation for professional growth. I am grateful for the opportunity to learn from such seasoned professionals."
Namrata PillayL&T TS
"The corporate training facilitated by Greater Insights was exceptional in every aspect. The trainers were engaging and created a positive and inclusive learning environment. The training materials were well-structured and easy to follow, making the learning process enjoyable. The practical exercises and case studies allowed me to apply the knowledge immediately, resulting in improved job performance. I highly recommend Greater Insights for their commitment to excellence."
Sandhya M SanuHoneywell
"I recently attended a corporate training program organized by Greater Insights, and I am incredibly impressed. The trainers were not only knowledgeable but also highly skilled at delivering the content in a relatable manner. The training sessions were interactive and encouraged open dialogue, which made the learning experience engaging and dynamic. The practical skills I gained have already made a significant impact on my professional growth."
Neha SahayHoneywell
"I had the privilege of participating in a corporate training program conducted by Greater Insights, and it was truly transformative. The trainers were passionate and dedicated professionals who took the time to address our individual needs. The training sessions were interactive and encouraged active participation, allowing us to learn from one another's experiences. I am grateful for the knowledge and skills I acquired, which have undoubtedly propelled my career forward."
Abishek KalliparambilAllianz
"The corporate training program offered by Greater Insights was a remarkable experience. The trainers had a wealth of knowledge and were adept at imparting it effectively. The training materials were comprehensive and provided valuable resources for further exploration. The interactive activities and practical exercises helped me internalize the concepts and apply them in real-world scenarios. This training has been instrumental in my professional development."
Kishor KumarITC Infotech
"I cannot thank Greater Insights enough for the incredible corporate training they provided. The trainers were not only subject matter experts but also skilled at creating a positive and inclusive learning environment. The training sessions were well-structured, and the trainers ensured that we understood the content thoroughly. I am grateful for the valuable insights and skills I gained, which have already made a noticeable difference in my professional life."
Anitha PrashanthIndegene