Hadoop and Big Data Engineering Certification Training
0% Completed
M1 — The Hadoop Stack as a System
HDFS, YARN, MapReduce and the layer each one adds to the platform
Reading
The master and worker model and where the single point of failure sits
Reading
A small cluster on local hosts and the configuration files that set it up
Reading
The data flow from an edge node to a file on the distributed filesystem
Reading
ZooKeeper coordination: leader election, failover and watchers
Reading
Where the open-source stack ends and a vendor distribution begins
Reading
M2 — HDFS: Blocks, Replication and the NameNode
Block size, replication factor and the block report heartbeat
Reading
The NameNode namespace, the fsimage and the edit log
Reading
Safe mode, checkpoints and reading fsck output on a damaged block
Reading
Reading and writing with the command line client and with Java
Reading
The write pipeline as a record travels through several nodes to disk
Reading
Adding a data node, decommissioning one and re-replicating its data
Reading
M3 — YARN: Resource Management and Containers
The resource manager, the node manager and the application master
Reading
How a job is queued, scheduled and assigned to a container
Reading
Container memory and CPU limits and sizing the map and reduce stages
Reading
Reading the application timeline and the container logs
Reading
Queue configuration, capacity and the preemption a backlog causes
Reading
Why an over-allocated map stage stalls an entire pipeline
Reading
M4 — MapReduce and the Move to Spark
The map, shuffle, sort and reduce stages and the role of combiners
Reading
Key, value and value types and the serialisation that carries them
Reading
Counters, job history and how a single slow task shows up
Reading
The Spark session, the DataFrame and lazy evaluation against MapReduce
Reading
Partitions, shuffles and the narrow and wide dependency boundary
Reading
Choosing batch, Spark SQL or streaming for one workload
Reading
M5 — Hive: Tables, Partitions and ORC
External and managed tables and what the filesystem holds afterwards
Reading
Partitioning by date and bucketing for directories too numerous to list
Reading
The metastore, the table definition and the schema on disk
Reading
ORC and Parquet files, compression and the columnar read advantage
Reading
SerDe, the data file format and the layout the reader expects
Reading
ACID transactions and the compaction that keeps small files in check
Reading
M6 — HBase, Sqoop and Flume
The HBase table, row key, column family and the region split
Reading
Row key design for a random read pattern and for a scan pattern
Reading
The HBase write path, the memstore and the region server flush
Reading
Sqoop import from a relational table with a parallel extract
Reading
Sqoop incremental import driven by a checkpoint column
Reading
Flume agents, sources, channels and sinks for a log stream
Reading
M7 — Schedulers, Pipelines and the Certification Tracks
Airflow DAGs, operators and sensors for an orchestrated pipeline
Reading
Backfill strategy, idempotence and a failed task recovery plan
Reading
Data quality checks that stop a bad file before it reaches the table
Reading
HDPC and the HDP administrator objectives mapped onto the stack
Reading
The Cloudera data engineer exam objectives and its hands-on tasks
Reading
The HDInsight path in the Microsoft data engineer role and its lab setup
Reading
M8 — Capstone: Build a Hadoop Pipeline on a Small Cluster
Stand up a five-node cluster with HDFS, YARN and a metastore service
Reading
Ingest a relational extract with Sqoop and a log stream with Flume
Reading
Model the data in Hive with partitions and an ORC layout
Reading
Write a Spark job that joins and aggregates the two sources
Reading
Write a second pipeline job and schedule it with the orchestrator
Reading
Document the cluster, the capacity plan and the exam revision plan
Reading
📞
Enroll Now
Start Your Journey
Fill in your details and our team will get back to you within 24 hours