Apache Kafka Engineering Training
0% Completed
M1 — The Log, Records and Partitions
How a record lands in a partition: offsets, keys and the ordering promise
Reading
Segment files, index files and why a topic is a directory of segments
Reading
Page cache behaviour and the difference between sendfile and zero copy
Reading
Topic configuration: partitions, replication factor and segment size
Reading
How delete, retention and compaction differ from one another
Reading
Inspecting a topic with the command line tools before writing any code
Reading
M2 — Replication, the ISR and Leader Election
Followers, the leader and the in-sync replica set in practice
Reading
acks=all, min.insync.replicas and the durability settings that matter
Reading
Under-replicated partitions and the metrics that flag them early
Reading
Leader election, unclean election and the data loss it can cause
Reading
Replica placement across racks, zones and failure domains
Reading
Rebalancing partitions after a broker is added or removed
Reading
M3 — The KRaft Metadata Quorum
Why ZooKeeper was removed in 4.0 and what KRaft replaced it with
Reading
Controller roles, the metadata quorum and how the leader is elected
Reading
The metadata log, snapshots and the controller state machine
Reading
Formatting storage and bootstrapping a combined-mode controller cluster
Reading
Rolling a controller change and recovering a failed metadata quorum
Reading
Reading quorum state through the administration tooling and JMX
Reading
M4 — Consumer Groups and Rebalancing
The classic group protocol, the cooperative assignor and rebalance triggers
Reading
Session timeouts, heartbeats and the cost of a large consumer group
Reading
The new consumer group protocol in 4.0 and the client opt-in it needs
Reading
Where offsets are stored and how a commit is written
Reading
Offset reset policies: earliest, latest and the per-group override
Reading
Static membership and when it reduces rebalance churn
Reading
M5 — Delivery Semantics, Idempotence and Transactions
At most once, at least once and exactly once and how each is configured
Reading
The idempotent producer: producer id, sequence number and dedup window
Reading
The transaction coordinator and the transaction state topic
Reading
Read_committed consumers and how aborted records are handled
Reading
Choosing a delivery guarantee per topic and writing it down
Reading
Replaying a poisoned record without breaking the downstream sink
Reading
M6 — Connect, Storage Topics and Multi-Cluster
The Connect protocol, worker configuration and how connector plugins load
Reading
Source and sink connector configuration, offsets and exactly-once support
Reading
Dead letter topics and the retry pattern built around them
Reading
MirrorMaker 2 and cluster linking for cross-region replication
Reading
Tiered storage topics and moving cold data off the broker disks
Reading
Converter choices for CSV, JSON and Avro on a schema-free pipeline
Reading
M7 — Performance Tuning, Security and Monitoring
Throughput tuning: batch size, linger, compression and the network ceiling
Reading
Latency tuning: queueing, page cache and the disk read path
Reading
Disk layout, striped volumes and the filesystem choice for a data directory
Reading
SASL mechanisms, TLS listener configuration and certificate handling
Reading
Authorising operations with ACLs and the principals that hold them
Reading
The metrics worth graphing: request latency, log flush, ISR shrinks and lag
Reading
M8 — Capstone: Run and Repair a Kafka Cluster
Build a three-broker KRaft cluster on local hosts and format the storage
Reading
Produce and consume a keyed stream and watch offsets and lag move
Reading
Break a broker, watch the ISR shrink and then restore cluster health
Reading
Rebalance a consumer group and prove no record was lost or duplicated
Reading
Add a sink connector with a dead letter topic and a retry path
Reading
Write the runbook, the capacity plan and the tuning change list
Reading
📞
Enroll Now
Start Your Journey
Fill in your details and our team will get back to you within 24 hours