Behavioural and case interviews - SkilBrill

Data Engineering Training Behavioural and case interviews

Data Engineering Training

0% Completed

M1 — Data Engineering Foundations

The data engineering lifecycle

Batch vs streaming trade-offs

Source-of-truth modelling concepts

Lab environment setup

M2 — SQL and Python for Data Engineers

Window functions and CTEs

Pandas and Polars essentials

PySpark in 90 minutes

Idempotent pipelines

M3 — Relational Modelling and Warehousing

Star schema and snowflake

Slowly Changing Dimensions (SCD2)

Surrogate keys and audit columns

dbt core models and snapshots

M4 — ETL and ELT Pipelines

Extract patterns: CDC, APIs, files

Transform with SQL, Spark, dbt

Load: warehouse, lake, lakehouse

Error handling and retries

M5 — Orchestration with Apache Airflow

DAGs, operators and sensors

Backfills and SLAs

Idempotency and retries

Production-grade Airflow on Kubernetes

M6 — Streaming with Apache Kafka

Topics, partitions, consumer groups

Exactly-once semantics

Schema Registry and Avro

Stream-table duality with Kafka Connect

M7 — Distributed Compute with Apache Spark

Spark architecture and DAGs

Structured Streaming APIs

Partitioning and shuffle tuning

Adaptive Query Execution

M8 — Lakehouse and Delta Lake

Medallion (bronze/silver/gold) architecture

Delta time travel and OPTIMIZE

Schema enforcement and MERGE

Unity Catalog basics

M9 — Data Quality and Observability

Great Expectations / Soda checks

Data contracts and lineage

Monitoring freshness, volume, quality

Alerting with PagerDuty / Slack

M10 — CI/CD and Infra-as-Code for Data

Git, CI pipelines and pull-request reviews

IaC with Terraform for data stacks

Schema migrations

Promoting dbt models across envs

M11 — Performance, Cost and Security

Query optimisation and Z-ordering

Cost tuning across Snowflake / Databricks

Row/column-level security

PII handling and masking

M12 — Capstone and Interview Prep

End-to-end lakehouse build

Failure-mode simulation

Behavioural and case interviews

Announcements Course Info