Auto-ingest and Kafka connector - SkilBrill

Skip to content

Home
Courses
About
Contact
Blog
- Freshers

Search for:

Home - Lessons - Data Engineering with Snowflake Training - M2 — Snowpipe and Continuous Ingestion - Auto-ingest and Kafka connector

Data Engineering with Snowflake Training Auto-ingest and Kafka connector

Data Engineering with Snowflake Training

0% Completed

M1 — Snowflake Recap and Project Setup

Snowflake architecture refresher

Account setup and trial

SnowSQL and Snowpark CLI

Project structure

M2 — Snowpipe and Continuous Ingestion

Snowpipe architecture

Auto-ingest and Kafka connector

Snowpipe Streaming

Error handling and offsets

M3 — Medallion with dbt

Bronze, silver and gold

dbt models and snapshots

Incremental and merge strategies

Tests and exposures

M4 — Streams and Tasks for CDC

Streams at table level

Task DAGs and CRON

MERGE-driven incremental loads

Failure handling

M5 — Streaming with Kafka and Snowpipe Streaming

Kafka topic design

Kafka connector configuration

Schema evolution

Backfill strategies

M6 — Orchestration with Apache Airflow

Airflow DAGs and sensors

SnowflakeOperator

Backfills and SLAs

Production-grade Airflow on K8s

M7 — Snowpark for Python and Java

Stored procedures with Snowpark

External functions

M8 — Data Quality and Observability

Soda and Great Expectations

Freshness and volume checks

Alerting with PagerDuty and Slack

M9 — Cost Optimisation and FinOps

Warehouse right-sizing

Resource monitors

Auto-suspend tuning

Cost allocation tags

M10 — CI/CD and IaC for Snowflake

Terraform Snowflake provider

GitHub Actions pipelines

dbt Cloud and dbt Core

Environment promotion

M11 — Security and Sharing

Row-access and masking policies

Secure views and data sharing

Marketplace listings

Audit and compliance

M12 — Capstone and SnowPro Advanced Mocks

End-to-end medallion pipeline

Cost and quality review

SnowPro Advanced mocks

Announcements Course Info

Global Trusted Software Training Partner Since 2022

SkilBrill Training Institute delivers industry-focused technology education that bridges the gap between academic learning and real-world demands. Our structured programs combine foundational theory with practical application, ensuring every graduate is confident and job-ready from day one.

Learn More About Us

© 2026 SkilBrill Training Institute. All Rights Reserved. Privacy Policy Terms Refunds Outcomes

📞 Enroll Now

Start Your Journey

Fill in your details and our team will get back to you within 24 hours

Please enable JavaScript in your browser to complete this form.

Full Name *

Email *

Course Interested In *

Mode of Course *