MLOps on AWS: SageMaker Pipelines, Feature Store and Model Monitoring

Quick Answer: MLOps on AWS SageMaker means three things working together — Pipelines for a versioned, reproducible train-and-register workflow; Feature Store so training and inference read the same features; and Model Monitor to catch drift before your metrics do. Skip Feature Store and you get training/serving skew; skip monitoring and you find out from customers.

What MLOps on AWS SageMaker Actually Covers in Production

MLOps is not a product. It is the engineering discipline of making models reproducible, deployable, observable and replaceable — and most teams meet it for the first time when a model that worked in a notebook starts making decisions that lose money.

In practice the discipline breaks into five obligations:

1. Reproducibility — the same code, data snapshot and hyperparameters must produce the same model six months later.
2. Versioning — every artefact (data, features, model, config) needs an identifier you can point at during an incident.
3. Automated promotion — training, evaluation and registration run as a pipeline, not as someone’s laptop script.
4. Serving parity — the features used at inference must be the ones used at training, computed the same way.
5. Observability — drift, accuracy decay, latency and data-quality alerts after deployment.

SageMaker gives you managed components for all five. Whether it is the right tool is a separate and fair question, addressed near the end.

SageMaker Pipelines: What It Actually Replaces

Before Pipelines, the default pattern in Indian data teams was a cron job or an Airflow DAG that ran a Python training script, wrote a pickle file to S3, and let someone decide by hand whether to deploy it. That works until the day someone asks which dataset produced the model currently serving traffic and nobody can answer without archaeology.

A SageMaker Pipeline is a directed acyclic graph of typed steps — Processing, Training, Condition, RegisterModel — with execution history, parameterisation and retry built in. The practical wins:

• Every run records its parameters, inputs, outputs and lineage, so “which data trained this model” is a click rather than an afternoon
• Condition steps gate promotion on an evaluation metric, so a model that scores below the current production baseline never registers
• It triggers from EventBridge, so a scheduled retrain is a rule rather than a person
• Runs execute on isolated infrastructure and clean up after themselves — no forgotten training instances

Feature Store: Why Teams Regret Skipping It

Feature Store is the component most teams defer, and it is the one that causes the worst incidents. The problem it solves is called training/serving skew: the features your model learned from are not the features it receives in production, because two different code paths computed them.

The classic shape of the bug: training uses a SQL query joining three tables; inference uses a hand-written transformation in the application. They agree for months. Then someone changes a null-handling rule in one path and not the other. Accuracy degrades quietly, the offline evaluation still looks fine because it uses the training path, and the damage only surfaces in business metrics.

Feature Store fixes this by making one definition per feature, materialised once, readable from both sides — SageMaker Feature Store gives you an offline store for training (Parquet, bulk reads) and a low-latency online store for inference. The cost is real: someone has to own feature definitions and treat them as a product. The alternative cost is worse.

Model Monitor and the Drift You Will Not See

Model Monitor watches three different things, and confusing them is common:

Monitoring requires baselines: a reference dataset captured at deployment and, for quality metrics, a delayed ground-truth feed. Teams without ground truth often underestimate this — you can measure data drift on day one, but you cannot measure accuracy decay until reality tells you what the right answer was.

Set alerts to fire on sustained violation over several windows rather than a single spike. A lone anomaly at 3 AM during a batch load is noise; a two-hour trend is signal.

A Working Pipeline Skeleton

The shape of a real pipeline, in the SageMaker SDK. This is the part that turns a notebook into a system.

from sagemaker.workflow.pipeline import Pipeline
from sagemaker.workflow.steps import TrainingStep, TuneStep, ConditionStep
from sagemaker.workflow.properties import PropertyFile
from sagemaker.estimator import Estimator
from sagemaker.inputs import TrainingInput
from sagemaker.model_metrics import MetricsSource, ModelMetrics

estimator = Estimator(
    image_uri=sklearn_container,
    role=role,
    instance_count=2,
    instance_type="ml.m5.xlarge",
    output_path="s3://acme-ml/models/fraud",
    max_run=3600,
    hyperparameters={"alpha": 0.01},
)

step_train = TrainingStep(
    name="train-fraud-model",
    estimator=estimator,
    inputs={
        "train": TrainingInput(
            s3_data="s3://acme-ml/features/fraud/train",
            content_type="text/csv",
        ),
        "validation": TrainingInput(
            s3_data="s3://acme-ml/features/fraud/valid",
            content_type="text/csv",
        ),
    },
)

# Evaluation lives in a ProcessingStep and writes metrics.json to S3
step_eval = ProcessingStep(
    name="evaluate",
    processor=evaluator,
    inputs=[ProcessingInput(source=step_train.properties.ModelArtifacts.S3ModelArtifacts, ...)],
    outputs=[ProcessingOutput(output_name="metrics", source="/opt/ml/processing/output")],
)

model_metrics = ModelMetrics(
    model_statistics=MetricsSource(
        s3_uri="{}/metrics.json".format(step_eval.properties.ProcessingOutputConfig.Outputs["metrics"].S3Output.S3Uri),
        content_type="application/json",
    )
)

condition = ConditionStep(
    name="check-f1",
    conditions=[JsonGet(property_name="metrics.f1", step_name=step_eval.name, parse_type="Json")],
    if_steps=[step_register],   # RegisterModel with model_metrics
    else_steps=[fail_step],
)

pipeline = Pipeline(
    name="fraud-scoring",
    parameters=[{"Name": "training-data", "DefaultValue": "s3://acme-ml/features/fraud/train"}],
    steps=[step_train, step_eval, condition],
)

pipeline.upsert(role_arn=role)
pipeline.start()

The ConditionStep is the important line. Without it, every retrain registers a new model regardless of whether it improved. With it, a bad run fails visibly and the production endpoint keeps serving the model it has.

The Model That Passed Every Test and Failed on Monday

A fraud-scoring team at a Mumbai fintech had a clean setup: notebook, sklearn pipeline, monthly retrain, model hand-promoted to an endpoint. Offline F1 was 0.91. It went to production on a Friday before a long weekend.

Over that weekend the risk team shipped a change to how merchant names were cleaned — stripping currency symbols and normalising to uppercase. It was a sensible change, made in an application service, with no mention of the model. The feature the model relied on, merchant_amount_norm, was derived from that string. Training data had one distribution; inference now produced another.

By Monday the model was declining roughly 8% of legitimate transactions. Nobody noticed for two days, because the model’s own logs looked healthy — latency was fine, the endpoint returned 200s, and the accuracy dashboard had no ground truth yet for the new week. The signal only appeared when the support queue filled with declined-card complaints.

The fix had two halves. Immediate: roll back to the previous model artifact, which took four minutes because artefacts were versioned. Structural: move every feature into Feature Store with a single owner, add a distribution comparison between the offline reference set and live traffic, and require an ML sign-off on any upstream change to a feature-producing service. That last rule is organisational, not technical — and it is most of what MLOps actually is.

Notice what saved them: versioned artefacts and a four-minute rollback. Teams that can only redeploy from a notebook have no such option.

Do You Actually Need SageMaker? The Build-vs-Buy Call

Honest answer: often, no. SageMaker is convenient and priced accordingly. A small team with two models and strong DevOps skills can assemble the same discipline from ECS or Lambda, S3, GitHub Actions, MLflow on a small instance, and CloudWatch — and pay a fraction of it.

The mistake in both directions is common: enterprises buy the full platform for two models and get bogged down in configuration, while startups hand-roll everything and discover at scale that nobody knows which model is serving. The tell for needing proper tooling is the number of people touching models, not the number of models.

What Interviews and Hiring Bars Look Like

MLOps roles in India sit at ₹15–30 LPA mid-level and ₹30–55 LPA senior, overlapping heavily with senior ML engineer positions. Interviewers care about three things: whether you can explain skew with a concrete example, whether you have actually implemented a retraining trigger and a rollback path, and whether you understand that a pipeline without a Condition step is just automation with extra steps.

If you are coming from a DevOps background rather than a modelling one, the fastest credible entry is pipeline engineering — you already know CI/CD, and the ML-specific surface is smaller than it looks. Our DevOps salary guide covers how that transition is usually priced by employers.

Training Resources

Build the whole loop in SkilBrill AWS Machine Learning Training — SageMaker Pipelines, Feature Store, deployment and model monitoring taught as one production workflow. If you are starting from the generative side, our Bedrock RAG guide is a useful companion read.

FAQ

What is MLOps on AWS SageMaker?

It is the set of practices — versioned data and models, automated training pipelines, gated promotion, monitored deployment — implemented with SageMaker Pipelines for orchestration, SageMaker Feature Store for feature reuse, and Model Monitor for post-deployment drift detection.

Is SageMaker Feature Store necessary for a small team?

If you have one model and one feature path, probably not yet. The moment two code paths compute the same feature — training SQL versus application code — you have training/serving skew risk, and that is when a single feature definition starts paying for itself.

How does SageMaker Model Monitor detect model drift?

By comparing live input distributions against a reference baseline for data drift, and against a delayed ground-truth feed for quality metrics. Without ground truth you can only detect data drift, which is an early warning rather than proof of degradation.

Can I do MLOps on AWS without SageMaker?

Yes. ECS or Lambda for training jobs, S3 for artefacts, GitHub Actions or CodePipeline for orchestration, MLflow or DVC for tracking, and CloudWatch for alerts. It costs less and takes more assembly; it is the right call for a small number of models with a strong platform team.

What salary does an MLOps engineer earn in India?

₹15–30 LPA at mid-level and ₹30–55 LPA senior in 2026, concentrated in Bangalore, Hyderabad and Mumbai. Candidates who can demonstrate a working retrain-and-rollback loop command the upper end, because most cannot.

Written by SkilBrill Training Institute

October 15, 2026 · 9 min read