Serverless MLflow on Amazon SageMaker AI for instant experiment tracking and automatic scaling

Accelerate AI development using Amazon SageMaker AI with serverless MLflow

Experimentation speed wins projects. The new serverless MLflow capability in Amazon SageMaker AI turns experiment tracking into an on-demand service - no servers, no capacity planning, and no waiting. You create an MLflow App, get an ARN, and start logging. That's it.

What's new

• Serverless MLflow Apps: Replaces tracking servers with a managed, application-first experience.
• Instant provisioning: Create an MLflow App in about 2 minutes and start tracking immediately.
• Automatic scaling: Scale up or down without sizing decisions or infra tickets.
• MLflow 3.4 support: Access enhanced Tracing for LLM and multi-step AI workflows, with detailed inputs, outputs, and metadata.
• Cross-domain and cross-account sharing: Use AWS RAM shares to collaborate securely across org boundaries.

Getting started in SageMaker AI Studio

• Open SageMaker AI Studio and select the MLflow application.
• You'll see a default MLflow App. Create another by choosing Create MLflow App and naming it.
• Your IAM role and S3 bucket are typically preconfigured; adjust in Advanced settings if needed.
• Provisioning completes in ~2 minutes. Copy the MLflow ARN for use in notebooks and pipelines.
• No server setup, clusters, or capacity tuning required - go straight to running experiments.

Develop faster with MLflow 3.4

MLflow Tracing gives you visibility across complex AI systems: every call, parameter, and output logged with context. That makes debugging, lineage, and reproducibility far more efficient across notebooks, services, and pipelines.

For API details and usage patterns, see the MLflow docs at mlflow.org.

Better together: Pipelines integration

Amazon SageMaker Pipelines now integrates directly with MLflow. If a pipeline runs without an existing App, a default MLflow App is created automatically. Define the experiment name in code - metrics, parameters, and artifacts flow into MLflow as runs.

This works alongside familiar SageMaker AI features like JumpStart and Model Registry, so you can automate from data prep through fine-tuning and promotion with consistent tracking.

How this changes your workflow

• Try ideas immediately: Remove setup lag and prototype more variants per day.
• Simplify ops: No servers to patch, scale, or migrate. Upgrades happen in place.
• Break down silos: Share MLflow Apps across domains and accounts for joint projects.
• Standardize tracking: One place for metrics, params, artifacts, and traces - from notebooks to pipelines.

Things to know

• Pricing: Serverless MLflow capability is available at no additional cost. Service limits apply.
• Availability: Regions include US East (N. Virginia, Ohio), US West (N. California, Oregon), Asia Pacific (Mumbai, Seoul, Singapore, Sydney, Tokyo), Canada (Central), Europe (Frankfurt, Ireland, London, Paris, Stockholm), South America (São Paulo).
• Automatic upgrades: In-place version updates. Current support targets MLflow 3.4, including enhanced tracing.
• Migration support: Use the open source MLflow export-import tool to migrate from existing Tracking Servers (SageMaker AI or self-hosted) to serverless MLflow: github.com/mlflow/mlflow-export-import.

Quick start checklist

• IAM role with access to an S3 bucket for MLflow artifacts.
• Create or use the default MLflow App in SageMaker AI Studio (takes ~2 minutes).
• Connect from your notebook using the provided MLflow ARN.
• Log metrics, parameters, and artifacts; enable Tracing for LLM workflows.
• Run with SageMaker Pipelines to standardize experiments across environments.

Common use cases

• Ad-hoc LLM experiments: Spin up, test prompts and configurations, compare runs, iterate fast.
• Team sandboxes: Give each team an App and share as needed across accounts.
• Pipeline governance: Enforce consistent tracking with artifacts and lineage for reviews and audits.
• Debugging distributed systems: Use Tracing to connect notebook steps, services, and model calls.

Next steps

Open SageMaker AI Studio, create your MLflow App, and start logging your next experiment. If you're migrating, plan a short window to export-import runs and artifacts and keep moving - without re-architecting infrastructure.