# What is Amazon MWAA Serverless?


 Amazon MWAA Serverless is a deployment option for MWAA that eliminates the operational overhead of managing Apache Airflow environments while providing cost-effective serverless scaling. This solution simplifies managing operational scale, optimizing costs, and handling access control. 

 With Amazon MWAA Serverless, teams can concentrate on developing their workflow logic instead of worrying about capacity management. The service automatically provides resources for both scheduled and on-demand workflow execution, with a cost model based on the actual compute time during task execution. 

 Amazon MWAA Serverless also features an enhanced security architecture leveraging AWS Identity and Access Management (IAM). Individual workflows can be assigned specific IAM permissions with tasks run in your chosen Virtual Private Cloud (VPC), enabling precise security controls without requiring separate Airflow environments. This design reduces security administration overhead while strengthening overall security measures. 

**Note**  
 Amazon MWAA Serverless uses [Apache Airflow v3](https://airflow.apache.org/docs/apache-airflow/3.0.0/release_notes.html#airflow-3-0-0-2025-04-22) with Python 3.12. 

**Topics**
+ [

## Key features
](#mwaas-feature-overview)
+ [

## How Amazon MWAA Serverless works
](#mwaas-architecture-overview)
+ [

## Accessing Amazon MWAA Serverless
](#acessing-mwaas)
+ [

## Regions
](#regions)
+ [

## Are you a first-time Amazon MWAA Serverless user?
](#first-time-user)
+ [

# Key concepts
](mwaas-concepts.md)
+ [

# Apache Airflow Parameter Support
](supported-airflow-parameters.md)
+ [

# Template Variables, Filters and Macros
](supported-jinja-parameters.md)

## Key features


The following are key features of Amazon MWAA Serverless:
+  **Serverless architecture**: Provides an Amazon MWAA deployment option that eliminates the need to provision, configure, and tune Apache Airflow infrastructure. 
+  **Automatic scaling**: Dynamically scales compute resources to meet your workflow demands without manual intervention. 
+  **Workflow Isolation**: Each workflow runs with its own execution role and worker, providing enhanced security and preventing cross-workflow interference. 
+  **Enhanced security**: Workflow-specific IAM permissions for least-privilege access (compared to shared permissions in provisioned Amazon MWAA). Additionally, you can choose to run workflows in your VPC, where you can control the security for each workflow. 
+  **Cost optimization**: You only pay for actual workflow run time, which removes costs associated with idle resources. There are no upfront costs and no minimum provisioning is required. 
+  **Flexible workflow format**: Use YAML-based definition files to define your workflows. 
+  **AWS integration**: Use built-in access to other AWS services through open source Apache Airflow operators. 
+  **Workflow versioning**: Built-in versioning system allows you to manage workflow evolution and rollback to previous versions when needed. 

### Amazon MWAA Serverless versus traditional Amazon MWAA


 Understanding when to choose Amazon MWAA Serverless over Amazon MWAA helps you select the right orchestration approach for your use case. 


| Category | Amazon MWAA Serverless | Traditional Amazon MWAA | 
| --- | --- | --- | 
| Infrastructure management | Fully managed - no infrastructure to configure or maintain | Managed Airflow environment with configurable capacity | 
| Scaling | Automatic scaling based on workflow demand |  Fixed always-on [environment capacity](https://docs.aws.amazon.com/mwaa/latest/userguide/environment-class.html) , from micro to 2XL sizes, with [automatic scaling](https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-autoscaling.html) of Airflow workers  | 
| Cost model | Pay-per-use: charged only when workflows run |  [Hourly rate](https://aws.amazon.com/managed-workflows-for-apache-airflow/pricing/) for environment, schedulers, web servers, and additional workers  | 
| Workflow isolation |  Each workflow has its own execution role and access permissions. Tasks each run in isolated compute assuming the workflow's execution role.  |  Each MWAA environment provides the same access to all tasks. Airflow's role-based authentication supports UI-level level isolation within an environment  | 
| Startup time | Each task provisions its own compute before execution | Environment always running, fast execution of tasks when worker capacity exists | 
| Workflow definition | YAML workflows definitions using the DAG factory format, supporting AWS operators, with option to convert Python definitions | Python DAG support with AWS and custom operators | 
| Airflow UI access | No direct Airflow UI access (monitoring via logs) | Airflow web interface access | 

### When to use Amazon MWAA Serverless


Choose Amazon MWAA Serverless when you want to focus on workflow logic rather than infrastructure management. Amazon MWAA Serverless is particularly well-suited for:
+  **Variable or unpredictable workloads**: Workflows that run sporadically or have varying resource requirements benefit from automatic scaling and pay-per-use pricing. 
+  **Cost-sensitive environments**: Development, testing, or production workloads where you want to minimize costs by avoiding idle infrastructure charges. 
+  **Multi-team organizations**: Teams that need workflow isolation for security or compliance reasons, with each workflow running under its own execution role. 
+  **AWS-native data pipelines**: Workflows primarily using AWS services like S3, Glue, EMR, Redshift, and SageMaker through built-in operators. 
+  **Simplified operations**: Organizations that want to reduce operational overhead by eliminating Apache Airflow environment management. 

Consider provisioned Amazon MWAA when you need:
+ Custom Python operators or complex DAG logic
+ Direct access to the Airflow web interface
+ Consistently high-volume workloads that benefit from always-on infrastructure
+ Specific Airflow plugins or configurations not supported in the serverless model

## How Amazon MWAA Serverless works


 Amazon MWAA Serverless uses a multi-tenant architecture that automatically manages infrastructure while maintaining strong isolation between workflows. Here's how it works: 
+ **Workflow submission**: You submit YAML-based DAG definitions through AWS CLI, or API.
+ **Automatic scheduling**: Amazon MWAA Serverless uses EventBridge Scheduler for reliable, timezone-aware scheduling of your workflows.
+ **Resource allocation**: When a workflow runs, Amazon MWAA Serverless automatically provisions the necessary compute resources.
+  **Isolated execution**: Each workflow runs with its own execution role and isolated compute environment, ensuring security and preventing interference. 
+  **Automatic cleanup**: Resources are automatically released when workflows complete, ensuring you only pay for actual usage. 

This architecture provides the benefits of Apache Airflow workflow orchestration without the operational complexity of managing Airflow infrastructure.

## Accessing Amazon MWAA Serverless


You can access Amazon MWAA Serverless through multiple interfaces:
+ **AWS Management Console**: Monitor workflows through the AWS Management Console.
+ **AWS CLI**: Use the AWS CLI for programmatic workflow management and automation.
+ **API**: Integrate Amazon MWAA Serverless into your applications using the REST API for workflow lifecycle management.
+ **SDKs**: Use AWS SDKs in your preferred programming language to build applications that manage workflows.

## Regions


Amazon MWAA Serverless is available in the following AWS Regions.
+ Europe (Ireland): eu-west-1
+ Europe (London): eu-west-2
+ Europe (Paris): eu-west-3
+ Europe (Frankfurt): eu-central-1
+ Europe (Stockholm): eu-north-1
+ US East (N. Virginia): us-east-1
+ US East (Ohio): us-east-2
+ US West (Oregon): us-west-2
+ South America (São Paulo): sa-east-1
+ Asia Pacific (Tokyo): ap-northeast-1
+ Asia Pacific (Seoul): ap-northeast-2
+ Asia Pacific (Mumbai): ap-south-1
+ Asia Pacific (Singapore): ap-southeast-1
+ Asia Pacific (Sydney): ap-southeast-2
+ Canada (Central): ca-central-1
+ Asia Pacific (Thailand): ap-southeast-7
+ Africa (Cape Town): af-south-1
+ Asia Pacific (Hong Kong): ap-east-1
+ Asia Pacific (Malaysia): ap-southeast-5
+ Europe (Spain): eu-south-2
+ Europe (Zurich): eu-central-2
+ US West (N. California): us-west-1
+ Asia Pacific (Osaka): ap-northeast-3

## Are you a first-time Amazon MWAA Serverless user?


If you are a first-time user of Amazon MWAA Serverless, we recommend that you begin by reading the following sections:
+ [Prerequisites for using Amazon MWAA Serverless](prerequisites-set-up.md)
+ [Get started with Amazon MWAA Serverless](get-started.md)
+ [Workflows](workflows.md)

# Key concepts


 Understanding Amazon MWAA Serverless concepts helps you design, deploy, and manage your workflow orchestration solutions. This section explains the core concepts and how they relate to traditional Apache Airflow terminology. 
+  **Workflow**: The Amazon MWAA Serverless resource that represents your orchestration logic. A workflow is created through the Amazon MWAA Serverless API, or CLI. It contains metadata about scheduling, execution roles, and the orchestrated data processing steps (tasks). 
+  **Workflow Definition**: The Apache Airflow concept that defines the structure and dependencies of your tasks. While creating your Workflow, you provide your workflow definition file in Amazon S3. You can use the [Python to YAML DAG converter ](https://pypi.org/project/python-to-yaml-dag-converter-mwaa-serverless/) to convert existing Python based DAG to YAML definitions. 
+  **[Task](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html)**: An individual unit of work within a workflow. Each task represents a specific operation that runs on a worker. 
+  **[Operator](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/operators.html)**: An Operator is a template for a predefined task. 

**Topics**
+ [

## Workflow isolation
](#workflow-isolation)
+ [

## Workflow versioning
](#workflow-versioning)
+ [

## Execution model
](#execution-model)
+ [

## YAML support
](#yaml-dag-authoring)
+ [

## Monitoring
](#concepts-monitoring)

## Workflow isolation


 One of the key benefits of Amazon MWAA Serverless is workflow isolation, which provides security and operational benefits: 
+  **Execution role isolation**: Each workflow runs with its own IAM execution role, ensuring that workflows can only access the resources they're explicitly granted permission to use. 
+  **Compute isolation**: Workflows run on dedicated compute resources that are provisioned when the workflow starts and released when it completes. 
+  **Network isolation**: If you choose to specify a VPC, each workflow's tasks run in isolated network environments with their own security group configurations. VPCs are optional with Amazon MWAA Serverless. 

This isolation model contrasts with traditional Amazon MWAA where all workflows share the same Airflow environment and execution context.

## Workflow versioning


Amazon MWAA Serverless automatically manages workflow versions to help you track changes and enable rollbacks:
+  **Automatic versioning**: Each time you update a workflow, Amazon MWAA Serverless creates a new version while preserving previous versions. 
+ **Version identification**: Versions are identified by alphanumeric ID.
+ **Immutable versions**: Once created, workflow versions cannot be modified, ensuring consistency and enabling reliable rollbacks.

 This versioning system allows you to safely evolve your workflows while maintaining the ability to rollback to previous working versions if issues arise. 

## Execution model


Amazon MWAA Serverless uses a serverless execution model that differs significantly from traditional Airflow deployments:

### On-demand provisioning


Resources are provisioned only when workflows need to run:
+  **Workflow startup**: When a workflow is triggered (by schedule or on-demand), Amazon MWAA Serverless provisions the necessary compute resources. 
+ **Task execution**: Individual tasks run on isolated compute instances with the appropriate execution role and network configuration.
+ **Automatic cleanup**: Resources are automatically released when the workflow completes, ensuring you only pay for actual usage.

### Scheduling model


Amazon MWAA Serverless uses EventBridge Scheduler for reliable workflow scheduling:
+ **Timezone support**: Schedules can be defined with specific timezones, ensuring workflows run at the correct local time regardless of AWS region.
+ **Flexible scheduling**: Support for cron expressions, rate expressions, and one-time schedules.
+ **Reliable delivery**: EventBridge Scheduler provides built-in retry logic and dead letter queue support for failed workflow triggers.

## YAML support


Amazon MWAA Serverless uses YAML based workflow definitions that provide a declarative approach to workflow authoring. You can use the Python to YAML converter tool to convert exiting Python based DAGs to YAML definition.
+ **Declarative syntax**: Define your workflow structure, dependencies, and task configurations using YAML syntax.
+ **Version control-friendly**: YAML files are easy to version control, review, and collaborate on.
+ **Validation**: Amazon MWAA Serverless validates your YAML definitions before execution to catch configuration errors early.
+ **Template support**: Use Jinja2 templating within your YAML definition for dynamic configuration.

Example YAML structure:

------
#### [ YAML ]

```
dag_id: my_data_pipeline
description: "Process daily data from S3 to Redshift"
schedule_interval: "0 2 * * *"  # Daily at 2 AM
start_date: "2024-01-01"

tasks:
extract_data:
operator: S3ListOperator
bucket: my-data-bucket
prefix: daily/

transform_data:
operator: GlueJobOperator
job_name: transform-daily-data
depends_on: [extract_data]

load_data:
operator: RedshiftSQLOperator
sql: "COPY target_table FROM 's3://processed-data/'"
depends_on: [transform_data]
```

------

## Monitoring


You can observe your Amazon MWAA Serverless resources using AWS native services:
+ **CloudWatch logs**: Access your workflow and task logs in CloudWatch for analysis and troubleshooting.
+ **CloudTrail integration**: API calls and workflow management actions are logged in CloudTrail for audit and compliance.
+ **Console**: View workflow status using the Amazon MWAA Serverless console.

Unlike traditional Amazon MWAA, you don't have direct access to the Apache Airflow web interface. Instead, you can build your custom monitoring and observability using AWS native tools.

# Apache Airflow Parameter Support
Supported Parameters

 Amazon MWAA Serverless leverages Apache Airflow as its workflow orchestration engine and supports YAML workflow definitions that are build using allowlisted [Supported operators](operators.md). To provide the serverless experience using these operators, Amazon MWAA Serverless supports a limited set of [Apache Airflow parameters](https://airflow.apache.org/docs/apache-airflow/3.0.6/core-concepts/params.html). While building your workflow definitions, we recommend you to refer to the follwoing paramter support: 

**Topics**
+ [

## DAG level parameters that Amazon MWAA Serverless will validate
](#dag-object-attributes-validation)
+ [

## Task level parameters that Amazon MWAA Serverless will validate
](#task-object-attributes-validation)
+ [

## DAG level paramters that are not supported by Amazon MWAA Serverless
](#dag-object-attributes-not-supported)
+ [

## Task level paratmers that are not suppored by Amazon MWAA Serverless
](#base-operator-attributes-not-supported)
+ [

## AWS base operator attributes
](#aws-base-operator-attributes)

## DAG level parameters that Amazon MWAA Serverless will validate


 The following table lists the supported Apache Airflow DAG level parameters that are subject to validation by Amazon MWAA Serverless during a create or update of workflow. 


| Parameter | Validation Rule | Default Value | 
| --- | --- | --- | 
| dag\$1id | Must be a valid, non-empty string |  | 
| schedule | Must be a valid CRON expression format |  | 
| start\$1date | Must be in the future |  | 
| end\$1date | Must be after or equal to start\$1date |  | 
| max\$1active\$1runs | Must be smaller than account limit. Please refer to [Quotas for Amazon MWAA Serverless](mwaa-serverless-quotas.md) | 16 | 

## Task level parameters that Amazon MWAA Serverless will validate


 The following table lists the Apache Airflow Task level parameters that are are subject to validation by Amazon MWAA Serverless during a task execution. 


| Parameter | Validation Rule | Default Value | 
| --- | --- | --- | 
| task\$1id | Must be a valid string |  | 
| retries | Must be between 0 to 3 | 1 | 
| retry\$1delay | Must be between 0 to 300 seconds | 300 seconds | 
| execution\$1timeout | Maximum 3600 seconds | 3600 seconds | 

## DAG level paramters that are not supported by Amazon MWAA Serverless


 The following table lists the Apache Airflow DAG level parameters that are not supported by Amazon MWAA Serverless and will be ignored. 


| Parameter | 
| --- | 
| dag\$1id | 
| template\$1searchpath | 
| template\$1undefined | 
| user\$1defined\$1macros | 
| user\$1defined\$1filters | 
| catchup | 
| access\$1control | 
| jinja\$1environment\$1kwargs | 
| render\$1template\$1as\$1native\$1obj | 
| tags | 
| owner\$1links | 
| auto\$1register | 
| fail\$1fast | 
| dag\$1display\$1name | 
| depends\$1on\$1past | 
| email\$1on\$1failure | 
| email\$1on\$1retry | 
| description | 
| max\$1consecutive\$1failed\$1dag\$1runs | 
| dagrun\$1timeout | 
| sla\$1miss\$1callback | 
| on\$1failure\$1callback | 
| on\$1success\$1callback | 
| is\$1paused\$1upon\$1creation | 

## Task level paratmers that are not suppored by Amazon MWAA Serverless


 The following table lists the Apache Airflow Task level parameters that are not supported by Amazon MWAA Serverless and will be ignored. 


| Parameter | 
| --- | 
| email\$1on\$1retry | 
| email\$1on\$1failure | 
| retry\$1exponential\$1backoff | 
| depends\$1on\$1past | 
| ignore\$1first\$1depends\$1on\$1past | 
| wait\$1for\$1downstream | 
| priority\$1weight | 
| sla | 
| max\$1active\$1tis\$1per\$1dag | 
| max\$1active\$1tis\$1per\$1dagrun | 
| task\$1concurrency | 
| resources | 
| run\$1as\$1user | 
| executor\$1config | 
| doc | 
| doc\$1md | 
| doc\$1rst | 
| doc\$1json | 
| doc\$1yaml | 
| task\$1display\$1name | 
| logger\$1name | 
| allow\$1nested\$1operators | 
| inlets | 
| outlets | 
| map\$1index\$1template | 
| email | 
| owner | 
| max\$1retry\$1delay | 
| on\$1execute\$1callback | 
| on\$1failure\$1callback | 
| on\$1success\$1callback | 
| on\$1retry\$1callback | 
| on\$1skipped\$1callback | 
| wait\$1for\$1past\$1depends\$1before\$1skipping | 
| do\$1xcom\$1push | 
| multiple\$1outputs | 
| start\$1date | 
| end\$1date | 
| weight\$1rule | 
| queue | 
| pool | 
| pool\$1slots | 
| pre\$1execute | 
| post\$1execute | 
| trigger\$1rule | 
| executor | 
| task\$1group | 

## AWS base operator attributes



| Parameter | Amazon MWAA | Amazon MWAA Serverless | 
| --- | --- | --- | 
| aws\$1conn\$1id | Supported | Controlled by service | 
| verify | Supported | Not supported | 
| botocore\$1config | Supported | Not supported | 
| region\$1name | Supported | Gets Region from environment | 

# Template Variables, Filters and Macros
Supported Airflow macros and Jinja paramters

 Amazon MWAA Serverless supports [Jinja templates](https://airflow.apache.org/docs/apache-airflow/stable/templates-ref.html) that come out of the box with Apache Airflow. Please refer below for the supported list of Jinja Variables, and Macros. 

**Topics**
+ [

## Supported Jinja template Variables
](#jinja-variables)
+ [

## Supported Macros
](#jinja-macros)

## Supported Jinja template Variables


 The following table lists the supported Vriabls available for Jinja templating in Amazon MWAA Serverless. 


| Variable | Type | Description | 
| --- | --- | --- | 
| \$1\$1 macros \$1\$1 |  | A reference to the macros package. See Macros below. | 
| \$1\$1 task\$1instance \$1\$1 | TaskInstance | The currently running TaskInstance. | 
| \$1\$1 ti \$1\$1 | TaskInstance | Same as \$1\$1 task\$1instance \$1\$1. | 
| \$1\$1 params \$1\$1 | dict[str, Any] | The user-defined params. This can be overridden by the mapping passed to trigger\$1dag -c if dag\$1run\$1conf\$1overrides\$1params is enabled in airflow.cfg. | 
| \$1\$1 ds \$1\$1 | str | The Dag run's logical date as YYYY-MM-DD. Same as \$1\$1 logical\$1date \$1\$1 ds \$1\$1. | 
| \$1\$1 ds\$1nodash \$1\$1 | str | Same as \$1\$1 logical\$1date \$1\$1 ds\$1nodash \$1\$1. | 
| \$1\$1 ts \$1\$1 | str | Same as \$1\$1 logical\$1date \$1\$1 ts \$1\$1. Example: 2018-01-01T00:00:00\$100:00. | 
| \$1\$1 ts\$1nodash \$1\$1 | str | Same as \$1\$1 logical\$1date \$1\$1 ts\$1nodash \$1\$1. Example: 20180101T000000. | 
| \$1\$1 macros \$1\$1 |  |  | 

## Supported Macros


 The following table lists the supported Macros that are supported in Amazon MWAA Serverless. 


| Macro | Type | 
| --- | --- | 
| macros.datetime | The standard lib's datetime.datetime | 
| macros.timedelta | The standard lib's datetime.timedelta | 
| macros.dateutil | A reference to the dateutil package | 
| macros.time | The standard lib's time | 
| macros.uuid | The standard lib's uuid | 
| macros.random | The standard lib's random.random | 
| datetime\$1diff\$1for\$1humans | datetime\$1diff\$1for\$1humans(dt, since=None): Return a human-readable/approximate difference between datetimes. When only one datetime is provided, the comparison will be based on now. | 
| ds\$1add | ds\$1add(ds, days): Add or subtract days from a YYYY-MM-DD. | 
| ds\$1format | ds\$1format(ds, input\$1format, output\$1format): Output datetime string in a given format. | 
| random | The standard lib's random.random |