Combine Amazon MWAA with Microsoft Entra ID utilizing SAML authentication

Amazon Managed Workflows for Apache Airflow (Amazon MWAA) gives a completely managed resolution for orchestrating and automating complicated workflows within the cloud. Amazon MWAA affords two community entry modes for accessing the Apache Airflow net UI in your environments: private and non-private. Prospects typically deploy Amazon MWAA in non-public mode and need to use current login authentication mechanisms and single sign-on (SSO) options to have seamless integration with the company Energetic Listing (AD). Additionally, the end-users don’t must log in to the AWS Administration Console to entry the Airflow UI.

On this publish, we illustrate configure an Amazon MWAA atmosphere deployed in non-public community entry mode with buyer managed VPC endpoints and authenticate customers utilizing SAML federated id utilizing Microsoft Entra ID and Utility Load Balancer (ALB). Customers can seamlessly log in to the Airflow UI with their company credentials and entry the DAGs. This resolution will be modified for Amazon MWAA public community entry mode as properly.

Answer overview

The architectural parts concerned in authenticating the Amazon MWAA atmosphere utilizing SAML SSO are depicted within the following diagram. The infrastructure parts embrace two public subnets and three non-public subnets. The general public subnets are required for the internet-facing ALB. Two non-public subnets are used to arrange the Amazon MWAA atmosphere, and the third non-public subnet is used to host the AWS Lambda authorizer operate. This subnet may have a NAT gateway connected to it, as a result of the operate must confirm the signer to substantiate the JWT header has the anticipated LoadBalancer ARN.

The workflow consists of the next steps:

1. For SAML configuration, Microsoft Entra ID serves because the id supplier (IdP).
2. Amazon Cognito serves because the service supplier (SP).
3. ALB has built-in assist for Amazon Cognito and authenticates requests.
4. Publish-authentication, ALB forwards the requests to the Lambda authorizer operate. The Lambda operate decodes the person’s JWT token and validates whether or not the person’s AD group is mapped to the related AWS Identification and Entry Administration (IAM) function.
5. If legitimate, the operate creates an online login token and redirects to the Amazon MWAA atmosphere for profitable login.

The next are the high-level steps to deploy the answer:

1. Create an Amazon Easy Storage Service (Amazon S3) bucket for artifacts.
2. Create an SSL certificates and add it to AWS Certificates Supervisor (ACM).
3. Deploy the Amazon MWAA infrastructure stack utilizing AWS CloudFormation.
4. Configure Microsoft Entra ID providers and combine the Amazon Cognito person pool.
5. Deploy the ALB CloudFormation stack.
6. Log in to Amazon MWAA utilizing Microsoft Entra ID person credentials.

Conditions

Earlier than you get began, ensure you have the next conditions:

• An AWS account
• Acceptable IAM permissions to deploy AWS CloudFormation stack assets
• A Microsoft Azure account is required for creating the Microsoft Entra ID app (IdP config) and Microsoft Entra ID P2.
• A public certificates for the ALB within the AWS Area the place the infrastructure is being deployed and a customized area identify related to the certificates.

Create an S3 bucket

On this step, we create an S3 bucket to retailer your Airflow DAGs, customized plugins in a plugins.zip file, and Python dependencies in a necessities.txt file. This bucket is utilized by the Amazon MWAA atmosphere to fetch DAGs and dependency information.

1. On the Amazon S3 console, select the Area the place you need to create a bucket.
2. Within the navigation pane, select Buckets.
3. Select Create bucket.
4. For Bucket sort, choose Normal goal.
5. For Bucket identify, enter a reputation on your bucket (for this publish, mwaa-sso-blog-<your-aws-account-number>).
6. Select Create bucket. 

   

7. Navigate to the bucket and select Create folder.
8. For Folder identify, enter a reputation (for this publish, we identify the folder dags).
9. Select Create folder.

   

Import certificates into ACM

ACM is built-in with Elastic Load Balancing (ALB). On this step,  you may request a public certificates utilizing ACM or import a certificates into ACM. To import group certificates linked to a customized DNS into ACM, you should present the certificates and its non-public key. To import a certificates signed by a non-AWS Certificates Authority (CA), you should additionally embrace the non-public and public keys of the certificates.

1. On the ACM console, select Import certificates within the navigation pane.
2. For Certificates physique, enter the contents of the cert.pem file.
3. For Certificates non-public key, enter the contents of the privatekey.pem file.
4. Select Subsequent.

   

5. Select Evaluate and import.
6. Evaluate the metadata about your certificates and select Import.

After the import is profitable, the standing of the imported certificates will present as Issued.

Create the Azure AD service, customers, teams, and enterprise utility

For the SSO integration with Azure, an enterprise utility is required, which acts because the IdP for the SAML movement. We add related customers and teams to the applying and configure the SP (Amazon Cognito) particulars.

Airflow comes with 5 default roles: Public, Admin, Op, Person, Viewer. On this publish, we deal with three: Admin , Person and Viewer. We create three roles and three corresponding customers and assign memberships appropriately.

1. Log in to the Azure portal.
2. Navigate to Enterprise purposes and select New utility.

   

3. Enter a reputation on your utility (for instance, mwaa-environment) and select Create.

   

   
   Now you can view the main points of your utility.
   
   Now you create two teams.

4. Within the search bar, seek for Microsoft Entra ID.
   
   
5. On the Add menu, select Group.
   
   
6. For Group sort, select a sort (for this publish, Safety).
7. Enter a gaggle identify (for instance, airflow-admins) and outline.
8. Select Create.
   
   
   
9. Repeat these steps to create two extra teams, named airflow-users and airflow-viewers.
10. Notice the item IDs for every group (these are required in a later step).
    
    
    Subsequent, you create customers.
11. On the Overview web page, on the Add menu, select Person and Create new person.
    
12. Enter a reputation on your person (for instance, mwaa-user), show identify, and password.
13. Select Evaluate + create.
    
    
    
14. Repeat these steps to create a person referred to as mwaa-admin.
15. In your airflow-users group particulars web page, select Members within the navigation pane.
16. Select Add members.
17. Seek for and choose the customers you created and select Choose.
    
    
    

18. Repeat these steps so as to add the customers to every group.

19. Navigate to your utility and select Assign customers and teams.
    
    

20. Select Add person/group.
    
    

21. Seek for and choose the teams you created, then select Choose.
    
    

 

Deploy the Amazon MWAA atmosphere stack

For this resolution, we offer two CloudFormation templates that arrange the providers illustrated within the structure. Deploying the CloudFormation stacks in your account incurs AWS utilization prices.

The primary CloudFormation stack creates the next assets:

• A VPC with two public subnets and three non-public subnets and related route tables, NAT gateway, web gateway, and safety group
• VPC endpoints required for the Amazon MWAA atmosphere
• An Amazon Cognito person pool and person pool area
• Utility Load Balancer

Deploy the stack by finishing the next steps:

1. Select Launch Stack to launch the CloudFormation stack.
   

2. For Stack identify, enter a reputation (for instance, sso-blog-mwaa-infra-stack).

3.  Enter the next parameters:

   1. For MWAAEnvironmentName, enter the atmosphere identify.

   2. For MwaaS3Bucket, enter the S3 artifacts bucket you created.

   3. For VpcCIDR, enter the specify IP vary (CIDR notation) for this VPC.

   4. For PrivateSubnet1CIDR, enter the IP vary (CIDR notation) for the non-public subnet within the first Availability Zone.

   5.  For PrivateSubnet2CIDR, enter the IP vary (CIDR notation) for the non-public subnet within the second Availability Zone.

   6. For PrivateSubnet3CIDR, enter the IP vary (CIDR notation) for the non-public subnet within the third Availability Zone.

   7. For PublicSubnet1CIDR, enter the IP vary (CIDR notation) for the general public subnet within the first Availability Zone.

   8. For PublicSubnet2CIDR, enter the IP vary (CIDR notation) for the general public subnet within the second Availability Zone.

4. Select Subsequent
   
   
5. Evaluate the template and select Create stack.

After the stack is deployed efficiently, you may view the assets on the stack’s Outputs tab on the AWS CloudFormation console. Notice the ALB URL, Amazon Cognito person pool ID, and area.

 

Combine the Amazon MWAA utility with the Azure enterprise utility

Subsequent, you configure the SAML configuration within the enterprise utility by including the SP particulars and redirect URLs (on this case, the Amazon Cognito particulars and ALB URL).

1. Within the Azure portal, navigate to your atmosphere.
2. Select Arrange single signal on.
   
3. For Identifier, enter urn:amazon:cognito:sp:<your cognito user_id>.
4. For Reply URL, enter https://<Your person pool area>/saml2/idpresponse.
5. For Signal on URL, enter https://<Your utility load balancer DNS>.
   
6. Within the Attributes & Claims part, select Add a gaggle declare.
7. Choose Safety teams.
8. For Supply attribute, select Group ID.
9. Select Save.
   
10. Notice the values for App Federation Metadata Url and Login URL.

    

Deploy the ALB stack

When the SAML configuration is full on the Azure finish, the IdP particulars should be configured in Amazon Cognito. When customers entry the ALB URL, they are going to be authenticated towards the company id utilizing SAML by means of Amazon Cognito. After they’re authenticated, they’re redirected to the Lambda operate for authorization towards the group they belong to. The person’s group is then validated towards matching IAM function. If it’s legitimate, the Lambda operate provides the net login token to the URL, and the person will achieve entry to the Amazon MWAA atmosphere.

This CloudFormation stack creates the next assets:

• Two goal teams: the Lambda goal group and Amazon MWAA goal group
• Listener guidelines for the ALB to redirect URL requests to the related goal teams
• A person pool shopper and SAML supplier (Azure) particulars to the Amazon Cognito person pool
• IAM roles for Admin, Person, and Viewer personas required for Airflow
• The Lambda authorizer operate to validate the JWT token and map Azure teams to IAM roles for acceptable Airflow UI entry

Deploy the stack by finishing the next steps:

1. Select Launch Stack to launch the CloudFormation stack:
   

2. For Stack identify, enter a reputation (for instance, sso-blog-mwaa-alb-stack).

3. Enter the next parameters:

   1. For MWAAEnvironmentName, enter your atmosphere identify.

   2. For ALBCertificateArn, enter the certificates ARN required for ALB. 

   3. For AzureAdminGroupID, enter the group identify for the Azure Admin persona.

   4. For AzureUserGroupID, enter the group identify for the Azure Person persona.

   5. For AzureViewerGroupID, enter the group identify for the Azure Viewer persona.

   6. For EntraIDLoginURL, enter the Azure IdP URI.

   7. For AppFederationMetadataURL, enter the URL of the metadata file for the SAML supplier. 

4. Select Subsequent.
   
   
5. Evaluate the template and select Create stack.

Take a look at the answer

Now that the SAML configuration and related AWS providers are created, it’s time to entry the Amazon MWAA atmosphere.

1. Open your net browser and enter the ALB DNS identify.
   The SP initiates the sign-in request course of and the browser redirects you to the Microsoft login web page for credentials.
2. Enter the Admin person credentials.
   
   The SAML request sign-in course of completes and the SAML response is redirected to the Amazon Cognito person pool connected to the ALB.
   
   The listener guidelines will validate the question URL and cross the requests to the Lambda authorizer to validate the JWT and assign the suitable group (Azure) to function (AWS) mapping.
   
   
   
3. Repeat the steps to log in with Person and Viewer credentials and observe the variations in entry.

Clear up

If you’re carried out experimenting with this resolution, it’s important to wash up your assets to keep away from incurring AWS prices.

1. On the AWS CloudFormation console, delete the stacks you created.
2. Take away the SSM parameters and personal webserver and database VPC endpoints (created by the Lambda occasions operate):

   aws ssm delete-parameters --names "MyFirstParameter" "MySecondParameter"
   aws ec2 delete-vpc-endpoints --vpc-endpoint-ids "Endpoint1" "Endpoint2"

3. Delete the customers, teams, and enterprise utility within the Azure atmosphere.