Managing Multiple AWS Accounts with Org-Formation

The Challenge

We need to create an AWS Organization with multiple Organizational Units (OUs) and accounts. But that’s not all! It will need to support different types of workloads, be scaleable to dozens, scores – if not hundreds – of accounts, configuration in a centralised code repository with versioning and pull requests / merge requests, and mostly be deployable via CI/CD pipeline. The accounts have to be ready within a few days. Last but not the least, this solution has to be replicable for use into other, new AWS Organizations – so let’s keep it as simple as possible!

For this example scenario we will have the following OUs:

• Security OU
• Shared Services OU
• Development OU
• Staging OU
• Production OU
• Sandbox OU

Here is a diagram I created to help visualise the AWS Organization structure – inspired by Account structure and OUs – AWS Prescriptive Guidance:

Solution

AWS Organization Formation (org-formation) is an Infrastructure as Code (IaC) tool for AWS Organizations. It is not a product from AWS, but open-source / community, the source code is available on https://github.com/org-formation/org-formation-cli.

AWS is aware of this tool, and had a blog post about this on https://aws.amazon.com/blogs/opensource/managing-aws-organizations-using-the-open-source-org-formation-tool-part-1/.

Please note AWS Control Tower / AWS Control Tower Account Factory / AWS Control Tower Account Factory for Terraform (AFT) are not in scope for this document, perhaps they can be a topic for a future blog post!

Github was selected for code repository and Github Actions for CI/CD pipeline.

An OIDC provider has been created in the AWS account, with an IAM Role assigned to it with only permissions to make changes to this AWS Organization, the state S3 bucket (organization-formation-*) and the EC2 Describe Regions API… and a couple of others – please see below!

Prerequisites

• AWS Management account from which to create the AWS Organization (this guide on https://www.learncmd.com.au/course/view.php?id=7 can help you get started with an AWS account, you can log in as guest)
• From the AWS account above, increase the Service Quotas / AWS services / AWS Organizations – Default maximum number of accounts (default: 10); Member accounts you can concurrently create (default: 5). Note that any requests to increase Service Quotas may take up to 24 hours (or more?) to provision.
• AWS CLI and access to the new AWS Management account via the CLI
• OIDC Provider and IAM Role, mentioned above
• Github Repository
• Org Formation CLI if you wish to test / validate / deploy manually (already installed in my demo, using npm)
• Email addresses / aliases with which to register for each AWS account.

Implementation

Please note that details have been redacted on purpose: AWS Account IDs, email addresses, Organization IDs, OUs.

Create the AWS Organization

We’re just going to use All Features enabled – for details please see Enabling all features in your organization – AWS Organizations.

aws organizations create-organization --feature-set ALL

view raw01_aws_org_create_org.md hosted by GitHub

 Here is the redacted output from this command:

So far so good. Checking from the AWS console we should have a basic structure with the Management Account below the Root of the Organisation.

or you can check from the command line:

aws organizations list-roots

aws organizations list-accounts

view raw02_aws_org_list.md hosted by GitHub

Prior to adding AWS accounts to this new AWS Organization, you will need to validate / confirm an email verification request from AWS:

Now might also be a good time to check if any Service Quotas need to be increased.

Initialise Org-Formation

Run this command:

org-formation init organizations/organization.yml

view raw03_aws_org_init.md hosted with by GitHub

If the AWS Organization had not been created yet either via the AWS Console or the command-line as per the example above, it will result in an error similar to this:

This is what we should expect to see, for the command’s output as well as the resulting initial configuration file which I had chosen to be placed in organizations/organization.yml:

Add to CI/CD Pipeline, Update the Config

Perhaps test the update command from the command line prior to adding it into the pipeline: 

org-formation update organizations/organization.yml

view raw04_aws_org_update.md hosted with by GitHub

At the moment since no changes were made to the AWS Organization configuration, the command should respond that no work is to be done:

Github Actions also agrees. Please note that I had already configured the Github Actions Workflow and tested prior with an AWS IAM role for it to assume.

Let’s add a Security OU and within it, an Audit and a Log Archive account – these are created automatically if we were to use AWS Control Tower.

Save, commit, push – let’s let the pipeline do the trick.

Success! We should now have a couple more AWS accounts within a new security OU:

Let’s add OUs for Shared, Dev, Stage and Prod – with one account in each except for Dev which has 2. Here is the config for the OUs:

And the AWS accounts:

The pipeline reported success! And the accounts and new OUs had indeed been creating in AWS Organization. The Warning message may have been due to the IAM role OrganizationAccountAccessRole not having been created just yet.

Now might be a good time to test being able to test access into the new AWS Accounts from the Management Account – using the IAM Role OrganizationAccountAccessRole.

Click on your profile on the top-right corner and select Switch Role:

In the Switch Role page which follows, enter for Role: OrganizationAccountAccessRole. Display Name and Color are optional.

For the Account field you can either use the AWS Account ID or the Account Alias if you had set one, eg yetanothermayatestaccount-dev01.

Click Switch Role:

How can you tell that you are in the destination account you wanted to switch to? The Profile in the top-right corner should reflect this, it will display a different Account ID, and allow you to switch back to the Management Account.

Next Steps

The demonstration above is just the tip of the iceberg with what Organization Formation can do. Ideally it can make managing an AWS Organization easier, having the configuration saved in code, updates ran from a pipeline, and less clicking around on the AWS Console!

In a separate Organization, we were able to create 60 accounts in a Sandbox OU, by updating the configuration file. We also enabled Cloudtrail across all the Organization using Organization Tasks. There are examples and templates you can use, and customise – you may be interested in enabling GuardDuty, SSO, cross account buckets lambdas and roles, as well as Budget Alarms.

You may want to run this in your own pipeline, and not Github Actions as I had done. Organization Formation comes with its own pipeline, which uses AWS CodePipeline and AWS CodeBuild.

Lastly, you may want to also check out AWS Control Tower Account Factory for Terraform.