Provisioning tfstate backend roles in refarch

[rauthur]

Spinning my wheels a bit on this one so figured I'd ask.

In the baseline steps there is a note:

The IAM User for SuperAdmin will be granted access to Terraform State by principal ARN. This ARN is passed to the tfstate-backend stack catalog under allowed_principal_arns. Verify that this ARN is correct now. You may need to update the root account ID.

And possibly related:

With the addition of support for dynamic Terraform roles, our baseline cold start refarch layer now depends on/requires that we have aws-teams and aws-team-roles stacks configured. This is because account-map uses those stacks to determine which IAM role to assume when performing Terraform in the account, and almost every other component uses account-map (indirectly) to chose the role to assume.

However, none of the steps in the baseline seem to provision the roles for accessing the tfstate. Tracing through it looks like the -var=access_roles_enabled=false prevents these roles from being created in the baseline tfstate backend workflow. The full deploy/tfstate workflow isn't run until later in the identity phase.

The result is that the atmos workflow deploy/accounts -f accounts workflow cannot run and does not create the account-map due to an error:

Error: error configuring S3 Backend: IAM Role (arn:aws:iam::1234567890:role/xxx-core-gbl-root-tfstate) cannot be assumed.

The role doesn't exist so the error is clear, however, passing access_roles_enabled=true doesn't work since the account-map needs to be created.

[dudymas]

We'll likely need to go over this during a workshop call, but just to be clear:

• which commands have you run so far?
• did you try re-running atmos workflow init/tfstate -f baseline followed by atmos workflow deploy/organization -f accounts and atmos workflow deploy/accounts -f accounts ?

[rauthur]

Yep, tried all of those things. First two completed without an issue. Since everything in the init/tfstate is -var=access_roles_enabled=false it seems like there isn't anything that creates the tfstate access roles for the SuperAdmin to assume before the actual deploy/tfstate workflow in the identity phase.

[dudymas]

Gotcha. I'll discuss internally and see if we might have a bug after doing all the updates this time around. Thanks for the patience!

[dudymas]

After looking through it, we probably should pair up on next week's workshop calls. We have one on Wednesday and another on Thursday. This shouldn't be hard to fix, but since there's so many moving pieces it's typically easier than sharing reams of logs and CLI commands.

[rauthur]

Got it. I'm traveling next week on Wednesday and Thursday for some in-person meetings unfortunately. No worries, I'll dive a bit deeper on it.

[rauthur]

I was able to move past this by temporarily manually adding the role and creating the necessary S3/Dynamo policies in the root account along with the ability for the SuperAdmin to assume the role.

After this, I ran into a second issue in that the component metadata for several accounts was empty so I add a condition in the dynamic-roles.tf of account-map to discard empty objects loaded by the utils helper.