How to Use Fargate Containers as Octopus Deploy Workers 

We would like to use Fargate to manage Octopus Deploy workers as it provides some great benefits. Fargate removes the burden of managing infrastructure, as well as providing an easily scalable worker pool. We would also like to use something similar to an EC2 instance role when executing AWS CLI commands. This way AWS credentials do not need to be stored on the Octopus server, and different credentials can be applied to each worker pool. 

Fortunately, Octopus Deploy has built-in steps that can leverage AWS resources, such as Run an AWS CLI Script.

Using the Run an AWS CLI Script will prompt the Octopus Deploy server to supply AWS credentials so that it has the authority to execute a script. This can be done either by providing an AWS account variable, which we’re trying to avoid or by selecting the Execute using the AWS service role for an EC2 instance option. Using an EC2 service role is an excellent choice as no credentials need to be stored on the Octopus server, and worker pools in different accounts can have different IAM roles assigned to them. 

This is exactly what we want, except that it only works for EC2 instances.

When Execute using the AWS service role for an EC2 instance is selected, Octopus retrieves the IAM credentials from the instance metadata URI. Attempting this with a worker that’s on a Fargate container will result in the following error: 
System.Exception: AWS-LOGIN-ERROR-0003: Failed to access the role information under http://169.254.169.254/latest/meta-data/iam/security-credentials/, or failed to parse the response. This may be because the instance does not have a role assigned to it. 

Fargate containers store their metadata differently than EC2 instances. Instead of the instance metadata URI that EC2 instances use, Fargate containers use the task metadata endpoint. Currently, Octopus doesn’t have an option that will tell it to access the task metadata endpoint. 

This problem is resolved by simply using the built-in Run a Script step instead of Run an AWS CLI Script.

This works because Fargate containers know how to retrieve their IAM credentials when accessing the AWS API.

Simply creating a  Run a Script step, telling it to use the Fargate Worker Pool and then providing the AWS CLI script will run the script with the ECS container permissions. Octopus does not need to retrieve the credentials.

Octopus Deploy provides many useful Step Templates that can simplify complicated deployments. In this case the Run an AWS CLI Script template is too clever, and gets in the way of authenticating with Fargate containers. Fortunately the Step Templates can be ignored, and all of the work done ourselves. Which, is actually less work in this case since we don’t need to provide any credentials at all!

We hope you found this post informational. If there are specific challenges you and your team face, the architects and engineers here at Clear Measure would be happy to work with you. Reach out to us now to find out all the ways we can help you!