How to Secure Octopus Deploy Variables in AWS

Octopus Deploy is a great tool for managing deployments. One of its most powerful features is how it organizes variables. The ability to scope variable values to environments, specific targets, and channels is extremely powerful. Not only is it powerful, but it’s done in a clean way where you don’t need to recreate the variable to apply it to a new environment, just add a new value. Octopus also allows you to use the variables to replace values in config and settings files, as well as in random tokenized files. In an earlier post, Colin Pear talked about the different way variables can be managed. Even though Octopus Deploy employs encryption on sensitive variables, sometimes requirements dictate more stringent controls over the values. To satisfy this requirement we can use AWS Secrets Manager.

Setting Up the Account

AWS Secrets Manager allows users to cycle encryption keys and segregate variables into separate accounts. By storing variables in AWS, we can still maintain a lot of the features that Octopus Deploy brings. We’ll start by giving our account an API key used to access our AWS account on our behalf.

Clicking on my user and adding an access key under the security credentials I’m given a new access key ID and secret access key. This secret will only be shown once so i recommend storing it in your secrets manager and labeled to the use who it is associated with.

Use this access key ID and secret to create a new AWS account under the infrastructure -> accounts tabs.

The name is how Octopus Deploy will reference the account in itself.

Make sure to copy the access key and secret key we retrieved from earlier. Once finished, we can click on save and test to ensure the account is configured correctly. Once we’re done here, lets move over to the project where we need to have a reference to this AWS account. We need to add the account as a variable to the project by changing the type to AWS Account.

Creating the Secret

From here we can move back to AWS Secrets Manger to create a new secret. Select the other type of secret and start filling out the key, value pair list.

On the next screen set a name for the secret. If you plan on doing multiple projects or environments, it might be a good idea to have them in a nested tree notation.

Once we set the name, we leave the rest as default. Once we store the secrets, we can now retrieve it from within our process.

Retrieving Our Value for Octopus Deploy

We need to save the secret name as a variable in our Octopus Deploy process. In this example I call the variable $secretName. We’ll want to retrieve our variables as our first step in our deploy process so that all other steps have access to this value. To accomplish this, we’re going to take advantage of the AWS CLI using the “Run an AWS CLI script” built in step template. The script content is relatively simple.

$secretValue = ((aws secretsmanager get-secret-value --secret-id $secretName) | ConvertFrom-Json).SecretString

The contents of the SecretString property is a JSON list. Now that we have the secret’s content, we need to expose it to the following steps, referencing Octopus Deploy docs. We do this with the following powershell commands.

Set-OctopusVariable -name "awsVariables" -value $secretValue

Assuming we name the step “retrieveAwsSecretsVariables”, the following steps can access the variables in the like this.

$allVariables = $OctopusParameters["Octopus.Action[retrieveAwsSecretsVariables].Output.awsVariables"] | ConvertFrom-Json

Because the variables are pulled out of AWS as a JSON string we want to convert them so they are easy to use by variable name. Now this is nice to use in a script, but what if we want to use one of the variables in a step template binding like we do traditional Octopus Deploy variables? That’s easy enough. In the first step were we set the output, lets just our specific variable its own variable.

$parsedVars = $secretValue | ConvertFrom-Json
Set-OctopusVariable -name "domain" -Value $parsedVars.Domain

It is important to note that for any variables that are truly sensitive, the flag -sensitive should be used with the Set-OctopusVariable command

This’ll output our domain variable under the name “domain”. Now we’ll add a shortcut handler in our Octopus Deploy project variables like so.

We can now use this just like we would as normal variable interpolation.

Now we can use our variables like we normally would but with the added benefits that AWS Secrets Manager bring.

One thing to note when using AWS Secrets Manager. Scoping these variables to environments or channels makes little sense except for the ones whose values are stored in Octopus Deploy. One example of this would be to use the secretKey value scoped to environment so that different environments can have different values. I recommend keeping the number of variables in AWS Secrets Manager to a minimum to prevent drift in the different environments.

Originally published November 23, 2021. Information refreshed March 20, 2022.