CDK - How to use Opensearch Domain MasterUserPassword within userdata

Question

In the Opensearch L2 construct, if you add fine grained access controls, a Secret in Secrets Manager will be created for you (accessible by the masterUserPassword).

I want to use this generated password within a CloudformationInit later on, but not sure how to.

from aws_cdk import aws_ec2 as ec2
from aws_cdk import aws_iam as iam
from aws_cdk import aws_opensearchservice as opensearch
from aws_cdk import aws_s3 as s3


class OpensearchStack(Stack):
    def __init__(
        self,
        scope: Construct,
        construct_id: str,
        **kwargs,
    ) -> None:
        super().__init__(scope, construct_id, **kwargs)

        vpc = ec2.Vpc(self, "generatorVpc", max_azs=2)
        bucket = s3.Bucket(self, "My Bucket")
        domain = opensearch.Domain(self,"OpensearchDomain",
            version=opensearch.EngineVersion.OPENSEARCH_1_3,
            vpc=vpc,
            fine_grained_access_control=opensearch.AdvancedSecurityOptions(
                master_user_name="osadmin",
            ),
        )
        instance = ec2.Instance(self, "Instance",
            vpc=vpc,
            instance_type=ec2.InstanceType.of(
                instance_class=ec2.InstanceClass.M5,
                instance_size=ec2.InstanceSize.LARGE,
            ),
            machine_image=ec2.MachineImage.latest_amazon_linux(
                generation=ec2.AmazonLinuxGeneration.AMAZON_LINUX_2,
            ),
            init=ec2.CloudFormationInit.from_elements(
                ec2.InitFile.from_string(
                            file_name="/home/ec2-user/logstash-8.4.0/config/my_conf.conf",
                            owner="ec2-user",
                            mode="00755",
                            content=f"""input {{
    s3 {{
        bucket => "{bucket.bucket_name}"
        region => "{self.region}"
    }}
}}
output {{
    opensearch {{
        hosts => ["{domain.domain_endpoint}:443"]
        user => "{domain.master_user_password.secrets_manager("What secret id do I put here?", json_field="username")}"
        password => "{domain.master_user_password.secrets_manager("What secret id do I put here?", json_field="password")}"
        ecs_compatibility => disabled
    }}
}}
""",
                )
            )
        )

Since SecretValue doesn't have a secretId property, I'm not sure how I can determine the Secret ID/Arn of the masterUserPassword.

Is there a better way to get the generated credentials inside my logstash config?

score 0 · Answer 1 · answered Jan 14 '23 at 13:32

0

The username value is easy, as you are explicitly setting it as osadmin. To get the password reference, call to_string method on the Domain's master_user_password attribute, which is a SecretValue:

domain.master_user_password.to_string()

In the synthesized template, this gets turned into a CloudFormation dynamic reference to the secret's password. The actual password is not known to the template. It will be resolved cloud-side at deploy time.

The SecretsValue.secrets_manager static method also synthesizes the same dynamic reference. However, you can't use it. The method requires the secret ID, which is not exposed if the Domain construct generates the secret for you.

answered Jan 14 '23 at 13:32

fedonev

20,327
2
25
34

using `domain.master_user_password.to_string()` returns an error saying to use [unsafeUnwrap](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.SecretValue.html#unsafewbrunwrap). When I alter the password to use `domain.master_user_password.unsafe_unwrap()`, the dynamic reference is written to the file. e.g. `password => "{{resolve:secretsmanager:arn:aws:secretsmanager:us-east-1:111111111111:secret:DomainMasterPassword-rUEVaqI9EIfz-3LZWaU:SecretString:password::}}"` – maafk Jan 15 '23 at 17:35
@maafk That is the expected behaviour if you have the `CHECK_SECRET_USAGE` [feature flag](https://github.com/aws/aws-cdk/blob/main/packages/@aws-cdk/cx-api/lib/features.ts) set. The two methods `to_string()` and `unsafe_unwrap()` are equivalent. – fedonev Jan 15 '23 at 18:34
This works, but based on this [similar question](https://stackoverflow.com/questions/53589880/dynamic-references-to-specify-secret-manager-values-in-aws-cloudformation) the values won't resolve in userdata or Cfn init – maafk Jan 15 '23 at 19:11
1

@maafk Hmmm. I am not familiar with cfn-init. I was just answering your question as written. Perhaps setting the values as env vars would cause the dynamic refs to resolve. According to the [docs](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-init.html), you can set env vars in `AWS::CloudFormation::Init`. – fedonev Jan 15 '23 at 19:38
A definite (and unsafe) solution would be to construct the Domain's Secret yourself, instead of letting the CDK produce one for you. CDK allows you to set an unsafe plaintext password value, which, being known at synth-time, you could simply inject into the `cfn-init` command. The downside is the password would be exposed in your CDK code and template. – fedonev Jan 15 '23 at 19:39

score 0 · Accepted Answer · answered Jan 15 '23 at 20:08

I ended up adding commands to the CloudFormationInit to pull the OS Credentials from Secrets Manager and did a find and replace which worked

from aws_cdk import aws_ec2 as ec2
from aws_cdk import aws_opensearchservice as opensearch
from aws_cdk import aws_s3 as s3
from aws_cdk import aws_secretsmanager as secretsmanager
from aws_cdk import Stack
from constructs import Construct


class OpensearchStack(Stack):
    def __init__(
        self,
        scope: Construct,
        construct_id: str,
        **kwargs,
    ) -> None:
        super().__init__(scope, construct_id, **kwargs)

        vpc = ec2.Vpc(self, "generatorVpc", max_azs=2)
        bucket = s3.Bucket(self, "My Bucket")
        domain = opensearch.Domain(self,"OpensearchDomain",
            version=opensearch.EngineVersion.OPENSEARCH_1_3,
            vpc=vpc,
            fine_grained_access_control=opensearch.AdvancedSecurityOptions(
                master_user_name="osadmin",
            ),
        )
        # Get the domain secret
        domain_secret: secretsmanager.Secret = domain.node.find_child("MasterUser")
        instance = ec2.Instance(self, "Instance",
            vpc=vpc,
            instance_type=ec2.InstanceType.of(
                instance_class=ec2.InstanceClass.M5,
                instance_size=ec2.InstanceSize.LARGE,
            ),
            machine_image=ec2.MachineImage.latest_amazon_linux(
                generation=ec2.AmazonLinuxGeneration.AMAZON_LINUX_2,
            ),
            init=ec2.CloudFormationInit.from_elements(
                ec2.InitFile.from_string(
                            file_name="/home/ec2-user/logstash-8.4.0/config/my_conf.conf",
                            owner="ec2-user",
                            mode="00755",
                            content=f"""input {{
    s3 {{
        bucket => "{bucket.bucket_name}"
        region => "{self.region}"
    }}
}}
output {{
    opensearch {{
        hosts => ["{domain.domain_endpoint}:443"]
        user => "REPLACE_WITH_USERNAME"
        password => "REPLACE_WITH_PASSWORD"
        ecs_compatibility => disabled
    }}
}}
""",
                ),
                ec2.InitPackage.yum("jq"),  # install jq
                ec2.InitCommand.shell_command(
                    shell_command=(
                        f"aws configure set region {self.region} && "
                        # save secret value to variable
                        f"OS_SECRET=$(aws secretsmanager get-secret-value --secret-id {domain_secret.secret_arn} "
                        "--query SecretString) && "
                        # Pull values from json string
                        "OS_USER=$(echo $OS_SECRET | jq -r '. | fromjson | .username') && "
                        "OS_PASS=$(echo $OS_SECRET | jq -r '. | fromjson | .password') && "
                        # Find and replace
                        "sed -i \"s/REPLACE_WITH_USERNAME/$OS_USER/g\" /home/ec2-user/logstash-8.4.0/config/my_conf.conf && "
                        "sed -i \"s/REPLACE_WITH_PASSWORD/$OS_PASS/g\" /home/ec2-user/logstash-8.4.0/config/my_conf.conf"
                    ),
                ),
            )
        )
        # Don't forget to grant the instance read access to the secret
        domain_secret.grant_read(instance.role)

CDK - How to use Opensearch Domain MasterUserPassword within userdata

2 Answers2