Post-deployment: Custom (BYOB) and Control Tower audit log collection
Configure custom and Control Tower audit log collection after AWS onboarding.
If you selected Custom (BYOB) or Custom Control Tower audit log collection, link your existing CloudTrail notification pipeline to the SQS queue created by the template. The SQS queue receives messages by subscribing to the SNS topic, which acts as the intermediary. You can route these notifications using either of the following methods:
CloudTrail → SNS: (Recommended) If your CloudTrail trail is already configured to publish delivery notifications to an SNS topic, the Cortex-created SQS queue subscribes to that topic automatically via the SNS subscription resource in the template. No additional S3 configuration is needed.
S3 → SNS: If your S3 bucket is not yet connected to an SNS topic, configure an S3 event notification on the bucket to send s3:ObjectCreated:* events to the SNS topic named in the CloudTrailSnsArn parameter. The SNS topic then fans out to the SQS queue.
The CloudFormation stack creates the SQS queue in your account. You must manually configure your S3 bucket to send event notifications to it.
In the AWS CloudFormation console, open the stack and select the Outputs tab.
Note the
sqs_urloutput value.Derive the SQS queue ARN from the URL (format:
arn:aws:sqs:<region>:<account-id>:<queue-name>), or retrieve it from the AWS SQS console.Confirm which notification route applies to your setup:
If your CloudTrail trail already publishes to the SNS topic you provided as CloudTrailSnsArn: No additional configuration is needed. The template's SNS subscription connects the topic to the SQS queue automatically.
If your S3 bucket does not yet send notifications to the SNS topic: In the AWS Management Console, navigate to your S3 bucket. Under Properties → Event notifications, add a notification with event type
s3:ObjectCreated:*and destination set to the SNS topic ARN you provided asCloudTrailSnsArn. Save the notification configuration.
The Control Tower BYOB CloudFormation stack deploys the SQS queue and SNS subscription automatically via CloudFormation StackSets into the account where your SNS topic resides. No manual S3 event notification configuration is required. The SNS-to-SQS subscription is created by the StackSet.
After the stack completes, you can verify the following in the AWS Management Console:
In the SNS topic account, confirm that the SQS queue named
<SqsQueueName>(the value you entered in the wizard) exists and has an active subscription to your CloudTrail SNS topic.In the logging account, confirm that the IAM role named
<CloudTrailReadRoleName>(the value you entered in the wizard) exists and has the correct trust policy and S3 read permissions.In Cortex Cloud, verify that the cloud instance transitions from Pending to Connected.
Important
If you are using Control Tower for audit log collection and you choose to encrypt your logs with a KMS key, you must grant cross-account KMS key access.
The Terraform template creates the SQS queue in your account. You must manually configure your S3 bucket to send event notifications to it.
In the
terraform applyoutput, note thesqs_urlvalue.Derive the SQS queue ARN from the URL (format:
arn:aws:sqs:<region>:<account-id>:<queue-name>), or retrieve it from the AWS SQS console.Confirm which notification route applies to your setup:
If your CloudTrail trail already publishes to the SNS topic you provided as
cloud_trail_sns_arn: No additional configuration is needed.If your S3 bucket does not yet send notifications to the SNS topic: In the AWS Management Console, navigate to your S3 bucket. Under Properties → Event notifications, add a notification with event type
s3:ObjectCreated:*and destination set to the SNS topic ARN you provided ascloud_trail_sns_arn. Save the notification configuration.
Troubleshooting custom audit log collection
Use this section to diagnose issues with custom (BYOB) audit log collection after deploying the authentication template. The most common causes of failure are:
S3 event notifications not configured or misconfigured: For Terraform and standard CloudFormation deployments, you must ensure your CloudTrail trail publishes delivery notifications to the SNS topic you provided, or configure your S3 bucket to send
s3:ObjectCreated:*events to that SNS topic. The SNS topic fans out to the SQS queue created by the template. If neither route is configured, or the wrong SNS topic ARN is used, logs will not reach Cortex.Cross-region resources: The S3 bucket, SNS topic, SQS queue, and authentication template deployment must all be in the same AWS region. Cross-region configurations are not supported.
KMS key policy not updated: (Only relevant for Custom Control Tower audit log collection) If your S3 bucket is encrypted with a customer-managed KMS key, you must manually update the KMS key policy to allow the Cortex log-collector IAM role to call
kms:Decrypt. This cannot be done automatically by the template.Instance remains in Pending state: This indicates the template deployed successfully but the notification to Cortex Cloud did not complete. You can connect the instance manually from the Cortex Cloud pending instances panel. See Manually connect a cloud instance for more details.
Select the tab for your deployment method for specific troubleshooting steps.
Stack creation fails with IAM permission errors
AWS credentials lack required CloudFormation or IAM permissions
Ensure your AWS credentials have permissions to create CloudFormation stacks, IAM roles, SQS queues, and the resources required by your selected capabilities.
CloudTrailSnsArn parameter rejected
SNS ARN format is incorrect
The ARN must match arn:(aws|aws-us-gov):sns:<region>:<account-id>:<topic-name>.
CloudTrailKmsArn parameter rejected
KMS ARN format is incorrect
The ARN must match arn:(aws|aws-us-gov):kms:<region>:<account-id>:key/<uuid>. Leave the field empty if no KMS key is used.
Instance remains in Pending state after stack creation
Lambda notification to Cortex Cloud failed
Check the Lambda function logs in CloudWatch for errors. If the notification failed, connect the instance manually: select the pending instance, click Connect manually, and provide the CortexPlatformRole ARN and External ID shown in the CloudFormation stack Outputs tab.
Logs not appearing in Cortex Cloud after stack creation
Notification pipeline not connected: CloudTrail is not publishing to the SNS topic, or the S3 bucket is not sending event notifications to the SNS topic
Confirm that either your CloudTrail trail is configured to publish delivery notifications to the SNS topic ARN you provided as CloudTrailSnsArn, or your S3 bucket has an event notification configured to send s3:ObjectCreated:* events to that SNS topic. The SNS topic fans out to the SQS queue. Ensure the S3 bucket, SNS topic, and CloudFormation stack are all in the same AWS region.
KMS decryption errors in Cortex Cloud
KMS key policy does not allow the cortex-logs-ingestion-access-* to call kms:Decrypt
Update the KMS key policy to allow kms:Decrypt for the CortexLogsReadRole-* IAM role created by the stack. This must be done manually after the stack is deployed.
StackSet operation fails with OPERATION_NOT_FOUND or permission errors
AWS Organizations service-managed StackSets not enabled, or the management account does not have trusted access enabled for CloudFormation
In AWS Organizations, enable trusted access for AWS CloudFormation StackSets. See the AWS documentation.
SQS queue not created in the SNS topic account
SnsTopicOuId does not contain the account where the SNS topic resides
Verify that the OU ID entered for SnsTopicOuId is the OU that directly contains the SNS topic account. The StackSet uses AccountFilterType: INTERSECTION and will not deploy if the account is not in the specified OU.
IAM role not created in the logging account
LoggingAccountOuId does not contain the logging account
Verify that the OU ID entered for LoggingAccountOuId is the OU that directly contains the logging account where the S3 bucket resides.
LoggingAccountId parameter rejected
Account ID format is incorrect
The value must be a 12-digit AWS account ID with no hyphens or spaces.
LoggingAccountOuId or SnsTopicOuId parameter rejected
OU ID format is incorrect
The value must match the format ou-<root-id>-<ou-id> (for example, ou-ab12-cd34ef56).
SqsQueueName parameter rejected
Queue name contains invalid characters or exceeds 80 characters
Queue names may only contain alphanumeric characters, hyphens, and underscores, and must be 80 characters or fewer.
Logs not appearing in Cortex Cloud after stack creation
SNS-to-SQS subscription not active, or S3 event notification not forwarding to SNS
In the SNS topic account, verify the SQS queue exists and has an active subscription to the CloudTrail SNS topic. Confirm that the S3 bucket is configured to send s3:ObjectCreated:* event notifications to the SNS topic (not directly to SQS — the SNS topic fans out to the SQS queue).
KMS decryption errors in Cortex Cloud
KMS key policy does not allow the CloudTrailReadRoleName role in the logging account to call kms:Decrypt
Update the KMS key policy in the logging account to allow kms:Decrypt for the IAM role named <CloudTrailReadRoleName>. This must be done manually after the StackSet deploys the role.
terraform init fails with provider download errors
No outbound internet access to the Terraform registry
Ensure outbound HTTPS access to registry.terraform.io and releases.hashicorp.com is allowed from your local machine.
terraform apply fails with IAM permission errors
AWS credentials lack required permissions
Ensure your AWS credentials have permissions to create IAM roles, policies, and the resources required by your selected capabilities.
cloud_trail_sns_arn validation error
SNS ARN format is incorrect
The ARN must match arn:(aws|aws-us-gov):sns:<region>:<account-id>:<topic-name>.
cloud_trail_kms_arn validation error
KMS ARN format is incorrect
The ARN must match arn:(aws|aws-us-gov):kms:<region>:<account-id>:key/<uuid>. Leave the field empty if no KMS key is used.
Tag validation errors during apply
Custom tags contain invalid characters
AWS tag keys and values may only contain Unicode letters, digits, whitespace, and the following symbols: _ . : / = + - @. Remove any other characters from your custom tags in the onboarding wizard.
Instance remains in Pending state after apply
Notification to Cortex Cloud failed
The notification requires outbound HTTPS access to *.storage.googleapis.com. If the notification failed, connect the instance manually: select the pending instance, click Connect manually, and provide the CortexPlatformRole ARN and External ID shown in the Terraform output.
Logs not appearing in Cortex Cloud after S3 event notification is configured
SQS queue ARN entered incorrectly in S3 event notification
Verify the ARN in the S3 bucket event notification matches the sqs_url output from terraform apply. Ensure the S3 bucket, SNS topic, and SQS queue are all in the same AWS region.
Last updated
Was this helpful?