Service Specific Configuration

There are certain services that require configuration for system wide operation of uploading/downloading files, events, and more.

Amazon S3 Event (Bucket) Notifications

To allow for event notifications for the S3 service, a service key must be configured for S3. Unlike the other Service Keys, this is not used for auth, but is used for creation and management of infrastructure needed to collect events from S3, such as regional SNS topics and SQS queues. The key can be added with the following command:

ke_manage_service_keys --action add --service s3 \
    --key "aws_access_key_id" --secret "aws_secret_access_key"

The AWS access keys used here should not be reused between different Kloudless Enterprise deployments or clusters as this will result in conflicts when trying to fetch events. The IAM credentials used must have the following permissions:

iam:GetUser
sqs:* (for ks3-eventq-* SQS queues)
sns:* (for ks3-event-* SNS topics)

Example Policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1444682999999",
            "Effect": "Allow",
            "Action": [
                "iam:GetUser"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Stmt1444682767000",
            "Effect": "Allow",
            "Action": [
                "sqs:*"
            ],
            "Resource": [
                "arn:aws:sqs:*:*:ks3-eventq-*"
            ]
        },
        {
            "Sid": "Stmt1444682948000",
            "Effect": "Allow",
            "Action": [
                "sns:*"
            ],
            "Resource": [
                "arn:aws:sns:*:*:ks3-event-*"
            ]
        }
    ]
}

Kloudless subscribes to bucket notifications for an S3 account, which will be sent to your AWS SNS topic. Kloudless creates an SQS queue for each region S3 buckets are present in. Each bucket will be configured to send events to the SNS topic. Kloudless subscribes the SQS queue to the appropriate topic. Each queue’s events will be retrieved by Kloudless Enterprise, processed, and stored for access through the API. Any configured Webhooks will be notified once the events are available.

Troubleshooting

If Events aren’t available at the events endpoint for an S3 account, grep the logs for the ID of the connected S3 account for further information. e.g.: ke_logs | grep 12345 on each node. This may yield valuable information such as error messages that occur while connecting to S3. If it is still unclear where the issue lies, the steps below may assist with identifying the cause.

Be sure to disconnect and reconnect the account if it was connected prior to S3 keys being added via ke_manage_service_keys. Confirm the following:

  • SNS topics for the connected accounts are visible in SNS for the region you’re modifying the bucket in.
    • For example, if data is being modified in an S3 bucket in ap-southeast-2 (https://console.aws.amazon.com/sns/home?region=ap-southeast-2), should show an SNS topic named ks3-event-topic-User_AWS_Account_Number. If the topics are not present, the AWS keys configured may not have the permission to create SNS topics in your account.
  • SQS queues for the connected accounts are visible in SQS for the region you're modifying the bucket in.
    • For example, if data is being modified in an S3 bucket in ap-southeast-2 (https://console.aws.amazon.com/sns/home?region=ap-southeast-2), should show an SQS queue named ks3-eventq-Kloudless_Account_ID-AWS_Key_ID. If the queues are not present, the AWS keys configured may not have the permission to create SQS queues in your account.
  • A subscription from the SQS queue to the SNS topic. This is found under attributes of the SNS topic.
  • Additionally, verify the IAM policy in the previous section is in place for the AWS Keys used.

If the queues are present, check if there are messages available on them. If there are queues with no messages, there could be an issue with configuring the S3 bucket's notifications. Click on the appropriate queue for the Kloudless Account in question and verify that the bucket you are modifying data in is listed under Permissions -> Conditions as one of the aws:SourceArn fields. If you have access to the S3 bucket, click on it and check its Properties -> Events to verify that an Event Notification entry exists. Click on the 'Edit' pencil icon next to it and verify that the Events list contains ObjectCreated (All) and ObjectRemoved (All), and the notification points to the right SNS topic you identified previously for that Kloudless Account.

Feel free to reach out to us at support@kloudless.com if you continue to require any assistance.