Skip to content

Conversation

HofiOne
Copy link
Collaborator

@HofiOne HofiOne commented Jun 10, 2024

This PR enables to use the server side encryption on the s3() destination. It is a common pattern that data stored on the S3 bucket is encrypted at the rest.

S3 supports multiple server side encryption possibilities. This is only for the aws:kms method. In theory aws:kms:dsse should also work but as it uses the same arguments but I didn't test it so I disabled that intentionally.

Check the s3api documentation for the details.

This means that the server-side-encryption() option accepts only aws:kms or an empty string, which is the default.

The kms-key() should be:

* an ID of a key

* an alias of a key, but in that case you have to add the `alias/` prefix

* an ARN of a key

For example:

destination d_s3 {
        s3(
                bucket( "log-archive-bucket" )
                object-key( "logs/syslog" )
                server-side-encryption( "aws:kms" )
                kms-key( "alias/log-archive" )
        );
};

To be able to use the aws:kms encryption the AWS Role or User has to have the following permissions on the given key:

* kms:Decrypt

* kms:Encrypt

* kms:GenerateDataKey

Check this page on why the kms:Decrypt is mandatory.

Setting server-side-encryption() to aws:kms without setting kms-key() will terminate the syslog-ng process.

Setting kms-key() without setting server-side-encryption() emits a warning and ignores the kms-key(). The server side encryption will be disabled in this case.

It also worth mentioning that kms-key() cannot be changed after the create_multipart_upload executed. I am not sure what will happen if someone modifies the key and then just reloads the syslog-ng configuration. A normal restart would work as syslog-ng finishes the multipart upload during the shutdown.

Signed-off-by: Arpad Kunszt akunszt@hiya.com

Backport: #127

akunszt added 3 commits June 10, 2024 16:50
Adding the server-side-encryption() and kms-key() options to the s3()
destination.

The server-side-encryption() supports only aws:kms at the moment.

Fixes syslog-ng#4920.

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>
With this the process should not fail if the JSON file was created with
an earlier version and the `server-side-encryption` and the `kms-key`
keys are missing.

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>
As @alltilla pointed out using an old JSON and NOT setting the `kms-key`
and/or `server-side-encryption` settings can cause an issue.

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>
@HofiOne HofiOne marked this pull request as draft June 11, 2024 09:39
Signed-off-by: Attila Szakacs <attila.szakacs@axoflow.com>
Signed-off-by: Hofi <hofione@gmail.com>
@HofiOne HofiOne force-pushed the s3-server-side-encryption branch from b67e713 to bca6f16 Compare June 11, 2024 09:45
@HofiOne HofiOne marked this pull request as ready for review June 13, 2024 08:59
@HofiOne HofiOne merged commit 8e6a17c into syslog-ng:master Jun 17, 2024
@HofiOne HofiOne deleted the s3-server-side-encryption branch June 18, 2024 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants