Adding sse and sse kms key id to the s3 destination plugin

Hey, if I wanted to add SSE and SSE KMS Key ID to the S3 Destination Plugin, what would be the proper process?

Hi @comic-pup - Currently, that is not possible. Would you mind opening up a feature request with the details of what you are looking to do? You can do so here.

If I was willing to make the change myself, is there anything I should know as far as code standards?

Contribution guides can be found here: Contribution Guide.

Overall, it is best to start with opening an issue, especially in this case where the user-facing spec for the S3 Destination will have to change.

I created the issue here. I tested in my local environment without KMS, with AWS KMS, and with AWS Customer managed, and all worked as described in this pull request.

kind: source
spec:
  name: "hostaway"
  registry: "grpc"
  path: "localhost:7778"
  version: "v1.1.0"
  tables: 
    ["hostaway_listings"]
  destinations:
    - "s3"
---
kind: destination
spec:
  name: "s3"
  registry: "grpc"
  path: "localhost:7777"
  write_mode: "append"
  spec:
    bucket: "cloudquery-binaries"
    region: "us-east-2" # Example: us-east-1
    path: "test/{{TABLE}}/{{UUID}}"
    format: "csv" # options: parquet, json, csv
    sse_kms_key_id: ***
    server_side_encryption: aws:kms

server_side_encryption is an enum, so I’m not sure if I need to mention that it can only be one of the options listed in the AWS SDK documentation.

Nice, you should be able to specify the valid enum values in JSON schema, like so:

jsonschema:"enum=AES256,enum=aws:kms,enum=aws:kms:dsse"

No need to specify an empty default on non-required fields, I think, but check the resulting schema.json to make sure :slightly_smiling_face:

Ok, I pushed up the changes requested and tested jsonschema: "enum=AES256,enum=aws:kms,enum=aws:kms:dsse" with aws:kms. I have not tested the AES256 and aws:kms:dsse.