# Arrow type for DateTime in CloudQuery

**URL:** <https://community.cloudquery.io/t/arrow-type-for-datetime-in-cloudquery/451>\
**Category:** CloudQuery Plugins\
**Created:** [January 13, 2024, 8:16am UTC](https://community.cloudquery.io/t/arrow-type-for-datetime-in-cloudquery/451 "2024-01-13T08:16:00Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![comic-pup](https://avatars.discourse-cdn.com/v4/letter/c/dec6dc/32.png) [@comic-pup](https://community.cloudquery.io/u/comic-pup)\
**Post date:** [January 13, 2024, 8:16am UTC](https://community.cloudquery.io/t/arrow-type-for-datetime-in-cloudquery/451/1 "2024-01-13T08:16:00Z")

</div>

What Arrow type should I use for a DateTime? Right now I’m using `arrow.PrimitiveTypes.Date64`, but that is clearly just the date.

---

<div class="post-metadata">

**Author:** ![herman](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/herman/32/27_2.png) [@herman](https://community.cloudquery.io/u/herman)\
**Post date:** [January 13, 2024, 9:22am UTC](https://community.cloudquery.io/t/arrow-type-for-datetime-in-cloudquery/451/2 "2024-01-13T09:22:14Z")

</div>

We generally use timestamp [pyarrow.timestamp](https://arrow.apache.org/docs/python/generated/pyarrow.timestamp.html#pyarrow.timestamp).

---

<div class="post-metadata">

**Author:** ![comic-pup](https://avatars.discourse-cdn.com/v4/letter/c/dec6dc/32.png) [@comic-pup](https://community.cloudquery.io/u/comic-pup)\
**Post date:** [January 14, 2024, 1:19am UTC](https://community.cloudquery.io/t/arrow-type-for-datetime-in-cloudquery/451/3 "2024-01-14T01:19:46Z")

</div>

```auto
arrow.FixedWidthTypes.Timestamp_us worked. Thank you.

How do you guys choose to have a parent-child relation or just have a column resolver with the other API route?

```

---

<div class="post-metadata">

**Author:** ![herman](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/herman/32/27_2.png) [@herman](https://community.cloudquery.io/u/herman)\
**Post date:** [January 15, 2024, 10:10am UTC](https://community.cloudquery.io/t/arrow-type-for-datetime-in-cloudquery/451/4 "2024-01-15T10:10:18Z")

</div>

If the relationship is 1:many, we use child tables. And generally, we prefer parent-child over column resolvers because this has better concurrency control in our SDK and gives end users more flexibility to skip the child tables if they don’t want it.

We prefer to keep tables as close as possible to the underlying APIs and leave further transformations to users. So we tend to use column resolvers only if they don’t trigger more API calls (maybe the data just needs to be massaged a bit to fit into the Arrow format, for example).

There are some historical tables that break these rules of thumb, but this is how I think about it now.

---

<div class="post-metadata">

**Author:** ![comic-pup](https://avatars.discourse-cdn.com/v4/letter/c/dec6dc/32.png) [@comic-pup](https://community.cloudquery.io/u/comic-pup)\
**Post date:** [February 2, 2024, 7:07pm UTC](https://community.cloudquery.io/t/arrow-type-for-datetime-in-cloudquery/451/5 "2024-02-02T19:07:47Z")

</div>

I need to use the S3 Destination Plugin, but I need to specify a KMS key for encryption. I noticed the following code snippet:

```go
s, err = c.Client.StartStream(table, func(r io.Reader) error {
    _, err := c.uploader.Upload(ctx, &s3.PutObjectInput{
        Bucket: aws.String(c.spec.Bucket),
        Key: aws.String(objKey),
        Body: r,
    })
    return err
})

```

I would need to turn it into this:

```go
s, err = c.Client.StartStream(table, func(r io.Reader) error {
    _, err := c.uploader.Upload(ctx, &s3.PutObjectInput{
        Bucket: aws.String(c.spec.Bucket),
        Key: aws.String(objKey),
        Body: r,
        
        SSEKMSKeyId: aws.String("your-kms-key-id"), // replace with your KMS key ID
        ServerSideEncryption: aws.String("aws:kms"),
    })
    return err
})

```

Can I just fork the repo and create a merge request (MR)? I was planning on having it go through the spec, so I think I would need to complete testing as well. Is there any way to pass these via environment variables?

---

<div class="post-metadata">

**Author:** ![herman](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/herman/32/27_2.png) [@herman](https://community.cloudquery.io/u/herman)\
**Post date:** [February 2, 2024, 8:03pm UTC](https://community.cloudquery.io/t/arrow-type-for-datetime-in-cloudquery/451/6 "2024-02-02T20:03:54Z")

</div>

You are welcome to open a PR; it would most likely need to be a config option in the spec, yes. If you make it a config option, you can use environment variable expansion syntax to use an env variable in the config, e.g. `${SSE_KMS_KEY_ID}`.