Can I pass input parameters to a REST API URL in CloudQuery

CloudQuery Plugins
1

Another question, this one is vague because I am still learning the basics of QC: Is it possible to specify a parameter in the config file that will get passed to the URL of a REST API? The use case is I would like to avoid extracting all of the source table’s rows, only the ones that match a condition that I can specify either in the path or the parameters of the URL. I would like for this to be an input parameter so that it can be part of the sequence of steps I plan to set up using Kestra.

2

Yes, that should definitely be possible. Typically, the way this is done is during the initialization call where the plugin is setting up the client. You can add another field to the Spec, which would then be accessible to the resolver during syncing.

3

That sounds good, I’ll try it.
So it never goes through the table file, just the spec to the client file.

4

Sorry, I think I am using sloppy language.

5

Tables and their resolvers are only impacted by the client and the spec passed to the plugin.

6

I am stumbling on this because it’s the same parameter that gets inserted from the parent, which happens in the table file.

7

If the value is available in the parent resource, you should be able to just grab it from there.

8

I want to get a list of all the pets of just one person.

For example, the parent table is persons and the child table is pets. I don’t want to get all of the parents and all of their children, just the children for a single parent.

How can I query this?

9

A child table can implement filtering so that if the parent value != “some-value”, then just return and don’t sync any data.

10

Ok, that would be done in the config spec?
I was thinking it would be easier to just feed one hard-coded parent ID to the child table resolver.

11

I think you should be able to access the spec in a resolver via client.spec, which would mean you could do something like:

if parent_id is not client.spec.filter_val:
  return None

Here is an example of a place where we follow a similar pattern: GitHub - CloudQuery

12

ok, thanks again. I’ll give it a go
great! that worked! I needed to add a method to expose the _spec property of the client, similar to how the _client property was done. thanks!