CloudQuery Okta connector throttling API calls for rate limit issues

Hey team. I’m using the Okta connector to grab my Okta group config. My Okta admin team tells me that I’m reaching the Okta API access limit. The limit is based on the number of calls in 1 minute. I can see I can set the concurrency, but I think this sets the number of simultaneous calls. Is there any way to configure CloudQuery to throttle the speed of the calls?

Hi @noble-mongoose,

We expose rate limiting configuration here. You could try different values from the defaults for the Okta plugin rate limiting.

I only see max_backoff and max_retries in the documentation. These would both only apply to the rate of the API after an API call fails, correct?

Yes, I believe so. Looking at the Go client options, I don’t see another way to configure it here.

The Go client would probably need to implement this.

I opened this issue.

@erez
I am getting the below error:

failed to transform table okta_threat_insights: failed to create all of the desired primary keys: [Id]

When fetching ThreatInsights. Any idea?

failed to transform table okta_threat_insights: failed to create all of the desired primary keys: [Id]

Hi @decent-viper,

The error you’re getting doesn’t seem related to this thread (which is about rate limiting). Let’s use your message here this link for discussing it.