# Error with table gcp\_iam\_deny\_policies due to quota exceeded issues

**URL:** <https://community.cloudquery.io/t/error-with-table-gcp-iam-deny-policies-due-to-quota-exceeded-issues/331>\
**Category:** CloudQuery Plugins\
**Created:** [March 1, 2024, 6:02pm UTC](https://community.cloudquery.io/t/error-with-table-gcp-iam-deny-policies-due-to-quota-exceeded-issues/331 "2024-03-01T18:02:07Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![pretty-wolf](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@pretty-wolf](https://community.cloudquery.io/u/pretty-wolf)\
**Post date:** [March 1, 2024, 6:02pm UTC](https://community.cloudquery.io/t/error-with-table-gcp-iam-deny-policies-due-to-quota-exceeded-issues/331/1 "2024-03-01T18:02:07Z")

</div>

Overall, most of my stuff is going great, but this table seems to error more than any other. Any tips on how to slow it down or optimize it?

```json
{
  "level": "error",
  "module": "gcp-src",
  "client": "project:redact-infra-cost",
  "error": "rpc error: code = ResourceExhausted desc = Quota exceeded for quota metric 'Policy Read Requests' and limit 'Policy Read Requests per minute' of service 'iam.googleapis.com' for consumer 'project_number:redact'.\nerror details: name = ErrorInfo reason = RATE_LIMIT_EXCEEDED domain = googleapis.com metadata = map[consumer:projects/redact quota_limit:ReadPolicyRequestsPerMinutePerCredentialProject quota_limit_value:10 quota_location:global quota_metric:iam.googleapis.com/read_policies/per_credential service:iam.googleapis.com]\nerror details: name = Help desc = Request a higher quota limit. url = https://cloud.google.com/docs/quota#requesting_higher_quota",
  "message": "table resolver finished with error",
  "table": "gcp_iam_deny_policies",
  "time": "2024-03-01T00:00:17Z"
}

```

My config is:

```yaml
---
kind: source
spec:
  # Source spec section
  name: "gcp"
  path: "cloudquery/gcp"
  registry: "cloudquery"
  version: "v11.0.2"
  destinations:
    - bq
  tables:
    - gcp_projects
    - gcp_iam*
  spec:
    project_filter: "NOT id:sys-*"

```

For context, without the `project_filter`, our GCP is pretty thicc:

```bash
➜ ~ gcloud projects list | wc -l
13732

```

---

<div class="post-metadata">

**Author:** ![ben](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/ben/32/88_2.png) [@ben](https://community.cloudquery.io/u/ben)\
**Post date:** [March 1, 2024, 6:10pm UTC](https://community.cloudquery.io/t/error-with-table-gcp-iam-deny-policies-due-to-quota-exceeded-issues/331/2 "2024-03-01T18:10:20Z")

</div>

I would try reducing the concurrency from the default of 50,000 to something like 500:

```yaml
---
kind: source
spec:
  # Source spec section
  name: "gcp"
  path: "cloudquery/gcp"
  registry: "cloudquery"
  version: "v11.0.2"
  destinations:
    - bq
  tables:
    - gcp_projects
    - gcp_iam*
  spec:
    project_filter: "NOT id:sys-*"
    concurrency: 500

```

The concurrency is the value that defines the upper bound of tables being resolved in parallel, so by lowering that value you are less likely to hit the rate limit.

---

<div class="post-metadata">

**Author:** ![pretty-wolf](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@pretty-wolf](https://community.cloudquery.io/u/pretty-wolf)\
**Post date:** [March 1, 2024, 7:09pm UTC](https://community.cloudquery.io/t/error-with-table-gcp-iam-deny-policies-due-to-quota-exceeded-issues/331/3 "2024-03-01T19:09:33Z")

</div>

Ok, neat call! I’ll try that. I don’t care if it takes long to run; our SLA right now is daily.

---

<div class="post-metadata">

**Author:** ![KernelKnight](https://avatars.discourse-cdn.com/v4/letter/k/9de053/32.png) [@KernelKnight](https://community.cloudquery.io/u/KernelKnight)\
**Post date:** [March 1, 2024, 8:42pm UTC](https://community.cloudquery.io/t/error-with-table-gcp-iam-deny-policies-due-to-quota-exceeded-issues/331/4 "2024-03-01T20:42:07Z")

</div>

Additionally, as outlined in the [docs](https://hub.cloudquery.io/plugins/source/cloudquery/gcp/latest/docs#configuration), I suggest to play with `backoff_delay` and `backoff_retries` settings (note that by default the `backoff_retries` is `0`).

---

<div class="post-metadata">

**Author:** ![pretty-wolf](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@pretty-wolf](https://community.cloudquery.io/u/pretty-wolf)\
**Post date:** [March 4, 2024, 3:51pm UTC](https://community.cloudquery.io/t/error-with-table-gcp-iam-deny-policies-due-to-quota-exceeded-issues/331/5 "2024-03-04T15:51:42Z")

</div>

I might need to tweak that also

```auto
~ k -n cloudquery logs pod/cloudquery-28491840-q7cx9 | jq '.error' | sed 's/.*Quota exceeded for/Quota exceeded for/g' | sed 's/of service .*//g' | sort | uniq

```

still has 229 errors of varying

```auto
Quota exceeded for quota metric 'Policy Read Requests' and limit 'Policy Read Requests per minute'
Quota exceeded for quota metric 'Policy Read Requests' and limit 'Policy Read Requests per minute per user'

```