# AWS Pricing plugin sync command failed due to api call limits and concurrency settings

**URL:** <https://community.cloudquery.io/t/aws-pricing-plugin-sync-command-failed-due-to-api-call-limits-and-concurrency-settings/666>\
**Category:** CloudQuery Plugins\
**Created:** [October 16, 2023, 3:28am UTC](https://community.cloudquery.io/t/aws-pricing-plugin-sync-command-failed-due-to-api-call-limits-and-concurrency-settings/666 "2023-10-16T03:28:13Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![calm-walrus](https://avatars.discourse-cdn.com/v4/letter/c/8edcca/32.png) [@calm-walrus](https://community.cloudquery.io/u/calm-walrus)\
**Post date:** [October 16, 2023, 3:28am UTC](https://community.cloudquery.io/t/aws-pricing-plugin-sync-command-failed-due-to-api-call-limits-and-concurrency-settings/666/1 "2023-10-16T03:28:13Z")

</div>

About the AWS Pricing plugin, the sync command stopped in the middle, even though I set `spec.concurrency` to 5:

```auto
source plugin process failed with signal: killed

```

After doing some research, I found this document:

[Using the AWS Price List Bulk API](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/using-the-aws-price-list-bulk-api.html#price-bulk-api-quotas)

It clearly shows that we need to set limits on the connections to the API call for the `AWS Price List Bulk API`.

What’s the best number I should define? Is the setting related to `spec.concurrency`? Any other parameters related?

```yaml
kind: source
spec:
  name: "awspricing"
  path: "cloudquery/awspricing"
  version: "v3.0.10"
  tables: ["*"]
  destinations:
    - "postgresql"
  spec:
    # Optional parameters
    # region_codes: []
    # offer_codes: []
    concurrency: 10000 # what number do I need to change to, 5?

```

How can I understand the relationship between `spec.concurrency` and the API’s `Token bucket size (10)` and `Refill rate per second (5)`?

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [October 16, 2023, 9:18am UTC](https://community.cloudquery.io/t/aws-pricing-plugin-sync-command-failed-due-to-api-call-limits-and-concurrency-settings/666/2 "2023-10-16T09:18:58Z")

</div>

Hi @calm-walrus, let me take a quick look at this.

---

<div class="post-metadata">

**Author:** ![calm-walrus](https://avatars.discourse-cdn.com/v4/letter/c/8edcca/32.png) [@calm-walrus](https://community.cloudquery.io/u/calm-walrus)\
**Post date:** [October 16, 2023, 11:15am UTC](https://community.cloudquery.io/t/aws-pricing-plugin-sync-command-failed-due-to-api-call-limits-and-concurrency-settings/666/3 "2023-10-16T11:15:42Z")

</div>

Enable debug and see some errors (`frame too large`)

```auto
2023-10-16T03:42:15Z INF Table exists, auto-migrating module=pg-dest table=awspricing_service_terms
2023-10-16T03:45:51Z INF End sync destinations=["postgresql (v5.0.1)"] module=cli source="aws-pricing (awspricing@v3.0.10)" sync_time=2023-10-16T03:42:05Z
2023-10-16T03:45:51Z INF Sending sync summary to analyticsv1.cloudquery.io:443 module=cli
2023-10-16T03:45:51Z WRN Failed to send sync summary error="rpc error: code = Unavailable desc = connection error: desc = \"error reading server preface: http2: frame too large\"" module=cli
2023-10-16T03:45:51Z INF waiting for source plugin to terminate module=cli

```

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [October 16, 2023, 11:21am UTC](https://community.cloudquery.io/t/aws-pricing-plugin-sync-command-failed-due-to-api-call-limits-and-concurrency-settings/666/4 "2023-10-16T11:21:57Z")

</div>

@calm-walrus Do you mind opening a bug report via [this link](https://github.com/cloudquery/cloudquery/issues/new?assignees=&labels=bug&projects=&template=bug_report.yml&title=bug%3A+%3Ctitle%3E)? That way, we’ll have all the debugging information in one place.

---

<div class="post-metadata">

**Author:** ![calm-walrus](https://avatars.discourse-cdn.com/v4/letter/c/8edcca/32.png) [@calm-walrus](https://community.cloudquery.io/u/calm-walrus)\
**Post date:** [October 16, 2023, 11:27am UTC](https://community.cloudquery.io/t/aws-pricing-plugin-sync-command-failed-due-to-api-call-limits-and-concurrency-settings/666/5 "2023-10-16T11:27:09Z")

</div>

Raised: [cloudquery/cloudquery#14552](https://github.com/cloudquery/cloudquery/issues/14552)