Extracting CloudWatch Alarm ARN using CloudQuery for reporting and analysis

CloudQuery Plugins
1

Hello Cloudquery Support Team,

I hope you’re all well. I’ve successfully integrated CloudWatch with PagerDuty, and it’s been an efficient setup. PagerDuty receives alerts whenever a CloudWatch alarm is triggered, and it creates corresponding incidents seamlessly.

In the custom details of these incidents (in PagerDuty), we find the CloudWatch Alarm ARN, which is a critical piece of information for our incident management processes.

I’m interested in learning whether Cloudquery can assist us in automatically extracting the CloudWatch Alarm ARN during its data collection process. This would be incredibly valuable for our reporting and analysis efforts.

If you have any insights, recommendations, or resources on how to achieve this integration or data extraction, I’d greatly appreciate your guidance.

Thank you for your assistance, and I’m looking forward to your response.

2

Hi @romantic-buffalo :wave:

Are you hoping to extract the CloudWatch Alarm ARN from AWS? Assuming that’s the case, yes, you can set up a CloudQuery sync that syncs the aws_cloudwatch_alarms table (AWS CloudWatch Alarms).

If you’re looking to get started, I’d recommend checking out the quickstart guide (Quickstart Guide) or one of the AWS integration guides, depending on where you want to sync the data to (AWS Integrations).

From the PagerDuty side, we also have a PagerDuty plugin with an incidents table (PagerDuty Incidents), which would allow you to compare what you have in AWS with what you have in PagerDuty.

3

Hi Herman, thanks for your response.

Just a slight issue. When I run the CloudQuery tool to extract data from incidents in PagerDuty, it produces all the columns as listed in the link you sent me: CloudQuery PagerDuty Incidents Documentation.

Which is all fine. However, what I required was a certain section in the custom details of a PagerDuty Incident which has the CloudWatch Alarm ARN to be part of the results when I export to file (one of the destinations listed in CloudQuery).

Unfortunately, the body column does not provide any details in my results. It just shows two curly brackets with nothing inside.

I’ll attach screenshots if it may help.

In the screenshots, it shows a section that says Alarm ARN. I need that part to be in the results when using the CloudQuery tool. The body column does not provide any details regarding that.

Many thanks :slightly_smiling_face:

4

Ah right, I see. I think we are missing this information, but the API docs suggest that it is possible to fetch it. Would you mind opening an issue on GitHub for this?
Here is the repo where issues can be reported: https://github.com/cloudquery/cloudquery/issues/new/choose
@romantic-buffalo I tried adding some support for this to the plugin but haven’t tested it on a PagerDuty instance yet. Would you mind giving it a try? Here’s the pre-release version: v3.12.0-rc2

You can use it in your config like this:

kind: source
spec:
  # Source spec section
  name: "pagerduty"
  path: "cloudquery/pagerduty"
  registry: "github"
  version: "v3.12.0-rc2"
  tables: ["*"]
  destinations: ["postgresql"]
  spec:
    # optional
    team_ids: ["<team_id>"]

(Note the registry is github here, that’s important)

5

Thanks a lot, Herman. Will it be the same procedure when running the query, or will I have to adjust the process?

6

Yes, it should be the same, except that now there should be a custom_fields column on the pagerduty_incidents table.

7

Thanks a lot, Herman. I’ve tested out the new change, and it manages to capture custom_fields as a column; however, no details are displayed. Is there something I’m missing?

I’ve attached some screenshots below.

8

Thanks for checking! Hmm, if you’re sure it’s always empty I’ll need to investigate a bit more. I’ve opened an issue to track this here #15115.

9

Thanks, Herman, for your help. I’ll follow up with the issue.