# Help needed understanding example code for CloudQuery source plugin development

**URL:** <https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543>\
**Category:** CloudQuery Plugins\
**Created:** [December 1, 2023, 2:06pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543 "2023-12-01T14:06:49Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![quiet-trout](https://avatars.discourse-cdn.com/v4/letter/q/5fc32e/32.png) [@quiet-trout](https://community.cloudquery.io/u/quiet-trout)\
**Post date:** [December 1, 2023, 2:06pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/1 "2023-12-01T14:06:49Z")

</div>

Regarding developing a source plugin: I am working from the example at [CloudQuery Documentation](https://docs.cloudquery.io/docs/developers/creating-new-plugin/go-source) and I want to make a source plugin that will collect the top viewed pages from the Wikipedia API. I’m having trouble understanding a part of the example code.

(More detail in thread:)

The example code is:

```go
func Comics() *schema.Table {
    return &schema.Table{
        Name: "xkcd_comics",
        Resolver: fetchComics,
        Transform: transformers.TransformWithStruct(&xkcd.Comic{}),
    }
}

```

What is the meaning of `xkcd` in `&xkcd.Comic{}`?

I assume I’ll change that to `wikipedia`, but I don’t see where else I may need to write `wikipedia`.

---

<div class="post-metadata">

**Author:** ![kemal](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/kemal/32/42_2.png) [@kemal](https://community.cloudquery.io/u/kemal)\
**Post date:** [December 1, 2023, 2:13pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/2 "2023-12-01T14:13:24Z")

</div>

`transformers` (and `Transform`) is a helper of sorts that generates fields dynamically from a given struct.

---

<div class="post-metadata">

**Author:** ![quiet-trout](https://avatars.discourse-cdn.com/v4/letter/q/5fc32e/32.png) [@quiet-trout](https://community.cloudquery.io/u/quiet-trout)\
**Post date:** [December 1, 2023, 2:13pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/3 "2023-12-01T14:13:43Z")

</div>

Okay. I did write a struct.

---

<div class="post-metadata">

**Author:** ![kemal](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/kemal/32/42_2.png) [@kemal](https://community.cloudquery.io/u/kemal)\
**Post date:** [December 1, 2023, 2:14pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/4 "2023-12-01T14:14:28Z")

</div>

You don’t need to use it, but it enables generating/updating the table columns _automatically_ as the data struct is updated. Especially useful if you’re using a third-party client which provides the data model.

---

<div class="post-metadata">

**Author:** ![quiet-trout](https://avatars.discourse-cdn.com/v4/letter/q/5fc32e/32.png) [@quiet-trout](https://community.cloudquery.io/u/quiet-trout)\
**Post date:** [December 1, 2023, 2:15pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/5 "2023-12-01T14:15:19Z")

</div>

So if the data model changes, CloudQuery will handle database migrations, right?  
My eventual aim is to write a Google Workspace plugin, and off hand I don’t know how stable that API is, so it seems worthwhile to learn about using a transformer.

---

<div class="post-metadata">

**Author:** ![kemal](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/kemal/32/42_2.png) [@kemal](https://community.cloudquery.io/u/kemal)\
**Post date:** [December 1, 2023, 2:16pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/6 "2023-12-01T14:16:56Z")

</div>

Yes. As you upgrade a client dependency to a newer version, the new version of your plugin will have the updated columns.

You’re not limited to using transformers (or _just_ transformers); of course, you can specify extra (or all) columns in the `Columns` field as well.

`WithPrimaryKeys` and `WithSkipFields` are the most used options for transformers. You might also want to look at `WithUnwrap*` options if the struct has some embedded data you’d like to “unwrap” as top-level columns.

These options provide some control on how the struct is transformed.

---

<div class="post-metadata">

**Author:** ![quiet-trout](https://avatars.discourse-cdn.com/v4/letter/q/5fc32e/32.png) [@quiet-trout](https://community.cloudquery.io/u/quiet-trout)\
**Post date:** [December 1, 2023, 2:22pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/7 "2023-12-01T14:22:23Z")

</div>

Okay, I can see how “unwrapping” could be useful.

---

<div class="post-metadata">

**Author:** ![kemal](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/kemal/32/42_2.png) [@kemal](https://community.cloudquery.io/u/kemal)\
**Post date:** [December 1, 2023, 2:24pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/8 "2023-12-01T14:24:11Z")

</div>

Hope this helps. If you get stuck, let us know for more explanations 🙂

---

<div class="post-metadata">

**Author:** ![quiet-trout](https://avatars.discourse-cdn.com/v4/letter/q/5fc32e/32.png) [@quiet-trout](https://community.cloudquery.io/u/quiet-trout)\
**Post date:** [December 1, 2023, 2:27pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/9 "2023-12-01T14:27:40Z")

</div>

That helped give general context on 🙂

---

<div class="post-metadata">

**Author:** ![kemal](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/kemal/32/42_2.png) [@kemal](https://community.cloudquery.io/u/kemal)\
**Post date:** [December 1, 2023, 2:27pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/10 "2023-12-01T14:27:52Z")

</div>

Our official open-source plugins live [here](https://github.com/cloudquery/cloudquery/tree/main/plugins/source) if you want to see transformers in action. I suggest checking the smaller plugins first; large ones can be a bit too complex sometimes.

---

<div class="post-metadata">

**Author:** ![quiet-trout](https://avatars.discourse-cdn.com/v4/letter/q/5fc32e/32.png) [@quiet-trout](https://community.cloudquery.io/u/quiet-trout)\
**Post date:** [December 1, 2023, 2:28pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/11 "2023-12-01T14:28:27Z")

</div>

I’m still unclear about the `xkcd` in `&xkcd.Comic{}`.  
Does `xkcd` refer to a file name? Something else?

---

<div class="post-metadata">

**Author:** ![kemal](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/kemal/32/42_2.png) [@kemal](https://community.cloudquery.io/u/kemal)\
**Post date:** [December 1, 2023, 2:29pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/12 "2023-12-01T14:29:07Z")

</div>

xkcd is the package name for the xkcd API client (which I think is also bundled with that plugin, let me check).

---

<div class="post-metadata">

**Author:** ![quiet-trout](https://avatars.discourse-cdn.com/v4/letter/q/5fc32e/32.png) [@quiet-trout](https://community.cloudquery.io/u/quiet-trout)\
**Post date:** [December 1, 2023, 2:29pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/13 "2023-12-01T14:29:16Z")

</div>

Ohh, okay.

And I think I see the detail I missed:

```auto
so we will create our own struct inside a new xkcd package

```

Where would the `xkcd` package go?

Like, would this be the directory structure:

```auto
.
├── README.md
├── client
│ ├── client.go
│ └── spec.go
├── cq-source-xkcd
├── go.mod
├── go.sum
├── main.go
├── plugin
│ └── plugin.go
├── xkcd
│ └── xkcd.go
└── resources
    └── table.go

```

Or would there be an `xkcd.go` file in `client`?

---

<div class="post-metadata">

**Author:** ![kemal](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/kemal/32/42_2.png) [@kemal](https://community.cloudquery.io/u/kemal)\
**Post date:** [December 1, 2023, 3:08pm UTC](https://community.cloudquery.io/t/help-needed-understanding-example-code-for-cloudquery-source-plugin-development/543/14 "2023-12-01T15:08:13Z")

</div>

it’s under `internal/xkcd/`  
if you’re implementing a new client for this plugin only, best practice is to put it under `internal/` so it’s not importable from “outside”
