Socket address already in use error for CloudQuery plugin

CloudQuery Plugins
1 
Error: failed to listen unix:C:\Users\Administrator\AppData\Local\Temp\cq-KmpksAOvAWwLxjQg.sock: listen unix C:\Users\Administrator\AppData\Local\Temp\cq-KmpksAOvAWwLxjQg.sock: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.
Usage:
  mongodb serve [flags]

Flags:
      --address localhost:7777              address to serve on. can be tcp: localhost:7777 or unix socket: `/tmp/plugin.rpc.sock` (default "localhost:7777")
  -h, --help                                help for serve
      --license string                      Path to offline license file or directory
      --log-format string                   log format. one of: text,json (default "text")
      --log-level string                    log level. one of: trace,debug,info,warn,error (default "info")
      --network string                      the network must be "tcp", "tcp4", "tcp6", "unix" or "unixpacket" (default "tcp")
      --no-sentry                           disable sentry
      --otel-endpoint string                Open Telemetry HTTP collector endpoint
      --otel-endpoint-headers stringArray   Open Telemetry HTTP collector endpoint headers
      --otel-endpoint-insecure              use Open Telemetry HTTP endpoint (for development only)
      --otel-endpoint-urlpath string        Open Telemetry HTTP collector endpoint URL path

2024/05/20 08:42:57 failed to serve: failed to listen unix:C:\Users\Administrator\AppData\Local\Temp\cq-KmpksAOvAWwLxjQg.sock: listen unix C:\Users\Administrator\AppData\Local\Temp\cq-KmpksAOvAWwLxjQg.sock: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.
Error: failed to run plugin .cq\plugins\destination\cloudquery\mongodb\v2.3.11\plugin.exe: exit status 1
2

From the error message, it seems like you probably have another process using port 7777 already. What is the command you are running to run CloudQuery?

Are you starting the plugin server manually? In that case, you could try running it on another port, like

plugin.exe --address localhost:7778

If you can give us a bit more context about what you’re trying to do and what you’ve tried so far, we might be able to provide more specific tips.

3

I built my custom plugin and run npm run dev:

"dev": "ts-node --esm src/main.ts serve --address [::]:7777"

This is the sync.yaml:

cloudquery sync ./sync.yml

And I run this.

I think it is because of this prefix unix, but I am not sure about the solution.

4

Hmm, thanks, gotcha. This is probably not the problem, but it seems like the config is syncing the test database to itself in the destination? Is that right?

Yeah, this might very well be a Windows-specific problem, but I believe Windows supports Unix sockets nowadays. I’m looking through the plugin SDK code now and don’t see anything that jumps out to me.
It might be best to file a bug ticket. But one workaround for now could be to serve the destination plugin as a gRPC server as well (on port 7778). So something like:

.cq\plugins\destination\cloudquery\mongodb\v2.3.11\plugin.exe serve --address [::]:7778

with corresponding registry: grpc and path: 127.0.0.1:7778 in the destination config.

Could you try that?

5

I am trying now
Wow, it is working with that
What would be the next?

6

Did it show exactly the same error every time you ran it before? Like, was it cq-KmpksAOvAWwLxjQg.sock every time, or was the random string different every time?

7

It is ramdom string
@herman

8

All I can think to suggest is to try one of the solutions mentioned here: Stack Overflow - Only one usage of each socket address (protocol/network address/port) is normally permitted or here: Stack Overflow - Python server: Only one usage of each socket address is normally permitted.

Some of them include rebooting your machine :man_shrugging: As far as I know, we’ve not had similar reports before, but if it persists, then feel free to open an issue: GitHub - CloudQuery Issues.

9

Okay, thanks.
I am gonna open an issue soon.

10

Ok, great, thank you :+1:

For faster turnaround, please include everything you tried so far that didn’t work as well.

One other thought: does this only happen when you’re running CloudQuery with your custom JS plugin as the source? Or does it happen with other source plugins as well? This will help us narrow down where the issue is really coming from.

To test this, you could try running the Hackernews source plugin with a config like this:

kind: source
spec:
  name: "hackernews"
  path: "cloudquery/hackernews"
  registry: "cloudquery"
  version: "v3.1.8"
  tables: ["*"]
  destinations:
    - "mongodb"
  spec:
    item_concurrency: 100
    start_time: 2024-05-20T00:00:00Z
11

I tried, but the same issue is occurring.
I opened the issue here: https://github.com/cloudquery/cloudquery/issues/18029