From ac297007f9fc055066e9427f1c946d411eaf3a0a Mon Sep 17 00:00:00 2001 From: Tedi Mitiku Date: Thu, 11 Jul 2024 11:19:29 -0400 Subject: [PATCH 1/4] add docs on 0.0.0.0 gotcha --- .../public-and-private-ips-and-ports.md | 6 ++++++ .../api-reference/starlark-reference/service-config.md | 10 ++++++++++ 2 files changed, 16 insertions(+) diff --git a/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md b/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md index c07f3db2e0..e7e13ae602 100644 --- a/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md +++ b/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md @@ -42,3 +42,9 @@ grafana-1670597488 grafana http: 3000/tcp -> 127.0.0.1:55998 The IP address used to reach these containers is your localhost address, `127.0.0.1`. This is the "public IP address" of each container in the cluster. The combination of public IP + port _will_ allow you to connect to a container from your command line. For example, from the output above, `curl 127.0.0.1:55947` on your command line would make a request to private port `4000` on the `cl-client-0-beacon` service. + +### Gotchas + +When exposing a port on a `ServiceConfig` via the `ports` field, the container responsible for managing the enclave, APIContainer (APIC), will perform an availability check (`nc -vz `) to ensure the port is exposed. +Some services by default expose ports on `localhost:` or `127.0.0.1:`. This would cause the availability check (eg. `nc -vz 172.0.0.3 9612`) to fail as the APIC communicates with the service on a subnetwork created by Kurtosis for that enclave, +whilst the port is only exposed on the localhost network interface. The workaround is to expose the port on all network interfaces via `0.0.0.0` eg. `--rpc.laddr tcp://0.0.0.0:36657`. \ No newline at end of file diff --git a/docs/docs/api-reference/starlark-reference/service-config.md b/docs/docs/api-reference/starlark-reference/service-config.md index 06e523c62d..78dd1217dd 100644 --- a/docs/docs/api-reference/starlark-reference/service-config.md +++ b/docs/docs/api-reference/starlark-reference/service-config.md @@ -270,6 +270,15 @@ See [`NixBuildSpec`][nix-build-spec] for more information on how to use the Nix You can view more information on [configuring the `ReadyCondition` type here][ready-condition]. +:::note +If you are experiencing issues with unsuccessful port check, try exposing the port on all network interfaces via `0.0.0.0` eg `--rpc.laddr tcp://0.0.0.0:36657`). See [here][port-ip-doc] for an in depth explanation. +```bash + == FINISHED SERVICE 'service-a' LOGS =================================== + Caused by: An error occurred while waiting for all TCP and UDP ports to be open + Caused by: Unsuccessful ports check for IP '172.16.0.10' and port spec '{privatePortSpec:0x400071d0b0}', even after '240' retries with '500' milliseconds in between retries. Timeout '2m0s' has been reached +``` +::: + :::tip If you are trying to use a more complex versions of `cmd` and are running into issues, we recommend using `cmd` in combination with `entrypoint`. You can set the `entrypoint` to `["/bin/sh", "-c"]` and then set the `cmd` to the command as you would type it in your shell. For example, `cmd = ["echo foo | grep foo"]` @@ -320,3 +329,4 @@ The `tolerations` field expects a list of [`Toleration`][toleration] objects bei [user]: ./user.md [toleration]: ./toleration.md [nix-build-spec]: ./nix-build-spec.md +[port-ip-doc]: ../../advanced-concepts/public-and-private-ips-and-ports.md \ No newline at end of file From a478188b483d85cf06ddad5f39067957b872a22e Mon Sep 17 00:00:00 2001 From: Tedi Mitiku Date: Thu, 11 Jul 2024 11:22:45 -0400 Subject: [PATCH 2/4] simplify a little bit --- .../advanced-concepts/public-and-private-ips-and-ports.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md b/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md index e7e13ae602..84814be3bf 100644 --- a/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md +++ b/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md @@ -45,6 +45,4 @@ The combination of public IP + port _will_ allow you to connect to a container f ### Gotchas -When exposing a port on a `ServiceConfig` via the `ports` field, the container responsible for managing the enclave, APIContainer (APIC), will perform an availability check (`nc -vz `) to ensure the port is exposed. -Some services by default expose ports on `localhost:` or `127.0.0.1:`. This would cause the availability check (eg. `nc -vz 172.0.0.3 9612`) to fail as the APIC communicates with the service on a subnetwork created by Kurtosis for that enclave, -whilst the port is only exposed on the localhost network interface. The workaround is to expose the port on all network interfaces via `0.0.0.0` eg. `--rpc.laddr tcp://0.0.0.0:36657`. \ No newline at end of file +When you expose a port on a `ServiceConfig` via the `ports` field, the container responsible for managing the enclave, APIContainer (APIC), will perform an availability check (`nc -vz `) on the port. Some services default to exposing ports on `localhost:` or `127.0.0.1:`. This causes the availability check (eg. `nc -vz 172.0.0.3 9612`) to fail, as the APIC communicates with the service on a subnetwork created by Kurtosis for that enclave, whilst the port is only exposed on the localhost network interface. The workaround is to expose the port on all network interfaces via `0.0.0.0` eg. `--rpc.laddr tcp://0.0.0.0:36657`. \ No newline at end of file From 6ccaa6ecfcbf832cec72d7accd34ce0f4075cdb3 Mon Sep 17 00:00:00 2001 From: Tedi Mitiku Date: Thu, 11 Jul 2024 11:24:08 -0400 Subject: [PATCH 3/4] update port ip doc ref to gotchas --- docs/docs/api-reference/starlark-reference/service-config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/api-reference/starlark-reference/service-config.md b/docs/docs/api-reference/starlark-reference/service-config.md index 78dd1217dd..eae844ed16 100644 --- a/docs/docs/api-reference/starlark-reference/service-config.md +++ b/docs/docs/api-reference/starlark-reference/service-config.md @@ -329,4 +329,4 @@ The `tolerations` field expects a list of [`Toleration`][toleration] objects bei [user]: ./user.md [toleration]: ./toleration.md [nix-build-spec]: ./nix-build-spec.md -[port-ip-doc]: ../../advanced-concepts/public-and-private-ips-and-ports.md \ No newline at end of file +[port-ip-doc]: ../../advanced-concepts/public-and-private-ips-and-ports.md#gotchas \ No newline at end of file From e2765c909dbf148e3d35b92cbed979da5d16e088 Mon Sep 17 00:00:00 2001 From: Tedi Mitiku Date: Thu, 11 Jul 2024 11:28:25 -0400 Subject: [PATCH 4/4] simplify --- docs/docs/advanced-concepts/public-and-private-ips-and-ports.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md b/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md index 84814be3bf..6f9f6abfc3 100644 --- a/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md +++ b/docs/docs/advanced-concepts/public-and-private-ips-and-ports.md @@ -45,4 +45,4 @@ The combination of public IP + port _will_ allow you to connect to a container f ### Gotchas -When you expose a port on a `ServiceConfig` via the `ports` field, the container responsible for managing the enclave, APIContainer (APIC), will perform an availability check (`nc -vz `) on the port. Some services default to exposing ports on `localhost:` or `127.0.0.1:`. This causes the availability check (eg. `nc -vz 172.0.0.3 9612`) to fail, as the APIC communicates with the service on a subnetwork created by Kurtosis for that enclave, whilst the port is only exposed on the localhost network interface. The workaround is to expose the port on all network interfaces via `0.0.0.0` eg. `--rpc.laddr tcp://0.0.0.0:36657`. \ No newline at end of file +When you expose a port on a `ServiceConfig` via the `ports` field, the container responsible for managing the enclave, APIContainer (APIC), will perform an availability check (`nc -vz `) on the port. Some services default to exposing ports on `localhost:` or `127.0.0.1:`. This causes the availability check (eg. `nc -vz 172.0.0.3 9612`) to fail, as the APIC communicates with the service on a subnetwork for that enclave, whilst the port is only exposed on the localhost network interface. The workaround is to expose the port on all network interfaces via `0.0.0.0` eg. `--rpc.laddr tcp://0.0.0.0:36657`. \ No newline at end of file