Health check support?

[smebberson]

Does docker-gen support anything to do with health checks? health_status events?

[hendywijaya]

Hi in my observation, this should be done by the nginx itself (https://www.nginx.com/products/nginx/load-balancing/) available on NGINX+ (plus), commercial license
OR use the open source tengine (nginx 1.8 drop-in) (http://tengine.taobao.org/document/http_upstream_check.html)(https://hub.docker.com/r/cloudposse/tengine/)

[jmorino]

@hendywijaya the nginx container has no knowledge whether the docker-gen process is still running and listening to the Docker socket or not

[budarin]

Need wait for health check before publish new config - service is not started properly but start to receive requests!

[buchdag]

Do you mean docker-gen need to wait for other containers healtchecks, or docker-gen needs to publish its own healtcheck for other services to check ?

[budarin]

I believe that docker-gen should add a new container to the nginx configuration (generate new version of config) only when container is really running and has passed the health check

it's strange that docker is still checking the container for its full start and nginx is already running traffic on it

in the end, the container may not start due to errors, but docker-gen will already place it in the nginx configuration and users will get a lot of errors.

[buchdag]

As I see it the features require two components:

1. docker-gen needs to listen to health_status events
2. the nginx-proxy template needs additional logic to check if the container have health status available (because the vast majority of nginx-proxy users probably use containers that don't and should not be affected by this) , and if yes add the proxying config for them only when they're healty

1 should be super easy to do but the tests will be somewhat trickier (because the test code is not easy to untangle), and I won't merge a feature change without proper tests.

I believe 2 should already be possible but I lack proper experience with health_status in general and without 1 it won't be of any use (because the configuration won't be re-rendered when the containers become healthy or unhealthy). Tests should be moderately easy to write.

[budarin]

can it be implemented behind a flag in the docker-gen command parameters?
--requireHealthCheck

[darklow]

Huge benefit already would be just by having docker-gen to wait for docker native container healthchecks (https://docs.docker.com/engine/reference/builder/#healthcheck) and start proxying requests to it only when container has healthy. I bet the health status is somewhere available. It seems right now requests are proxied even at container starting state, not healthy. Or an option to choose wether to reload master nginx at container "starting" or getting "healthy" events would be great.

This would allow us to use following syntax for target containers like:

...
  healthcheck:
    test: [ "CMD", "curl", "-H", "Host: example.com", "http://localhost:80/health-check/" ]
    interval: 3s
    timeout: 5s
    retries: 10

[Killerkiwi2005]

#442 seems related

[budarin]

not yet - only PR is presented (