Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up deletion of containers #2

Open
nikidimi opened this issue Oct 26, 2016 · 4 comments
Open

Speed up deletion of containers #2

nikidimi opened this issue Oct 26, 2016 · 4 comments

Comments

@nikidimi
Copy link
Member

nikidimi commented Oct 26, 2016

Stopping the containers is currently very slow. This is because docker-py waits 10 sec before sending SIGKILL.

nikidimi added a commit that referenced this issue Oct 26, 2016
@nikidimi
Copy link
Member Author

Currently temporary fixed with explicit call to kill.
@edudev Do you know a better way?

@azlekov
Copy link
Member

azlekov commented Oct 31, 2016

For example in dokku when you are deploying application and there's old container, dokku swaps them:

  1. Dokku deploy the application in new container
  2. Configure the network interface or nginx to points to the new container
  3. Destroy the old container in background after 60 sec.

Can some similar logic helps you?

@edudev
Copy link
Collaborator

edudev commented Oct 31, 2016

The reason SIGTERM isn't working is because bash traps it and does nothing.
Now the question is: why are we using bash as our init process?
If you change bash to something that doesn't ignore SIGTERM (like "sleep infinity"), everything will work fine. In this case, "sleep" will take the role of the init process (PID 1) and the container will be up, as long as "sleep" sleeps for infinity.
Actually, if you change infinity to, let's say 10s ("sleep 10s"), we'll have a basic time limit implementation. When the init process (PID 1) terminates, the container dies, as well.
A possible flaw in this scenario are zombie processes. The init process' job (PID 1) is to deal with zombie processes. Bash is a general purpose garbage collector. I doubt that "sleep" has the same "behavior". But I'm willing to let a zombie live for at most 10 secs.
An alternative would be to use a "simple" init script. There are already some wide spread ones (https://engineeringblog.yelp.com/2016/01/dumb-init-an-init-for-docker.html, https://github.com/Yelp/dumb-init). But to me, they are are overkill (and will probably fix this problem, but won't give us a time limit).

PS: If you'd like to stick with bash, adding a few lines that explicitly trap for SIGTERM and exit, might do the trick.

@nikidimi
Copy link
Member Author

nikidimi commented Nov 2, 2016

@L3K0V We can use something similar - there is no need to wait for the deletion to finish, but we are very far from such optimizations.

We discovered with @edudev that the problem is that PID 1 usually ignores SIGTERM. I think the explicit call to kill will do for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants