Use HEAD requests to avoid being rate limited by DockerHub

[tripplet]

Based on how exactly manifest requests will be counted, how many containers are monitored and which interval is configured this will probably affect many watchtower users.
https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/
Hacker News discussion

[zoispag]

Maybe we need to rethink about the skopeo solution @simskij @piksel

[simskij]

I totally agree, we also need to consider moving watchtower itself to GitHub packages.

[victorcmoura]

Yeah, now it sounds more interesting =S

[prabu-ssb]

Yes, skopeo (#607) might be a good long term solution.

[piksel]

I should just add that we pretty much ruled out skopeo since it doesn't keep track of the digests for the manifests and will do a full rate-limited manifest GET for every query, so it doesn't help us much.

[simskij]

I've tried to answer this in more detail in #668 (comment), but the executive summary is that we don't need to change anything, the docker client will take care of this for us and make sure requests counting towards the rate limit only happens when the image digest has changed. 🎉

[piksel]

Turns out this wasn't the full story. Docker sometimes doesn't do this, but performs a full GET request when checking for changes.
The most up to date discussion is currently in #669.

[willheineman]

Since this went live today, I am seeing the rate limit errors appearing frequently. I have 16 containers that I run on a home server, and most of the images I run don't even update every 24 hours. But starting at about 12pm EST I am getting the following error:

toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit. Proceeding to next.

I am going to try to change the frequency to 60 minutes, so that I can stay under the rate limit (16 images * 6 pulls per period = 96 every 6 hours)

But is there anything else I should be doing? I am looking into making authenticated calls, which would let me go to every 30 minutes.

[simskij]

Not at this moment, I'm afraid. We're currently tracking this issue in #669 if you'd like to stay informed.

[nhgiang196]

Despite the pull rate limit message. Watchtower in my computer mangage to update the exactly image by polling every 30 seconds.

The only problem is I got TOO MANY emails that said:...

[simskij]

We've moved discussions on this topic to #669. Locking this discussion to make sure everything is available at the same place.