Provide pre-build docker images

[JJ-8]

I get quite often the comment that CTFNote requires a large amount of memory to host (see also #126). This is also my own experience, since it requires around 2-3 GB RAM to build the frontend. However, afterward the building of especially the frontend, CTFNote requires much less RAM to run.

I think it would be nice to provide prebuilt docker images hosted via Github Container Registry or a similar service (which one?). This way, you can bypass the building and host CTFNote even on a 2 GB RAM VPS.
quay.io
However, I think we should not drop the manual build procedure since it may be preferred to not trust the prebuilt images and an easy alternative should be provided. But we may change the default docker-compose.yml file to use the prebuilt images.

Would this be supported by the CTFNote maintainers?

[XeR]

I think it would be nice to provide prebuilt docker images hosted via Github Container Registry or a similar service (which one?). This way, you can bypass the building and host CTFNote even on a 2 GB RAM VPS.

I agree.

I'd love to get more familiar with Github Actions (or similar) and have actions that automatically build and deploy packages and containers.
I think we should have a container tag for each git tag (e.g. have a new container for each release) and use the latest tag for HEAD.
Unfortunately, I have never used Github Actions so I have no idea how it works. I don't want to mess up and leak secrets.
@Anthares101's fork, which is the most popular image on Docker Hub is probably a good place to start.

via Github Container Registry or a similar service (which one?)

GHCR free plan has a 500MB limit.
Quay gives away unlimited storage space for FOSS projects.
Hub has a limit of 100 pulls every 6 hours

I feel like we would reach the 500MB relatively fast.

% docker image list ctfnote-api
REPOSITORY    TAG       IMAGE ID       CREATED        SIZE
ctfnote-api   latest    68b6233fcb76   26 hours ago   313MB

The 100 pulls limit on the other hand seems very high. (How often do you deploy a new version of CTFNote ?)
Do people even care about the containers used ? e.g. corporate policy that requires the use of Docker Hub.
If not, quay is the clear winner here. It seems to pull repositories, compile them and publish them automatically.
See this Gitlab blog post.

However, afterward the building of especially the frontend, CTFNote requires much less RAM to run.

If you are low on memory, here is how I compile the front for my server :

diff --git a/front/Dockerfile b/front/Dockerfile
index 4c35f732..19911867 100644
--- a/front/Dockerfile
+++ b/front/Dockerfile
@@ -11,6 +11,10 @@ WORKDIR /usr/src/app
 
 RUN yarn build
 
+# _--------_
+FROM scratch AS export
+COPY --from=build-deps /usr/src/app/dist/spa /
+
 # _--------_
 FROM nginx@sha256:5ba534070ae1e5e83d52141b11ddced689b476c0001e7205f50979dc0cbdde3d

Then I compile with docker image build --target export --output - . > www.tar on my own machine.
I upload the archive on my server and extract it. Since I already have a nginx server, I configured it and do not run the front container saving additional MB of memory.

Questions:

1. How does one learn Github Actions ?
2. Can I run a Github Action server locally, or is it 100% proprietary ?
3. Do we even want to use Github Action ? Quay looks dead simple, and I like simple.

[Anthares101]

If you end up wanting to combine forks, I will be glad to help if you need it 😄 . My action is not really managing Docker tags (Just latest tag used) because to be honest did not expect the popularity of the images, I just needed a way of deploying to my K3S ARM cluster and I built something fast, but should be easy to implement that management.

About ARM, I use QEMU for cross-compiling the image in a AMD64 environment (No native ARM for now in actions 😢) it obviously take some time but using the cache is no too bad.

I use the same login action for DockerHub setup, I guess with GHCR you avoid the "DockerHub site configuration" but the free limit is pretty tight.

[JJ-8]

@Anthares101, thanks for joining the discussion!

My action is not really managing Docker tags (Just latest tag used) because to be honest did not expect the popularity of the images, I just needed a way of deploying to my K3S ARM cluster and I built something fast, but should be easy to implement that management.

Do you have an example of how to implement this? The only way I know is through the docker/metadata-action.

I use the same login action for DockerHub setup, I guess with GHCR you avoid the "DockerHub site configuration" but the free limit is pretty tight.

GHCR just works, but is not the best option to choose I think. You always need a Github personal access token in order to pull the images, which takes time to setup. Something like Docker Hub would be better since you can access that anonymously.

[Anthares101]

Do you have an example of how to implement this? The only way I know is through the docker/metadata-action.

Yeah same, maybe there is another option but I don't really know about it. The examples in the docker/metadata-action readme seems pretty solid, maybe some modifications needed but not much.

[JJ-8]

I have updated the workflow in my fork, tested it and created a PR: #217. Feel free to leave any comments and/or suggest improvements! Scope of the PR is to only add GHCR for now and as soon as we have decided which Docker platform to use, add that one as well in a future PR.

[Anthares101]

The PR looks great, did not know about the matrix thing for jobs paralel execution 😲. Thanks for including the ARM build ❤️.

[JJ-8]

For some time we already provide images through GHCR which works solid, so we can close this discussion.