ADMB Docker Images available for testing

[johnoel]

Hi all,

For those that use Docker, the ADMB Images are now available for testing.

To use the ADMB Images, view the Readme.md for links to the quick start guides.

Any feedback or questions, please post to this discussion.

Thank you,
Johnoel

[arni-magnusson]

Hi Johnoel,

Thanks for making the ADMB images available for testing. I'd like to learn how to use these. I just barely pass as someone 'that uses Docker'. I've tried Docker every couple of years since 2016 and always reached the same conclusion: hey, this stuff is cool! I can see potential use cases for ADMB images.

My test platform is my work laptop, which has Windows 10 and Docker Desktop installed with a few r-base images from previous (successful) tests.

• Step 1. Before running Docker commands in the Dos shell, I first had to start Docker Desktop from the start menu.
• Step 2. Starting with admb:windows, I first try

C:\simple> docker run -rm --volume C:\simple:C:\simple --workdir C:\simple admb simple.tpl

and get the error message

unknown shorthand flag: 'r' in -rm
See 'docker run --help'.

Then I try two hyphens

C:\simple> docker run --rm --volume C:\simple:C:\simple --workdir C:\simple admb simple.tpl

and get the error message

docker: Error response from daemon: the working directory 'C:\simple' is invalid, it needs to be an absolute path.
See 'docker run --help'.

Then I try tinkering with the path format

C:\simple> docker run --rm --volume //c/simple:/simple --workdir /simple admb simple.tpl

and get the error message

Unable to find image 'admb:latest' locally
docker: Error response from daemon: pull access denied for admb, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

Then I specify the image with creator and tag

C:\simple> docker run --rm --volume //c/simple:/simple --workdir /simple johnoel/admb:windows simple.tpl

and get the error message

Unable to find image 'johnoel/admb:windows' locally
windows: Pulling from johnoel/admb
382f2b0667e3: Pulling fs layer
ab3b0f679434: Downloading [>                                                  ]  2.681MB/973.4MB
0238ca97a540: Downloading [>                                                  ]  87.75kB/8.415MB
0bc291451ca4: Waiting
2751bacc2319: Waiting
79b0f7075d7d: Waiting
57388cfaa697: Waiting
8ecd223b3a51: Waiting
bb432a02307f: Waiting
docker: image operating system "windows" cannot be used on this platform: operating system is not supported.
See 'docker run --help'.

The error message seems to be saying that my host platform (Windows 10 Enterprise) does not support the guest platform (Windows 10 LTSC 2019)...

And here I was, hoping that: Win 10 + Win 10 = Profit!

[johnoel]

Corrected many of the Quickstart typos. On Windows, I have only tested using the Command Prompt which does not need the //c/ syntax for the directory.

[johnoel]

I don't have the Enterprise version installed anywhere. Maybe, another Windows base system can be used?

[arni-magnusson]

• Step 3. Moving on to admb:linux, I now try

C:\simple> docker run --rm --volume //c/simple:/simple --workdir /simple johnoel/admb:linux simple.tpl

All I did was changing the :windows tag to :linux and this time the slot machine is pouring out money:

*** Parse: simple.tpl
tpl2cpp simple || tpl2rem simple

*** Compile: simple.cpp
g++ -c -std=c++17 -O3 -D_USE_MATH_DEFINES -DUSE_ADMB_CONTRIBS -I. -I"/usr/local/admb/include" -I"/usr/local/admb/include/contrib" -osimple.obj simple.cpp

*** Linking: simple.obj
g++ -std=c++17 -O3 -osimple simple.obj "/usr/local/admb/lib/libadmb-contrib-x86_64-linux-g++13.a"

Successfully built executable.

Now my C:\simple folder contains

-rw-r--r-- 1 2169872  2023-10-16 16:02  simple
-rw-r--r-- 1    4132  2023-10-16 16:02  simple.cpp
-rw-r--r-- 1     134  2022-05-18 11:53  simple.dat
-rw-r--r-- 1    1734  2023-10-16 16:02  simple.htp
-rw-r--r-- 1   30376  2023-10-16 16:02  simple.obj
-rw-r--r-- 1     296  2022-05-18 11:53  simple.tpl

which is pretty neat, but now I'd like to run the model using the Docker image and I'm not sure how.

Can the Docker image be used to run models or can it only compile?

I guess I'm pretending that my host machine is a vanilla Windows laptop that does not have the capability to run Linux executables. Is that not a use case we have in mind, a Windows machine with very few things installed?

• Step 4. Feeling lucky, I try

C:\simple> docker run --rm --volume //c/simple:/simple --workdir /simple debian ./simple

but get the error message

./simple: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by ./simple)

[johnoel]

For the Linux image, you may be able to do either build-only or run-only. It would depend on whether the Linux shared libraries are compatible with executable. You could create a static executable (admb -p) to avoid the shared library headaches. It's probably best to build and run models using the Docker Linux Image especially if you are on a Windows or MacOS platform.

[johnoel]

For Step 4, use

C:\simple> docker run --rm --volume %CD%:/simple --workdir /simple --entrypoint ./simple johnoel/admb:linux

[arni-magnusson]

• Step 5. Feeling slightly more desperate, I now try running the model using WSL2 (Ubuntu 20.04)

$ ./simple

and get a similar error message:

./simple: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./simple)
./simple: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by ./simple)

Can you help me reach my traditional conclusion that hey, this stuff is cool :)

[johnoel]

Wow, I never thought about WSL2. I will try this today.

[johnoel]

Looks like the Image base Debian 12.1 is not compatible with Ubuntu 20.04. There are other Linux distributions available for Docker that can be used.

[johnoel]

It is best to keep using the same Image johnoel/admb:linux.

[arni-magnusson]

• Step 6. Aha, I found a way to run the executable, using the 'gcc' Docker image,

C:\simple> docker run --rm --volume //c/simple:/simple --workdir /simple gcc ./simple

resulting in the familiar:

Starting optimization of 'simple' in phase 1 of 1 at Tue Oct 17 00:11:32 2023
phase= 1 | nvar=  2 | iter=  0 | nll=2.50e+01 | mag=3.61e+00 | par[  2]=b1
phase= 1 | nvar=  2 | iter=  7 | nll=3.45e+00 | mag=7.00e-05 | par[  2]=b1
Optimization completed after 0.002 s with final statistics:
  nll=3.451260 | mag=7.00140e-05 | par[  2]=b1

Calculating Hessian: 1, 2 done (0 s)
Inverting Hessian: 1, 2 done (0.006 s)
Starting standard error calculations...  done (0.014 s)

Finished running model 'simple' after 0.038 s.

I'm still wondering if the same ADMB image could both compile and run models?

Juggling two Docker images (ADMB and GCC) to compile and run my model seems to invite version conflicts and other maintenance problems that might require more Docker expertise than I have :)

[johnoel]

Yes Arni, the same Docker image can be used to build and run models.

[johnoel]

Once again, it is best to keep using the same Image johnoel/admb:linux

[johnoel]

Thanks Arni for testing and providing feedback. Sorry, I was a bit sloppy. The QuickStarts have been updated and tested by me. Please try again and let me know.

[iantaylor-NOAA]

@johnoel thanks for updating the QuickStart guides (at https://github.com/admb-project/admb/tree/main/scripts/docker#readme).

@arni-magnusson, are you will to try again to see if you have better luck following the updated instructions?

[e-perl-NOAA]

@johnoel can you provide any guidance on how to use a makefile instead of a tpl? I'm struggling to get the stock synthesis makefile to run with the docker image.

[johnoel]

I have added the docker target the GNUmakefile in stock-synthesis to build the ss executable using the ADMB Docker Linux image.

See, the pull request in stock synthesis.

To build, use the command below

stock-synthesis$ make docker

This will build a linux executable. If the host computer is linux, you could run the ss executable on a local run directory. If the host computer is not linux (ie MacOS or Windows), you will need to copy the ss executable to the run directory, then use ADMB Docker Linux image to run ss.

ie,

cd ~/ss-run
cp ~/stock-synthesis/ss .
docker run --rm --volume $PWD:/ss-run --workdir /ss-run --entrypoint ./ss johnoel/admb:linux

Let me know if you need more details

[e-perl-NOAA]

Yes, I would love some more details with running on a MacOS and windows. The linux build is easy and running for me (although it's not creating the ss_opt file which i noted in the PR). When you say the "run directory", what are you meaning by this? Is that a new directory that you create to house the outputs of the building ss3? My first use case for this is to be able to use the docker image for our build-ss3.yml workflow for each of the machines rather than the current structure for building admb. IT is struggling to put docker on my machine so I'm only able to use it in codespaces right now which is a linux machine, so testing on windows locally is not an option unfortunately.

[johnoel]

The run directory is where you execute the compiled program ss3 with all of the model input files (ie ss3.dat). There is an alternative to copying the executable, I will test and post later.

Also, I have added the docker_opt target to build ss_opt with docker.

[johnoel]

Below is alternative command to avoid copying the executable to the run directory. Instead of copying, the ~/stock-synthesis directory with the compiled ss3 program is mounted in the Docker container. The default application (--entrypoint) is reset to call the /stock-synthesis/ss3 in the ss-run work directory. This was tested using MacOS.

~ % cd ~/ss-run
ss-run % docker run --rm --volume ~/stock-synthesis:/stock-synthesis --volume $PWD:/ss-run --workdir /ss-run --entrypoint /stock-synthesis/ss johnoel/admb:linux

[johnoel]

See Quickstart for Windows.