SSH auth failure deploying from GitLab CI

[christopher-b]

I'm having trouble getting an mrsk deploy to work from a GitLab CI job. The SSH authentication fails. SSH connection to the server works from the host, but not from the mrsk container, despite having the ssh-agent socket volume-mounted to the container. I have the public key added to authorized_keys on the server. How can I continue to troubleshoot this? Thank you!

GitLab CI Config:

image: docker:20.10.16
services:
  - docker:20.10.16-dind
before_script:
  - eval $(ssh-agent -s)
  - mkdir -p ~/.ssh
  - echo "$SSH_KEY" | tr -d '\r' | ssh-add -
  - echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
  - chmod 700 ~/.ssh
  - chmod 600 ~/.ssh/known_hosts

deploy:
  script:
    - alias mrsk="docker run --rm -t -v '${PWD}:/workdir' -v '${SSH_AUTH_SOCK}:/ssh-agent' -v /var/run/docker.sock:/var/run/docker.sock -e 'SSH_AUTH_SOCK=/ssh-agent' ghcr.io/mrsked/mrsk:latest"
    - mrsk version
    - ssh -T user@host
    - mrsk deploy

Output:

$ eval $(ssh-agent -s)
Agent pid 22

$ echo "$SSH_KEY" | tr -d '\r' | ssh-add -
Identity added: (stdin) (user@host)

$ mrsk version
0.12.1

$ ssh -T [email protected]
... connects successfully

$ mrsk deploy -v
Acquiring the deploy lock
 INFO[...] Running [/usr/bin/env mkdir mrsk_lock && echo "..." > mrsk_lock/details on host
[ERROR (Net::SSH::AuthenticationFailed): Authentication failed for user user@host

And the server SSH log:

Connection from [client IP] port 53392 on [server IP] port 22
Client protocol version 2.0; client software version Ruby/Net::SSH_7.1.0 x86_64-linux-musl
no match: Ruby/Net::SSH_7.1.0 x86_64-linux-musl
Local version string SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.7
permanently_set_uid: 110/65534 [preauth]
list_hostkey_types: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
SSH2_MSG_KEXINIT sent [preauth]
SSH2_MSG_KEXINIT received [preauth]
kex: algorithm: ecdh-sha2-nistp521 [preauth]
kex: host key algorithm: ssh-ed25519 [preauth]
kex: client->server cipher: aes256-ctr MAC: [email protected] compression: none [preauth]
kex: server->client cipher: aes256-ctr MAC: [email protected] compression: none [preauth]
expecting SSH2_MSG_KEX_ECDH_INIT [preauth]
rekey after 4294967296 blocks [preauth]
SSH2_MSG_NEWKEYS sent [preauth]
expecting SSH2_MSG_NEWKEYS [preauth]
SSH2_MSG_NEWKEYS received [preauth]
rekey after 4294967296 blocks [preauth]
KEX done [preauth]
Connection closed by [client IP] port 53392 [preauth]
ssh audit_event: unhandled event 12

[lgastler]

More or less exactly the same issue here, also trying to get it to work as a GitLab CI job, seems we have done about the same debugging. Maybe connected to #218.