First download succeeds but all subsequent downloads fail with error "Couldn't get data/incomplete/..." #2660

stanifert · 2023-07-14T18:34:21Z

stanifert
Jul 14, 2023

Is there a pinned issue for this?

I have read the pinned issues and could not find my issue

Is there an existing or similar issue/discussion for this?

I have searched the existing issues
I have searched the existing discussions

Is there any comment in the documentation for this?

I have read the documentation, especially the FAQ and Troubleshooting parts

Is this related to a provider?

I have checked the provider repo for issues
My issue is NOT related to a provider

Are you using the latest release?

I am using the latest release

Have you tried using the dev branch latest?

I have tried using dev branch

Docker run config used

version: '3.3'
services:
    transmission-openvpn:
        cap_add:
            - NET_ADMIN
        volumes:
            - /Users/me/transmission:/config
            - /Volumes/samsung-t7/files:/data
        environment:
            - OPENVPN_PROVIDER=EXPRESSVPN
            - OPENVPN_USERNAME=xxxxx
            - OPENVPN_PASSWORD=xxxxx
            - LOCAL_NETWORK=192.168.0.0/16
        logging:
            driver: json-file
            options:
                max-size: 10m
        ports:
            - '9091:9091'
        image: haugene/transmission-openvpn

Current Behavior

I'm having an issue where the first torrent I download via the web UI succeeds without issue, but all subsequent download attempts fail with error:

Couldn't get '/data/incomplete/<file_name>.part' No such file or directory (2)

The failure occurs immediately after the torrent download begins. The first file has no issue reaching the destination (/data/completed/).

Expected Behavior

The up and running container should continue downloading torrents.

How have you tried to solve the problem?

I've ran through all of the debug options and did not receive any errors, all output matched what the article expects.
Running docker ps or dock container ls shows the container as active and healthy.
I'm able to connect to the container via docker exec -it <container_id> bash and verify my IP address is masked via curl ifconfig.me.
I can also verify the VPN is active via docker exec transmission-transmission-1 curl --silent "http://ipinfo.io/ip". No issues accessing the web UI via port:9091.
First download succeeds without a hitch, fast download and file ends up in the correct specified directory (/data/completed/).
I've verified the /Volumes/samsung-t7/files/completed and /Volumes/samsung-t7/files/incomplete folders exist at the correct paths, and are setup in the compose.yaml file. Also, the first downloaded file ends up in the correct place so the initial setup seems to at least be correct.
Completely uninstalled & re-installed Docker for macOS
Tried multiple different torrent magnet links to rule out bad torrent link. Mainly used the magnet links for Big Buck Bunny et. al. from here
Verified same behavior occurs when passing in torrent files directly rather than torrent/magnet urls only.
Verified that both Docker and Terminal have full disk access, does not seem to be a directory permissions issue (again, the first downloaded file is placed in the correct directory)

Log output

2023-07-14 11:50:22 transmission-transmission-openvpn-1  | Starting container with revision: 1103172c3288b7de681e2fb7f1378314f17f66cf
2023-07-14 11:50:22 transmission-transmission-openvpn-1  | TRANSMISSION_HOME is currently set to: /config/transmission-home
2023-07-14 11:50:23 transmission-transmission-openvpn-1  | Creating TUN device /dev/net/tun
2023-07-14 11:50:23 transmission-transmission-openvpn-1  | Using OpenVPN provider: EXPRESSVPN
2023-07-14 11:50:23 transmission-transmission-openvpn-1  | Running with VPN_CONFIG_SOURCE auto
2023-07-14 11:50:23 transmission-transmission-openvpn-1  | No bundled config script found for EXPRESSVPN. Defaulting to external config
2023-07-14 11:50:23 transmission-transmission-openvpn-1  | Downloading configs from https://github.com/haugene/vpn-configs-contrib/archive/main.zip into /tmp/tmp.BVCWd2hjys
2023-07-14 11:50:24 transmission-transmission-openvpn-1  | Extracting configs to /tmp/tmp.UDLnnTRc3R
2023-07-14 11:50:24 transmission-transmission-openvpn-1  | Found configs for EXPRESSVPN in /tmp/tmp.UDLnnTRc3R/vpn-configs-contrib-main/openvpn/expressvpn, will replace current content in /etc/openvpn/expressvpn
2023-07-14 11:50:24 transmission-transmission-openvpn-1  | Cleanup: deleting /tmp/tmp.BVCWd2hjys and /tmp/tmp.UDLnnTRc3R
2023-07-14 11:50:24 transmission-transmission-openvpn-1  | No VPN configuration provided. Using default.
2023-07-14 11:50:24 transmission-transmission-openvpn-1  | Modifying /etc/openvpn/expressvpn/default.ovpn for best behaviour in this container
2023-07-14 11:50:24 transmission-transmission-openvpn-1  | Modification: Point auth-user-pass option to the username/password file
2023-07-14 11:50:24 transmission-transmission-openvpn-1  | Modification: Change ca certificate path
2023-07-14 11:50:24 transmission-transmission-openvpn-1  | Modification: Change ping options
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | Modification: Update/set resolv-retry to 15 seconds
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | Modification: Change tls-crypt keyfile path
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | Modification: Set output verbosity to 3
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | Modification: Remap SIGUSR1 signal to SIGTERM, avoid OpenVPN restart loop
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | Modification: Updating status for config failure detection
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | Setting OpenVPN credentials...
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | adding route to local network 192.168.0.0/16 via 172.22.0.1 dev eth0
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 DEPRECATED OPTION: --cipher set to 'AES-256-CBC' but missing in --data-ciphers (AES-256-GCM:AES-128-GCM). Future OpenVPN version will ignore --cipher for cipher negotiations. Add 'AES-256-CBC' to --data-ciphers or change --cipher 'AES-256-CBC' to --data-ciphers-fallback 'AES-256-CBC' to silence this warning.
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 WARNING: --keysize is DEPRECATED and will be removed in OpenVPN 2.6
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 OpenVPN 2.5.5 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jul 14 2022
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 library versions: OpenSSL 3.0.2 15 Mar 2022, LZO 2.10
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 WARNING: --ns-cert-type is DEPRECATED.  Use --remote-cert-tls instead.
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 TCP/UDP: Preserving recently used remote address: [AF_INET]193.29.107.11:1195
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 Socket Buffers: R=[212992->425984] S=[212992->425984]
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 UDP link local: (not bound)
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 UDP link remote: [AF_INET]193.29.107.11:1195
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 TLS: Initial packet from [AF_INET]193.29.107.11:1195, sid=870b96d5 5f48bd3b
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 VERIFY OK: depth=1, C=VG, ST=BVI, O=ExpressVPN, OU=ExpressVPN, CN=ExpressVPN CA, [email protected]
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 VERIFY OK: nsCertType=SERVER
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 VERIFY X509NAME OK: C=VG, ST=BVI, O=ExpressVPN, OU=ExpressVPN, CN=Server-8801-0a, [email protected]
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 VERIFY OK: depth=0, C=VG, ST=BVI, O=ExpressVPN, OU=ExpressVPN, CN=Server-8801-0a, [email protected]
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 2048 bit RSA, signature: RSA-SHA256
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 [Server-8801-0a] Peer Connection Initiated with [AF_INET]193.29.107.11:1195
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-option DNS 10.173.0.1,comp-lzo no,route 10.173.0.1,topology net30,ping 10,ping-restart 60,ifconfig 10.173.1.38 10.173.1.37,peer-id 27,cipher AES-256-GCM'
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 OPTIONS IMPORT: timers and/or timeouts modified
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 OPTIONS IMPORT: compression parms modified
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 OPTIONS IMPORT: --ifconfig/up options modified
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 OPTIONS IMPORT: route options modified
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 OPTIONS IMPORT: peer-id set
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 OPTIONS IMPORT: adjusting link_mtu to 1629
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 OPTIONS IMPORT: data channel crypto options modified
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 Data Channel: using negotiated cipher 'AES-256-GCM'
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 NCP: overriding user-set keysize with default
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 net_route_v4_best_gw query: dst 0.0.0.0
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 net_route_v4_best_gw result: via 172.22.0.1 dev eth0
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 ROUTE_GATEWAY 172.22.0.1/255.255.0.0 IFACE=eth0 HWADDR=02:42:ac:16:00:02
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 TUN/TAP device tun0 opened
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 net_iface_mtu_set: mtu 1500 for tun0
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 net_iface_up: set tun0 up
2023-07-14 11:50:25 transmission-transmission-openvpn-1  | 2023-07-14 17:50:25 net_addr_ptp_v4_add: 10.173.1.38 peer 10.173.1.37 dev tun0
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | 2023-07-14 17:50:27 net_route_v4_add: 193.29.107.11/32 via 172.22.0.1 dev [NULL] table 0 metric -1
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | 2023-07-14 17:50:27 net_route_v4_add: 0.0.0.0/1 via 10.173.1.37 dev [NULL] table 0 metric -1
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | 2023-07-14 17:50:27 net_route_v4_add: 128.0.0.0/1 via 10.173.1.37 dev [NULL] table 0 metric -1
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | 2023-07-14 17:50:27 net_route_v4_add: 10.173.0.1/32 via 10.173.1.37 dev [NULL] table 0 metric -1
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | RTNETLINK answers: File exists
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Up script executed with device=tun0 ifconfig_local=10.173.1.38
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Updating TRANSMISSION_BIND_ADDRESS_IPV4 to the ip of tun0 : 10.173.1.38
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | 
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | -------------------------------------
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Transmission will run as
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | -------------------------------------
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | User name:   root
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | User uid:    0
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | User gid:    0
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | -------------------------------------
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | 
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Updating Transmission settings.json with values from env variables
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Attempting to use existing settings.json for Transmission
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Successfully used existing settings.json /config/transmission-home/settings.json
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Overriding bind-address-ipv4 because TRANSMISSION_BIND_ADDRESS_IPV4 is set to 10.173.1.38
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Overriding download-dir because TRANSMISSION_DOWNLOAD_DIR is set to /data/completed
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Overriding incomplete-dir because TRANSMISSION_INCOMPLETE_DIR is set to /data/incomplete
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Overriding rpc-password because TRANSMISSION_RPC_PASSWORD is set to [REDACTED]
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Overriding rpc-port because TRANSMISSION_RPC_PORT is set to 9091
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Overriding rpc-username because TRANSMISSION_RPC_USERNAME is set to 
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Overriding watch-dir because TRANSMISSION_WATCH_DIR is set to /data/watch
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | sed'ing True to true
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | STARTING TRANSMISSION
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | Transmission startup script complete.
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | 2023-07-14 17:50:27 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
2023-07-14 11:50:27 transmission-transmission-openvpn-1  | 2023-07-14 17:50:27 Initialization Sequence Completed

HW/SW Environment

- OS: macOS Ventura 13.4.1 (2018 Mac mini, intel)
- Docker: 4.21.1

Anything else?

I've noticed the RTNETLINK answers: File exists output in the above log quite a few times, but have not been able to resolve that from the FAQ or even determine if it's an issue. For the LOCAL_NETWORK variable, the current value of 192.168.0.0/16 seems to be the only one that works I think, otherwise the container fails to start. Gateway is 192.168.0.1 via running route get default | grep gateway.

Side note, I'm not totally confident I'm setting the container up correctly. Currently, I've set the container up via the compose.yaml file above & running docker compose up -d to start it. However, previously I've tried setting up via downloading the image directly in Docker Desktop, but I couldn't find out how to set that up with docker compose which seems much nicer to work with rather than running docker run + params each time.

I'm assuming I'm missing something or have set something up wrong as I am a lowly iOS dev with little web/network/sysadmin knowledge. Any advice or help is much appreciated!

Answered by pkishino

Jul 16, 2023

I transferred this to discussions as this is user setup miss configuration and not a container issue.
Yes, as far as I recall, in osx you need to run with privileged mode to get correct access to external drives

View full answer

stanifert · 2023-07-14T19:41:08Z

stanifert
Jul 14, 2023
Author

The mounted destination (=mapped to /data/ in the container) directory /Volumes/samsung-t7/files is an external SSD. I tried switching this to instead point to the local Downloads folder rather than an external drive, and so far that seems that it may have resolved the issue.

So perhaps the issue is related to using an external drive as a mounted volume? How can I fix this?

0 replies

stanifert · 2023-07-14T20:37:32Z

stanifert
Jul 14, 2023
Author

Running docker inspect --format='{{.HostConfig.Privileged}}' <container_id> revealed that I was not running the container in privileged mode. Adding privileged: true to my compose.yaml file seems to have fixed the issue & now has ability to place completed files into the specified directory on the external SSD. Turns out having full disk access/file permissions enabled wasn't enough alone, also need to be a privileged container as well in my case. Solved! 🎉

I will leave this post as-is for mods/contributors to decide whether to keep or not, I'm unable to mark this comment as an answer.

0 replies

stanifert · 2023-07-14T21:22:58Z

stanifert
Jul 14, 2023
Author

With the below config, I've all but confirmed this issue was caused by trying to send the completed downloads to a directory on an external drive. By instead passing a local directory on the machine (/Users/me/Desktop), I'm not seeing any issues after downloading nearly a dozen torrents.

So is this expected behavior? Is there a way to successfully pass a directory on an external SSD for /data?

version: '3.3'
services:
    transmission-openvpn:
        cap_add:
            - NET_ADMIN
        volumes:
            - /Users/me/transmission:/config
            - /Users/me/Desktop:/data
        environment:
            - OPENVPN_PROVIDER=EXPRESSVPN
            - OPENVPN_USERNAME=xxxxx
            - OPENVPN_PASSWORD=xxxxx
            - LOCAL_NETWORK=192.168.0.0/16
        logging:
            driver: json-file
            options:
                max-size: 10m
        ports:
            - '9091:9091'
        image: haugene/transmission-openvpn
        privileged: true

0 replies

pkishino · 2023-07-16T12:08:07Z

pkishino
Jul 16, 2023
Collaborator

I transferred this to discussions as this is user setup miss configuration and not a container issue.
Yes, as far as I recall, in osx you need to run with privileged mode to get correct access to external drives

1 reply

stanifert Jul 20, 2023
Author

Thank you!

stanifert · 2023-07-20T02:07:18Z

stanifert
Jul 20, 2023
Author

Solution I've ended up going with is to just choose a local download location and use an Automator folder action to automatically move the files to the SSD once added. Much easier

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

First download succeeds but all subsequent downloads fail with error "Couldn't get data/incomplete/..." #2660

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 5 comments 1 reply

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

First download succeeds but all subsequent downloads fail with error "Couldn't get data/incomplete/..." #2660

stanifert Jul 14, 2023

Is there a pinned issue for this?

Is there an existing or similar issue/discussion for this?

Is there any comment in the documentation for this?

Is this related to a provider?

Are you using the latest release?

Have you tried using the dev branch latest?

Docker run config used

Current Behavior

Expected Behavior

How have you tried to solve the problem?

Log output

HW/SW Environment

Anything else?

Replies: 5 comments · 1 reply

stanifert Jul 14, 2023 Author

stanifert Jul 14, 2023 Author

stanifert Jul 14, 2023 Author

pkishino Jul 16, 2023 Collaborator

stanifert Jul 20, 2023 Author

stanifert Jul 20, 2023 Author

stanifert
Jul 14, 2023

Replies: 5 comments 1 reply

stanifert
Jul 14, 2023
Author

stanifert
Jul 14, 2023
Author

stanifert
Jul 14, 2023
Author

pkishino
Jul 16, 2023
Collaborator

stanifert Jul 20, 2023
Author

stanifert
Jul 20, 2023
Author