-
-
Notifications
You must be signed in to change notification settings - Fork 739
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SIGINT / Ctrl-C handling #8155
Comments
What result do you expect by using CTRL-C while the process is running? |
Ctrl-C is a request to finish nicely. In the Borg 1.2 release notes:
This commit message adds support for it. Also to carefully wrap up the current deletion, if that's what it's doing. This normally happens well, but there are occasions, mostly seen with new archives, that crash like the above. For reference, this is the normal interrupt output of a small test (replaced 'clear line' char with 'newline', so it's readable):
This process can take a few dozen seconds sometimes. |
That https://github.com/python/cpython/blob/3.9/Lib/signal.py#L39 |
Wondering why we see |
I can absolutely guarantee that's not the case. Here's my history: I normally ctrl-c like a maniac. That's because often I abort shell scripts or loops with subprocesses. So, the first time I used Borg I did that too. This triggered a problem like this, and then I started investigating how Borg deals with that. Then I found it was documented behavior that one ctrl-c aborts gracefully, and two is harder. I've been using that method regularly. But, it seems that under certain conditions, and it looks like it has to do with it being a new archive or no checkpoint being present yet, this crash happens? |
I tried to reproduce with borg 1.2.7 (built from git tag) on macOS. I used a |
@halfgaar Sure, I believe you. Maybe there is a bug somewhere. The generator it complains about is in the I am trying to reproduce now on a ubuntu system, let's see if that behaves differently. I doubt that it has to do with the presence of checkpoints. borg always starts a fresh archive when using |
OK, tried on Ubuntu Mantic (23.10) using Python 3.11.6 and borg 1.2.7 (from git tag) with a ssh: repo (with and without prior stuff in it): it also behaved as it should, not tracebacks. I sshed into the ubuntu machine, so my ctrl-c keypress was transmitted via ssh to the borg client (not on a local text console or a gui terminal). I see you always used python 3.9.18 on 2 different ubuntu versions - can you retry with some more recent python? Or just use Ubuntu's system python3? Also, what's the borg version on the repo side? In my case it was both the same 1.2.7 version (although the traceback does not look like the remote borg version should have an influence on that). |
Another try: On Ubuntu Mantic (23.10) using Python 3.11.6 and borg 1.2.7 - this time installed using |
@halfgaar Do you use a borg fat binary downloaded from github releases page? If so, please give me the link to the binary you use and the sha256sum of the binary on your system. |
https://github.com/borgbackup/borg/releases/download/1.2.7/borg-linux64 I tried to reproduce using that file and interestingly I sometimes got "Keyboard interrupt" (looks like the default python handler) and sometimes "Got Ctrl-C" (borg's checkpoint-saving handler), never a traceback though. |
OK, so I guess this happens when one uses a pyinstaller binary (which creates a bootloader process and a borg process, both being in same process group):
So the borg process ends up getting SIGINT twice. |
About adding
|
I the fat binary, sha256 hash I completely forgot about the fact that my target server is Debian 12, with Python 3.11.2. My source machines are stock Ubuntu, with Python Python 3.8.16 and Python 3.10.12. Does the 3.9.18 come from the fat binary? Here's another different result:
I've been trying to reproduce again, but now I can't. I don't know what I'm doing differently. But I see you may have a culprit. |
Yes, the py39 comes bundled within the fat binary (that's how I noticed you likely use that). In my most recent experiment, I used the same binary as you use on your machine. Not sure yet how a good solution for this issue would look like. It could be avoided by not using that binary, because the cause is the binary's "bootloader" process. |
Guess a solution would involve not calling If then a duplicate SIGINT comes in a microsecond later, our handler won't be called any more and it'll terminate the process like if we pressed Ctrl-C more than once. |
See #8162 - I tested it with a fresh pyi-made binary and it "debounced" successfully! |
Sounds great. I'll be busy/gone for a while the coming days so won't be able to test more, but it looks like it's all done practically. |
fix Ctrl-C / SIGINT behaviour for pyinstaller-made binaries, fixes #8155 (1.2-maint)
fix Ctrl-C / SIGINT behaviour for pyinstaller-made binaries, fixes #8155
fix Ctrl-C / SIGINT behaviour for pyinstaller-made binaries, fixes #8155 (master)
I've had this with Borg 1.2.6 and now with 1.2.7 again. Interrupting it (on the initial, non-finished backup) results in a crash and even a crash in a crash. I did ctrl-c once, not multiple times.
Ubuntu 20.04 with Borg 1.2.6:
This is Ubuntu 22.04 with Borg 1.2.7:
And another variant on Ubuntu 22.04 with Borg 1.2.7:
Systems are plain ext4, backing up over SSH to encrypted and unencrypted repo.
The text was updated successfully, but these errors were encountered: