Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snapshot sizes support? #703

Open
biji opened this issue Mar 27, 2022 · 14 comments
Open

Snapshot sizes support? #703

biji opened this issue Mar 27, 2022 · 14 comments
Labels

Comments

@biji
Copy link

biji commented Mar 27, 2022

Does snapper support showing snapshot sizes for btrfs?
I've read http://snapper.io/2018/10/04/used-space.html , still don't understand what to put in config: QGROUP

enable quota and rescan, causes lock up :

btrfs quota enable /
btrfs quota rescan /
# wait until rescan finished

# not responding:
snapper -c root --iso list

Thanks

@aschnell
Copy link
Member

From what I see that should work. Additionally, if querying the sizes fails the column is simply not shows, but there should be a response anyway.

What distributions are you using? And suspicious is dmesg, journtalctl or /var/log/snapper.log?

@biji
Copy link
Author

biji commented Mar 28, 2022

while doing snapper list, high cpu
image

dmesg:

[18387.092068] BTRFS warning (device nvme0n1p7): qgroup rescan is already in progress
[18434.101623] BTRFS warning (device nvme0n1p7): qgroup rescan is already in progress
[18440.735865] BTRFS warning (device nvme0n1p7): qgroup rescan is already in progress
[18476.714101] BTRFS info (device nvme0n1p7): qgroup scan completed (inconsistency flag cleared)

I'm using fedora 35

I think snapper trigger rescan again , everytime i run snapper list
Yes, there is a response with sizes column if i waited longer

probably related to #692

@aschnell
Copy link
Member

Yes, a rescan is triggered very time - it is so far the only way to get the values right.

@biji
Copy link
Author

biji commented Apr 5, 2022

Is it necessary after qgroup scan completed (inconsistency flag cleared)?

@aschnell
Copy link
Member

aschnell commented Apr 5, 2022

Right after a qgroup scan is should not be needed but snapper does not know that a qgroup scan was just done.

Simply looking for the inconsistency flag did not work back when I implemented the display of those values in snapper.

Anyway, if the qgroup rescan takes that long it looks like a kernel or filesystem problem to me (unless the filesystem is very big, has lots of snapshots, ...).

@pa-sowa
Copy link
Contributor

pa-sowa commented May 30, 2022

On my system with 938 GiB already allocated, qgroup scan takes 13 minutes, this makes snapper list unusable.

@aschnell
Copy link
Member

If it is too slow you can use the option --disable-used-space.

@pa-sowa
Copy link
Contributor

pa-sowa commented May 31, 2022

Yes, the problem is I would like to see the used space information, even if it is slightly incorrect (I prefer that instead of waiting 13 mins for the exact numbers). I started working on --no-quota-rescan option for the list command.

@biji
Copy link
Author

biji commented May 31, 2022

i think after qgroup scan completed , qgroup will be updated in realtime

@pa-sowa
Copy link
Contributor

pa-sowa commented May 31, 2022

i think after qgroup scan completed , qgroup will be updated in realtime

The problem is snapper always triggers a rescan. So every time I invoke snapper list I have to wait 13 minutes.

@thomwiggers
Copy link

The Arch wiki states that qgroups might be slightly out of date until sync is done; doing qgroup scan each time, which trashes all known numbers and re-computes them from scratch, seems overkill.

Additionally, the implementation currently first rescans, then syncs the filesystem. This also seems like doubling up the work.

@blubberdiblub
Copy link

@thomwiggers

Snapper should probably do the ioctl equivalent of btrfs filesystem sync /path/to/.snapshots/<no>/snapshot or btrfs qgroup show --sync /path/to/.snapshots/<no>/snapshot before reading the qgroup properties in order to not unnecessarily sync unrelated devices, filesystems and subvolumes.

(sync works globally for the entire system as opposed to the btrfs-specific ioctls, which are per-subvolume, and the former doesn't guarantee that all btrfs-specific metadata processing and update of counters is completed when the sync completes, as it is only documented/specified to ensure that dirty application-visible file data and file metadata (in a broad sense) ends up safely on the storage device)

@patphobos
Copy link

Hi, it seems we're encountering the same issue. The snapper list command, via the dbus service, appears to trigger BTRFS_IOC_QUOTA_RESCAN every time it runs, even if no new snapshots have been created.

The filesystem isn't particularly large: 17 TB, with 234,954 files and 12,676 folders. Currently, there are only 5 snapshots, and the refresh time is relatively long, about 15 minutes. However, I assume this behavior is normal and expected.

It's unfortunate that I'm considering repackaging snapper with quota support disabled so that the client isn't disrupted by quota refreshes. Instead, I'd monitor snapshot usage using btrfs qgroup show...

@ilkka-ollakka
Copy link

Yes, a rescan is triggered very time - it is so far the only way to get the values right.

Is this still the case? For me it seems that btrfs fi sync path is only needed after quota is enabled to get right values with kernel 6.x.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

7 participants