Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use reconstructed ListBlobs marker to provide list offset support in MicrosoftAzure store #6173

Open
andrebsguedes opened this issue Aug 1, 2024 · 1 comment · May be fixed by #6174
Open
Labels
enhancement Any new improvement worthy of a entry in the changelog

Comments

@andrebsguedes
Copy link
Contributor

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
BlobStorage currently does not expose a way to start at a specific location on ListBlobs requests, this makes implementing GC-like routines that lists millions of objects and have to outlive a single process very cumbersome.

Describe the solution you'd like
There is a workaround for this limitation which involves reconstructing the marker token used for list pagination to specify a start key where listing should begin.

Describe alternatives you've considered
There are no efficient alternatives given that ListBlobs only supports prefix filtering.

@andrebsguedes andrebsguedes added the enhancement Any new improvement worthy of a entry in the changelog label Aug 1, 2024
@tustvold
Copy link
Contributor

tustvold commented Aug 1, 2024

Potentially related #5653

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
2 participants