Skip to content
This repository has been archived by the owner on May 15, 2024. It is now read-only.

Last pack job should get prepared after a certain duration #169

Closed
Tracked by #139
xinaxu opened this issue Oct 18, 2023 · 1 comment · Fixed by #178
Closed
Tracked by #139

Last pack job should get prepared after a certain duration #169

xinaxu opened this issue Oct 18, 2023 · 1 comment · Fixed by #178
Assignees
Labels
beta Feature for Beta release P0

Comments

@xinaxu
Copy link
Collaborator

xinaxu commented Oct 18, 2023

In the current implementation, only when the size of a pack job is more than a threshold (default 16GB), it will be prepared by the dataset worker.

This means if the pack job never reaches 16GB, it will never get prepared, which can happen if the client has less than 16GB data or their last chunk of data is less than 16GB and they are no longer pushing new blobs.

The fix is to pack remaining files after a certain duration (24h, hardcoded or configurable)

@xinaxu xinaxu added bug Something isn't working beta Feature for Beta release labels Oct 18, 2023
@xinaxu xinaxu self-assigned this Oct 19, 2023
@masih masih added the P0 label Oct 20, 2023
@xinaxu xinaxu removed their assignment Oct 20, 2023
@xinaxu xinaxu removed the bug Something isn't working label Oct 20, 2023
@xinaxu xinaxu changed the title Retrieval does not work when blob is sealed but deleted in Motion Last pack job should get prepared after a certain duration Oct 20, 2023
@xinaxu
Copy link
Collaborator Author

xinaxu commented Oct 26, 2023

#178

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
beta Feature for Beta release P0
Projects
Status: Done
3 participants