feat: Add DynamoDB based locking to prevent race conditions in index.yaml generation #447
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…yaml generation I have set this up so it is sufficiently abstracted that other lock backends could be added in the future. Fixes hypnoglow#18
|
In this screenshot you can see it working:
|
|
This would also be really helpful for my org, we're going to try to deploy these changes to our CI from your branch. Do you have any additional information that would be helpful to know? Possibly any documentation you may have recorded regarding how to set this up and configure the dynamo backend? |
|
Just an update to confirm that I have been able to test this and can confirm that it is working for me as well. Is there anything we can do to expedite getting this merged? |
|
@hypnoglow can we please expedite this? we've been suffering from this issue for a long time |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have set this up so it is sufficiently abstracted that other lock backends could be added in the future.
Fixes #18
Important Note:
Because of where I've placed the locking it prevents breaking the index.yaml but if a lock timeout occurs the chart still gets uploaded just not added to the index.yaml
This means that follow up uploads need to use the
--forceflag or still require ahelm s3 reindexto solve. I could acquire the lock for the entirety of the push operation which would prevent this but that increases the chance to hit these lock timeouts so wasn't sure which compromise you would prefer.