-
Notifications
You must be signed in to change notification settings - Fork 470
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[State CosmosDb] Built-in compression of value
#3376
Comments
Hey @olitomlinson Thanks for raising this issue. We had plans to raise it ourselves but got a bit wrapped up in work and day-to-day. We would absolutely welcome this change. We were going to raise under dapr/dapr against actors as the usual state management is already base64 encoded while actor state is raw, formatted JSON. Having it as a configurable flag on the component YAML actually feels like a neater solution than what we had planned. We were hoping for an overload on the actor state management API. With it on the YAML, the team will be able to set it once during project bootstrapping as a best practice and ensure costs don't ever grow wildly beyond their expected range. I like it! Giving it more thought, it makes sense to be an option for all forms of state management and for any consumption-based database, not just CosmosDB. This same behaviour would apply neatly to DynamoDB as well, for example. |
Thanks @olitomlinson for detailed information on the request. I raised the issue under dapr/dapr yesterday. |
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions. |
Bump |
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions. |
Bump |
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions. |
Keep alive |
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions. |
Keep alive |
Do you envision this being something where the developer can specify one of multiple built-in compression algorithms or did you have one in mind that you think would be a better fit than others? |
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions. |
This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as pinned, good first issue, help wanted or triaged/resolved. Thank you for your contributions. |
Sad to see this close but I can understand why this wouldn't be included in Dapr. For those looking for more details, we've detailed these changes on the Cosmos DB state management docs: https://docs.dapr.io/reference/components-reference/supported-state-stores/setup-azure-cosmosdb/#optimizing-cosmos-db-for-cost-savings |
This customer blog-post highlights a very simple and warranted need to compress state before it hits CosmosDb.
The customer rolled their own client side compression solution for this, but I think the CosmosDb component should include its own turn-key compression implementation to achieve the same goal.
This behavior could be enabled via a flag on the spec
enableValueCompression
Pros
Cons
value
would no longer be adocument
but astring
, meaning thequery API
capability would not be possible.Thinking out loud...
Given that Postgres V2 has dropped support for
query API
due to changing the way it stores thevalue
data- I do not see this as a controversial move to make for CosmosDb. Maybe it makes sense to leave the CosmosDb v1 component alone, and implement this compression feature as the standard behavior for CosmosDb V2?@KrylixZA Maybe you could provide some thoughts on if this would feature/change would be welcome to you and how a turnkey solution provided by the component would simplify your client-side concerns.
The text was updated successfully, but these errors were encountered: