Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline keydir #4

Open
dgrnbrg opened this issue Dec 23, 2013 · 1 comment
Open

Pipeline keydir #4

dgrnbrg opened this issue Dec 23, 2013 · 1 comment

Comments

@dgrnbrg
Copy link
Contributor

dgrnbrg commented Dec 23, 2013

The goal here is to pipeline put/alter in the keydir. alters and puts can run without waiting for the IO to complete, since each alter only needs to have the result of the previous alter visible to it. To implement this, we must have a "future file", which is an in-memory copy of the latest uncommitted puts and alters. Then, all gets will actually ready from the future file. When IO operations complete, they will delete the data from the future file so that reads will proceed normally to the main keydir chm. There's a potential slowdown case here, in that most reads will first miss in reading from the future file; however, given that those reads need to do IO anyway, maybe it doesn't matter, or maybe the reads can be done in parallel if it is a performance issue.

@dgrnbrg
Copy link
Contributor Author

dgrnbrg commented Jan 29, 2014

core.async's message passing overhead is 0.2% of the runtime of get on my machinev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant