Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move global ID map into the shards #1751

Open
ljeub-pometry opened this issue Sep 6, 2024 · 0 comments
Open

Move global ID map into the shards #1751

ljeub-pometry opened this issue Sep 6, 2024 · 0 comments
Labels
Rust Core Features/Fixes within the core of Raphtory

Comments

@ljeub-pometry
Copy link
Collaborator

With the new bulk data frame loaders, a significant amount of time is spend waiting on the DashMap locks.

If we make the shard assignment depend on the (hashed) global id instead of local id, we can move the global-to-local id mapping inside of the shards which would allow to use the scanning approach for the node id resolution.

@ljeub-pometry ljeub-pometry added the Rust Core Features/Fixes within the core of Raphtory label Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Rust Core Features/Fixes within the core of Raphtory
Projects
None yet
Development

No branches or pull requests

1 participant