Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Sugarscape IG - polars with numba vectorized loop #91

Draft
wants to merge 35 commits into
base: main
Choose a base branch
from

Conversation

adamamer20
Copy link
Collaborator

@adamamer20 adamamer20 commented Sep 1, 2024

This PR adds a numba implementation of the loop in SugarscapePolars to get the best moves for each of the agent, avoiding the (slightly) more complex reasoning of the DF Loop.

Some observations so far:

  • Looping DF is still the fastest, followed by numba and the completely iterative (non vectorized) process (see polars_comparison.png)
  • Working with numba is a bit tedious as:
    • you have to specify types and sizes of vectors
    • must have an input vector which is the size of the output
    • must use 1d vectors
  • Most of the simulation time is spent on join operations, outside of getting the best moves, mostly on getting the neighborhood and setting the cells (which could be sped up with a lazy approach and a faster backend overall). Results in the flame graph polars_comparison.svg

I am still trying to get the cuda target to work, will update.
Also the non vectorized loop could potentially be faster if you iterate directly on the polars DF but the point was to check how much worse non vectorization would have been.

@adamamer20 adamamer20 linked an issue Sep 1, 2024 that may be closed by this pull request
@adamamer20 adamamer20 marked this pull request as draft September 1, 2024 14:38
Copy link

codecov bot commented Sep 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

…o 67-sugarscape-instantaneous-growback-polars-with-numba
…hborhood. For both numba and completely vectorized it's easier to reason this way then update the current sugar and "best moves" ranking when agents move
…ght make the same move and haven't found the optimal move yet). This avoids race conditions.
…ssary since we prepare the neighborhood looking at potential/max sugar anyway)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SugarScape Instantaneous Growback (Polars-With-Numba)
2 participants