Deprecate all schedulers and be free to do whatever you want

[EwoutH]

Update: View the migration guide on how to replace the deprecated Schedulers with AgentSet functionality.

Mesa currently utilizes a time module and various schedulers to manage agent activation and time progression within simulations. However, for each specific order and selection of agents a new type of scheduler needs to be created. This is highly restrictive and doesn't allow really interesting and complex, conditional interactions.

Recent developments and proposed changes allow moving away from fixed schedulers. In this discussion I want to talk about if we shouldn't just deprecate them all, and embracing a more direct and efficient approach to agent activation and time management, inspired by the NetLogo framework.

The old and the new

Mesa's existing architecture includes a time module that provides different schedulers: BaseScheduler, RandomActivation, SimultaneousActivation, StagedActivation, RandomActivationByType, and DiscreteEventScheduler. These schedulers offer various strategies for activating agents (e.g., in a fixed order, randomly, or simultaneously) and managing time progression in the simulation.

However, we already built a lot of the prerequisites to move away from fixed schedulers:

• With Native support for multiple agent types #1894 we track all agents in the Model itself.
• With Add select_agents method to Model #1911 we can select a subset of agents from the Model.

If we add a ask_agents() method and a "step" counter to the model itself, we're basically there.

Replacing current schedulers

The proposal to deprecate schedulers in the Mesa library and replace them with more direct methods of selecting agents and executing actions can be illustrated through several examples. These examples demonstrate how the model.select_agents and model.do_each methods can effectively replace the functionality of various schedulers.

1. Replacing RandomActivation Scheduler

Current Approach with RandomActivation

In the existing system, the RandomActivation scheduler activates each agent once per step, in a random order. An example of its usage might be:

class MyModel(Model):
    def __init__(self):
        self.schedule = RandomActivation(self)
        # ... (agent initialization)

    def step(self):
        self.schedule.step()  # Activates each agent in a random order

New Approach

Using model.select_agents and model.do_each, the same functionality can be achieved as follows:

class MyModel(Model):
    def step(self):
        random_agents = self.select_agents()
        self.ask_agents("step", agent_keys=random_agents)

2. Replacing SimultaneousActivation Scheduler

Current Approach with SimultaneousActivation

The SimultaneousActivation scheduler is used when agents need to first calculate their actions (step) and then update their state (advance) simultaneously:

class MyModel(Model):
    def __init__(self):
        self.schedule = SimultaneousActivation(self)
        # ... (agent initialization)

    def step(self):
        self.schedule.step()  # First calls 'step', then 'advance' on each agent

New Approach

This can be replicated as follows:

class MyModel(Model):
    def step(self):
        self.ask_agents("step")
        self.ask_agents("advance")

3. Replacing StagedActivation Scheduler

Current Approach with StagedActivation

StagedActivation allows for multi-stage activation, where agents execute different stages in a defined order:

class MyModel(Model):
    def __init__(self):
        self.schedule = StagedActivation(self, stage_list=["move", "eat", "sleep"])
        # ... (agent initialization)

    def step(self):
        self.schedule.step()  # Executes "move", "eat", "sleep" for each agent in order

New Approach

The same can be done with the new method:

class MyModel(Model):
    def step(self):
        self.ask_agents("move")
        self.ask_agents("eat")
        self.ask_agents("sleep")

4. Replacing RandomActivationByType Scheduler

Current Approach with RandomActivationByType

This scheduler activates each type of agent in a random order:

class MyModel(Model):
    def __init__(self):
        self.schedule = RandomActivationByType(self)
        # ... (agent initialization by type)

    def step(self):
        self.schedule.step()  # Randomly activates each type of agent

New Approach

Using the new approach, different agent types can be activated as follows:

class MyModel(Model):
    def step(self):
        for agent_type in [AgentType1, AgentType2]:
            agents_of_type = self.select_agents(type=agent_type)
            self.ask_agents("step", agent_keys=agents_of_type)

Be free, free to do whatever you want

The proposed transition to using model.select_agents and model.do_each in the Mesa library offers unprecedented flexibility and enables a wide range of new behaviors that were not easily achievable with the traditional scheduler system. Below are several examples illustrating this enhanced flexibility and the potential for innovative agent behaviors:

1. Conditional Activation Based on Dynamic Criteria

New Behavior

Activate agents based on dynamic, complex conditions that can change each step.

Example

class MyModel(Model):
    def step(self):
        # Activate agents based on a dynamic condition, e.g., a threshold value that changes every step
        threshold = self.compute_threshold()  # Compute a new threshold every step
        agents_above_threshold = self.select_agents(condition=lambda a: a.value > threshold)
        self.ask_agents("react_to_threshold", agent_keys=agents_above_threshold)

2. Sequential Group Activation

New Behavior

Activate groups of agents sequentially, where the activation of one group depends on the state of another.

Example

class MyModel(Model):
    def step(self):
        # First, activate one group of agents
        self.ask_agents("plan_action", agent_keys=self.select_agents(type=PlannerAgent))

        # Then, based on the outcome of the first group, activate another group
        if self.some_condition_met_by_planners():
            self.ask_agents("execute_plan", agent_keys=self.select_agents(type=ExecutorAgent))

3. Staggered or Time-Shifted Activation

New Behavior

Activate agents at different intervals, not necessarily in every step.

Example

class MyModel(Model):
    def step(self):
        # Activate a set of agents every 2 steps
        if self.schedule.steps % 2 == 0:
            self.ask_agents("bi_step_action", agent_keys=self.select_agents(type=SpecialAgent))

        # Regular activation for all other agents
        self.ask_agents("regular_action")

4. Interleaved and Prioritized Actions

New Behavior

Interleave actions of different agent types or prioritize certain actions over others.

Example

class MyModel(Model):
    def step(self):
        # Interleave actions between two types of agents
        for agent_type in [AgentTypeA, AgentTypeB]:
            self.ask_agents("interleaved_action", agent_keys=self.select_agents(type=agent_type))

        # Prioritize urgent actions
        urgent_agents = self.select_agents(condition=lambda a: a.is_urgent())
        self.ask_agents("urgent_action", agent_keys=urgent_agents)

5. Complex Interaction Patterns

New Behavior

Facilitate complex interactions between agents based on multiple criteria, such as proximity, state, or history.

Example

class MyModel(Model):
    def step(self):
        # Activate agents to interact based on proximity and a specific state
        for agent in self.agents:
            nearby_agents = self.get_nearby_agents(agent)
            relevant_agents = [a for a in nearby_agents if a.state == 'ready']
            for other_agent in relevant_agents:
                agent.interact_with(other_agent)

Let's discuss

Let's discuss if this is the approach we want to go! I think it's great, because it feels really Pythonic.

Explicit is better than implicit.

By explicitly defining what happens in the step, you are more in the flow and in control of what happens.

Flat is better than nested.

Currently, the behavior is hidden away in a Time module. Let's bring it out in the open.

Readability counts.

The examples read like the best programming language in the world: English

If the implementation is hard to explain, it's a bad idea.

I feel some of our schedulers are a bit like this. I think the new syntax is way easier to comprehend.

[Corvince]

That's super interesting! I thought about something similar some time ago, but without do_each and select_agents it seemed a bit unwieldy for complex schedulers. But now seems like a good time!

Looks like we can do more with less

[wang-boyu]

Is there a way to perform uniform activation, or activate agents in a user-defined, deterministic order? For example, users may want to sort by agent ids and activate them one-by-one in this order.

[EwoutH]

Great question. In my opinion we give do_each() a sorting option, which can be anything we like, including unique_id.

Alternatively, you could use the sorting capability in #1911 to do select agents like:

selected_agents = model.select_agents(sort=['unique_id'], direction=['lowest'])  # Return all agents sorted by unique_id from low to high
model.do_each("step", selected_agents)  # Run step for all the agents without shuffling

[wang-boyu]

Probably it's still better to have a sort option to do_each(), even though it's possible to sort when selecting agents?

For example,

self.select_agents()  # could be sorted or not
# users may do something to agents and destroy order unintentionally
...
self.do_each("step")  # default is random activation 
self.do_each("step", sort="age")  # activate by age
self.do_each("step", sort=lambda x: x.some_attr)  # activate by whatever lambda function, similar to how `sorted` key works

[wang-boyu]

I'm also thinking about some options to do_each() to specify activation order such as

self.do_each("step", activation="random")
self.do_each("step", activation="simultaneous")

This can probably be somehow merged with the sort option (or not).

[EwoutH]

I was thinking of a shuffle boolean variable, that shuffles the order of agents if true.

I don't think activation="simultaneous" it's the way to go, since it could be split up into individual steps as in the example.

class MyModel(Model):
    def step(self):
        self.do_each("step")
        self.do_each("advance")

We could make a seperate sort_agents() method and let both select_agents() and do_each() call that internally.

Maybe do_each() could be called ask_agents(), also a nice little shout-out to NetLogo.

[quaquel]

I like the cleanliness of the resulting code. I have some random thoughts to add to the discussion.

First, do_each is an operation on a set of agents. So, I would at least consider having an AgentSet class. select_agents would then return an AgentSet. do_each would be a method on the AgentSet class. Likewise, you could handle the ordering within this set through additional methods (e.g., shuffle, sort, or additional selections to yield subsets).

If you don't have an AgentSet class, do_each would require quite some documentation to cover all the possible ways it can be used and the many possible paths through the code. For example, what is the behavior if you don't pass a set of agents as a keyword argument? And if you pass a set of agents, would do_each still perform some operation on the order, or would you explicitly need to control this? From an advanced user point of view, the full control is amazing. For the novice user, it can become quickly overwhelming.

From a logical point of view, you need to (1) select the agents, (2) impose some order (so sort and activation are, in my view, the same thing), and (3) activate them. Therefore, I recommend separating this into three steps while providing some default behavior or a separate default method that covers the most basic use case (i.e., random activation of all agents).

[quaquel]

with chaining, you could do agentset.sort('wealth').select(n=5). The nice thing is that the order of method calls also becomes flexible.

[Corvince]

I was just about to say that! Yes that's awesome how flexible this approach is

[jofmi]

Hi @EwoutH ! Great to see that Mesa is developing so well, I will have to revisit it the next time I build an ABM! In general, if there are any parts of AgentPy (like the sequences module) that you find useful, feel free to use the code - it's open-source after all :)

For the AgentSet:

• I would keep in mind that it should be easy to switch between different containers, e.g. set and list.
• I like @quaquel suggestion to return the set. I'd add that functions should have a copy attribute to define whether the function is applied to the original set or to a copy of it.
• A big drawback for me was that sets in Python were unordered (but maybe this is not the case anymore). This is why I often used an ordered dict instead of a set: https://github.com/jofmi/agentpy/blob/603693ba6bcfd8603a057bc38005b1bd2e80c85a/agentpy/sequences.py#L318C34-L318C34

[quaquel]

@jofmi good points. Building on those

1. I like the idea of having an inplace or copy keyword argument on AgentSet methods. If inplace=True or copy=False, you would update the internal representation of the agents. Otherwise, you return a new AgentSet.
2. If the internal representation of the agents is again the horrible dict of dicts with value None (i.e., the dirty hack for Python's lack of an OrderedSet), you would maintain insertion order. Updating the internal order should be possible but might require a bit of a Python deep dive, or the creation of a new dict that overwrites the protected _agent attribute
3. By maintaining an order through the dict of dict trick, you can have the best of both lists and sets. In fact, you might even support indexing: agents[5]. So, there would not be a need to have both AgentSet and AgentList.
4. Adding an internal _agent property makes it easy to use weakrefs to avoid possible memory leaks.

@EwoutH If you want help with the draft implementation, let me know, or we can discuss it in person.

[EwoutH]

I just want to say how nice it is to think together about this. Especially the velocity with which we now make progress is really remarkable. Prime example of great open source development. Thanks everyone!

@jofmi Thanks for joining the conversation, and great insights! I think we have to make an design decision if we A) want to include a (default) inplace parameter in each method, or that B) we want users to just use copy() explicitly. A built-in to_list() method to AgentSet could help in the cases that you explicitly want a list instead.

If any of you have the time, please write up why an OrderedSet would be useful to have in Python stdlib. The main discussion is the proportionality, so more use cases shift that in the direction of beint proportional to include: https://discuss.python.org/t/add-orderedset-to-stdlib/12730

@quaquel I invited you as collaborator to my fork. If you want you can make modifications to the draft PR branch directly: https://github.com/EwoutH/mesa/tree/AgentSet (by the way, @Corvince, as a maintainer, you should also be able to do that). Go wild, we can always create more branches / draft PRs.

[Corvince]

Something that poped up in my mind this morning is that without any scheduler users will have to manually track the number of steps. That is right now the number of steps is stored in model.schedule.steps and for example this is used by the datacollector. Its also just nice to keep track of how many steps happened. In this new approach one way would be to have steps as a model attribute and make sure that in the model.step function it is incremented. Not the best solution imho. Maybe there is a way to automatically track the number of times model.step() is called? Any other ideas?

[quaquel]

Tracking the number of calls to model.step() should be quite straightforward.

However, I am still not convinced of the desirability of removing schedulers altogether. From a teaching perspective, schedulers have proven to be quite useful. Moreover, there likely are many models already using schedulers that would have to be rewritten if schedulers are fully dropped.

So, yesterday, I suggested to @EwoutH to reimplement the existing schedulers on top of AgentSet. Users can still leverage the many new possibilities offered by interacting with AgentSet directly for a more advanced control flow. It might even be explained in the documentation somewhere as the preferred future approach.

Also, in #1893, it is suggested to add Discrete event scheduling. If that idea is built upon, you could reintroduce ABM style scheduling (so schedule model.step() for n fixed time steps) as part of that.

[EwoutH]

In my head this was one of the easier problems to solve, so I didn't really gave it much thought yet. But now I'm playing with it, it might be a little more complicated than I thought, especially because you want it to be consistent and out of the way for users that don't want to modify it, but also flexible and straightforward to modify for those who do.

Some of the current schedulers track both steps and time. Maybe we can let steps be - by definition - the number of times Model.step() is called. Then time is equal to steps by default, but we provide an easy way to modify / increase it.

Another thought I have was giving Model.step() a timestep argument. Default would be timestep=1, but you can change that (once or every step if you want).

We might want to get some discrete event people in here though, they might have useful insights on how to approach this and support their use cases.

[EwoutH]

I'm also still thinking about the pre-fab existing schedulers. I'm leaning against deprecating them at some point, since I still think most of PEP 20 falls in favor of the new approach (see discussion start above).

And then there is:

There should be one-- and preferably only one --obvious way to do it.

I think the obvious way are the example above. What I'm afraid of, it that by only thinking about RandomActivation and StagedActivation, users and students stay ignorant and don't think about what it actually means to call a scheduler. That's a comfort, but by steering to define by hand you actually understand what the scheduler is doing under the hood.

Once you learn to think in that way, it becomes way easier to talk about hybrid approaches to agent activation.

From a teaching perspective, schedulers have proven to be quite useful.

I do see this value. But I think the value is mainly conceptual. The value is in the "design patterns" of agent activation, like random, staged, simultaneous. That should be your default starting point. But implementation wise, it's just activating a bunch of agents. And if you make that explicit in implementation, students learn to think about it from multiple perspectives. That's almost always better for understanding and retention.

TL;DR: I think teaching value can be even higher by learning them to think about agent activation design patterns and how and when to deviate from those, than learning them a few fixed tricks.

[quaquel]

Agent based modeling is a particular type of Discrete Event Simulation. "A discrete-event simulation (DES) models the operation of a system as a sequence of events in time. Each event occurs at a particular instant in time and marks a change of state in the system". At the core of DES is the event list. This is a time sorted list of events to be executed at a given instant in time.

In case of ABM, there are only fixed time intervals, or ticks, between sets of events. Since the invocation of the step methon on an agent is an event, the current schedulers in MESA (perhaps with the exception of DiscreteEventScheduler) are not proper discrete event schedulers that schedule events on an event list. Rather, they are ways of sorting the set of events to be executed at a given instant of time (i.e., a tick). Here the set of events is the invokation of a step method on each agent in the simulation. MESA lacks an eventlist. Instead, it is up to the user to advance the eventlist by one tick at a time by calling model.step(). Because of this, there is now the problem that time becomes undefined if all schedulers are removed.

So, one possible way forward would be to add an event list to MESA, and build various forms of event scheduling and event sorting. This would enable various forms of hybrid ABM-DES models to be built with MESA. Moreover, something like a PoissonArrivalScheduler, could be more easily expressed in this setup. If you have a poisson arrival process, the time between arrivals is exponentially distributed. So, you can now just schedule the arrival events using some kind of AgentGenerator.

A key question, however, is what the scope of MESA should be. If you want to keep MESA focussed on purely ABM, the event list might not be required. However, in that case, I would be hesitant with removing the current schedulers. Because if you do everything through AgentSets, it is easy to create nonsene models where agents are activated multiple times within the same tick.