Executor

Executor

This executor's jobs is to understand the student API and execute the student code using the corresponding Runtime functions. This process is probably the most delicate and had the most design changes as it has to accommodate any type of student code. To understand the executor properly, you need to first understand the Python C API and our Student API.

Details

This process will read the current mode from the shm_wrapper which will be supplied by the net_handler. Then whenever the mode changes to AUTO or TELEOP, the main function will create a new subprocess to run the new mode, using fork and run_mode. If mode changes to IDLE, it will kill the previously running process using kill_subprocess. To avoid any robot safety concerns, we must also reset the robot parameters on IDLE, which is done with reset_params. The main function handling the mode changes will loop forever and is only cancellable by sending a SIGINT signal.

To actually have the student code call the student API functions, we need to insert the API functions into the student module's namespace. This is done in executor_init where the Python interpreter is initialized. The insertion is done by setting the student code's attributes Robot and Gamepad to the corresponding attributes in the student API.

Finally, run_mode will call the <mode>_setup and the <mode>_main Python functions using run_py_function. These functions will end up calling the Python C API located in "Python.h" to run the Python functions in the given student code, which is assumed to be in studentcode.py. You will note that there are several error checks during the running of the Python function. We have to safeguard for 2 things:

1. If a Python exception occurred in the action thread but not the main thread, it won't interrupt the main thread's execution. Thus, we have an event set in the Robot class in studentapi.pyx whenever an action exception happens. We continuously check this event status in run_py_function to know if we need to prematurely end the Python function execution.
2. If the run mode changes and so the current robot code stops execution, and additionally the main thread is in a Robot.sleep call, we need to specifically tell the main thread to interrupt its sleep. This is since the main Python code is run in the same thread as the C code in executor.c and so we have to kill the Python's sleep function, again by using an event in Robot that we set inside run_py_function.

More can be read about the Python C API at our wiki.

Caveats

One thing to note which can cause unwanted interactions is that the executor process is now linked dynamically, and so it shares the symbols for the shared memory and logger with the studentapi.pyx file. This means that if you initialize it once on executor.c, you don't need to initialize it in studentapi.pyx.

Another important caveat to be aware of is Python's global interpreter lock (GIL). The Python interpreter can run in only 1 thread at a time and which thread is running is the one that acquires the GIL. As a result, whenever any Python C API functions need to be called, you must always be careful to acquire the GIL first and then release it when you are done. To make it simpler, we designed the architecture such that each mode runs in a separate process instead of a separate thread, which avoids any issue with the Python interpreter state.

Design Choices

Threads vs Processes

The first design of executor used threads for every new mode that was run, but this causes several issues with the GIL not being locked/unlocked properly and with the interpreter state not being reset between each mode. Thus, we moved to processes which fixed both of these issues since the mode subprocess only has 1 thread and since when the process dies, all the variables state are deleted by the OS then recreated for the new process.