-
Notifications
You must be signed in to change notification settings - Fork 2
Processes
On UNIX systems, every program that is running is its own process. A process on a computer can be thought of as the program that is running and everything that it needs in order to sustain itself. To be technical, every process has its own process space that no other process can touch, which contains memory (divided into four sections: static, heap, stack, and code), environment variables, and system code (to tell the kernel to do things; the kernel is a small set of processes on your computer that manages all of the other processes on your computer).
Every process must be started by some other process, except the special init process that is the first process started up when you boot up your computer. Another way of saying that is that every process on your system has a parent (except init).
For example, when you open the "terminal" application on your computer, that terminal is a process. When you type a command, such as ls, cd, etc., what happens is that the terminal (or, to be more precise, the shell running in your terminal) forks a new process and becomes its new parent, and then tells this new process (often called the child) to execute the command that you typed. That process runs and prints some stuff to the screen, and then it terminates.
The same thing happens when you execute and executable from the terminal; the shell will fork itself and tell the child process to become the executable that you specified. The child process will then run the executable (and the effect is that it looks like your program is running in the terminal... which, in a sense, it is).
How does a process spawn a new process? Generally, this happens in two steps:
- The existing process calls the
fork()function, which does exactly what it sounds like it does: it splits a process into two processes. Both are running the same code and control is at the same location; the point is that where there was one process, there are now two (almost) identical processes. More on the details in the "Functions Used When Working With Processes" section. This step is called the "fork" step. - In the child process that results after the call to
fork(), there is a call to one of the sixexec()functions. These functions all essentially do the same thing (and differ only in the method that they accomplish it), which is to replace the program running in the child process (recall that currently the child and the parent are both running the same program still) with a new program specified as an argument to theexec()function. When this new program begins, it starts running at the beginning of itsmain()function, just like it was called from a terminal. This step is called the "exec" step.
So, in short, the way a new process running another program is created from an existing process is that a new process is fork-ed from the existing process and then the child is exec-ed.
When a process is spawned, here are two sets of parameters that the parent process can use to control the child process:
The first is the argument list, a.k.a. command-line arguments. For example, if you type a command like ls -ial, the -ial part is the command-line argument to the ls program. In a shell, the command-line arguments are separated by spaces and are passed into the program as an array of strings (that's why the function signature of main is int main(int argc, char **argv)). But if you want to create a process from another process in a program that you're writing, you must also supply the command-line arguments in the function that starts the other process.
The second set of parameters is the environment list a.k.a. process environment variables. These are (generally) variables with names in all caps that are (generally) defined by the system. For example, the PATH environment variables contains a list of directories that the shell should search for in order to find system executables (for example, the executable for the ls, cd, etc. commands discussed earlier) and is usually a string that looks something like:
PATH=/usr/bin:/usr/local/bin:/usr/sbin:/bin:/sbin
Another pretty useful one is the PWD environment variable, which is the Path to the current Working Directory. When an existing process spawns a new process, the new process inherits the environment variables of the existing process. If the existing process wants to add, modify, or delete items from the environment variables of the child process, it must do so using some special system functions between the two steps in spawning a new process. This makes sense; when a process is forked, the two processes are nearly identical copies, and both contain the same environment variables. In the child process, we write some code to change the environment variables as necessary, and then the child process calls an exec function to replace itself with a new program, retaining the changes to the environment variables made before the exec.
If you're curious, you can do a printenv command in the terminal and it will output the current environment variables set in your shell process!
Lastly, every process has a unique Process ID (pid) which is a positive integer. Every process also has a Parent Process ID (ppid), but this is not unique across all processes (as one process can spawn multiple children, which will all have the same ppid). You can see the processes running on your system and their process IDs and parent process IDs with the command ps -ef (static view) or htop (or top if you don't have htop) which gives a dynamic (live) view.
This will be a relatively in-depth look at these functions, since the way most of them work is very subtle. For a broader picture of how they all fit together, see the above section.
The fork function has the following definition:
pid_t fork();
Pretty simple-looking, right? This function takes no arguments and returns a pid_t, which is a process ID (a "positive integer"). However, it is more complicated than that.
Recall from the previous section that this function, when called from an existing process, creates a nearly identical new process that is running the exact same program, with control starting at the same point as the parent process. That's true, but that means that fork() actually has two return values: one in the parent process, and one in the child process. If you think about it, this is REALLY smart, because if this were not the case and fork() returned the same thing in both processes, there would be no way for either process to know which was the child and which was the parent!
Now we explain the return values. In the parent process, fork() returns the process ID of the child process that was just forked. This allows the parent to send the child signals using the kill() function (explained in the wiki page on signals), or wait for the child to terminate using the waitpid() function (explained below, after the exec functions). In the child process, fork() returns 0. Since 0 is never a valid process ID, the child knows definitively that it is not the parent process. Notice that the child does not know who its parent is.
If fork() returns with a negative number, that means that it was not successful for some reason.
This means that, in most cases, the existing process that is trying to spawn a new process will have code that looks like this:
pid_t pid;
if ((pid = fork()) < 0) {
// error handling code
} else if (pid == 0) {
// code that executes in the child
} else { // pid > 0
// code that executes in the parent
}
Make sure that the above logic makes sense before proceeding!
Recall from the previous section that these six functions all accomplish the same thing: cause a process to replace the program that it is running with a new program with zero or more command-line arguments and an environment defined by some environment variables. They differ in the way that we specify these things to the system (the program to run, the command-line arguments, and the environment). With that being said, here are the definitions of the six exec functions:
int execl(const char *pathname, const char *arg, ... , (char *) NULL);
int execlp(const char *file, const char *arg, ... , (char *) NULL);
int execle(const char *pathname, const char *arg, ... , (char *) NULL, char *const envp[]);
int execv(const char *pathname, char *const argv[]);
int execvp(const char *file, char *const argv[]);
int execve(const char *pathname, char *const argv[], char *const envp[]);
The functions that have const char *pathname expect a fully-formed pathname (something like ../net_handler/net_handler or /usr/bin/ls, for example) to specify the program that is to be run. The functions that have const char *file can take pathnames but can also take things like ls, cd, grep, etc; the functions that use const char *file searches the directories listed in the PATH environment variable for a corresponding executable to use.
The first three functions (the ones with l in the name) use a variable-length list of strings to pass the command-line arguments to the specified process. The argument list MUST be terminated by a NULL pointer (to tell the exec function that it has reached the end of the argument list). For example, if you want to call the executable a.out with the arguments "hello", and "world", you would write:
execl("a.out", "a.out", "hello", "world", (char *) NULL);
which is exactly equivalent to typing
./a.out hello world
in the terminal (assuming no changes in the environment variables.
The other three functions (the ones with v in the name) use an array of strings to pass the command-line arguments. To do the exact same command as before, you might do something like:
char *argv[] = { "hello", "world" };
execv("a.out", argv);
These functions are useful when you don't know ahead of time how many command-line arguments you'll need to pass to the spawned process.
Lastly, the functions that have an e in them (execle and execve) allow us to specify an additional array of strings that has all of the environment variables in them (of the form VAR=value, just like you would see them in the output from printenv in the terminal). This is a really advanced thing to do and we don't use it in Runtime. The other four functions simply copy the existing environment variables into the new process.
The waitpid function has the following definition:
pid_t waitpid(pid_t pid, int *stat_loc, int options);
This function is relatively straightforward. The process that calls this function will block until the process with the specified pid terminates. The exit status of that process will be stored in the location pointed to by stat_loc, unless it is NULL, in which case the exit code is ignored. The return value of the waitpid is the process ID of the child process being waited on (for our purposes, it will always be equal to the provided argument). There are several options that can be specified in the third argument, but for our purposes, it will always be equal to 0 (default options).
If you've read the wiki page on threads, another way to think about this function is that is the process equivalent of pthread_join().
The putenv function has the following definition:
int putenv(char *string);
This function is also relatively straightforward. This will insert the specified string (of the form VAR=value) into the environment variable list of the current process. For example, if we wanted to insert the environment variable with name FOO and value "bar" into the environment variable list, we could write: putenv("FOO=bar"); in our code. When spawning a new process, if we need to modify the environment variables of the child process before the child process calls an exec function, we do so in the clause of the if statement resulting from the call to fork() that corresponds to the code that the child process is running, like so:
pid_t pid;
if ((pid = fork()) < 0) {
// error handling code
} else if (pid == 0) {
// code that executes in the child
putenv("FOO=bar");
...
// any other code that modifies or adds to the environment of the new process
execlp(<some arguments>); // doesn't have to be execlp, any of the exec functions
} else { // pid > 0
// code that executes in the parent
}
This construction will cause all of the environment modifications we make in that clause to persist in the child process after the call to the exec function, but the environment variables of the parent process remain the same as before the call to fork().
The getenv function has the following definition:
char *getenv(const char *name);
This function is used to fetch the value of the environment variable with the specified name. For example, if we had previously called putenv("FOO=bar");, then getenv("FOO") would return the string "bar".
Process management is not used in too many places in Runtime, but when it is, it is often confusing, which is why we wrote so much information about processes. That being said, here is where Runtime deals with spawning processes; read more about the specifics in the corresponding wiki pages!
Each time the student changes the run mode of the robot (IDLE, AUTO, TELEOP, CHALLENGE), executor spawns a new child process that is responsible for running the student code in that mode (after killing off any previous child process that was running a previous mode). There are tons and tons of insane checks and quirks about this, because in order to handle the API that we give to the students to program their robots, and the fact that their code is arbitrary and written in Python, the shutting down and spawning of a new process gets very complex. The executor wiki page is a great place to learn more about those quirks.
Each test client is responsible for spawning the associated Runtime process; net_handler_client spawns net_handler, executor_client spawns executor, etc. However, each client has its own quirks that are explained in detail in the Test Framework wiki page. For example, executor_client must modify some environment variables before spawning executor in order to work, and net_handler_client must do some work from the parent process in order to "connect" the client to the newly spawned process.
Admittedly, I used a physical book I had when I programmed most of this, and not the Internet. However, during this writing I did look at some of the man pages for some of these functions, and those are really helpful, as long as you understand the first section ("What are Processes?") thoroughly.
- Important
- Advanced/Specific