-
Notifications
You must be signed in to change notification settings - Fork 2
Shared Memory
Shared memory is a method for processes to talk to each other (more technically, shared memory is a form of interprocess communication (IPC)). I don't usually prefer drawing analogies to explain things, but in this case I think the following the analogy is helpful:
If you think about normal communication, you often think about phones, text messages, radios; these are all forms of communication that rely on there being some sort of "sending" end and some sort of "receiving" end. These forms of communication more closely resemble things like pipes and sockets. That's not what shared memory is.
Shared memory is more like a Google doc. In order to use a Google doc, somebody has to create it first, and they need to give everyone who intends to view and edit the document permissions to view and edit the document. Once that's done and all the editors have opened the document, whenever one person makes a change, it is immediately viewable by everyone else. The actual information that is stored in the Google doc is not stored on any individual editor's computer, however; that information is stored in the cloud. When any editor that has the Google doc open on their computer, what they are really viewing on their screens is connected, or mapped, to the data on the cloud. That's how one editor's changes are able to be propagated to all other editors instantly—since only one real copy of the data exists.
Likewise, shared memory blocks (documents) behave very similarly. In order to use a shared memory block, a process must create it first, and name it. Each process that wishes to use that shared memory block must know the name of that shared memory block in order to read and write to that block. Once all processes have opened the block, whenever one process modifies the contents of the block, that modification is immediately made available to all other processes. The reason this is possible is that the contents of the shared memory block is not stored in the memory space associated with any individual process; it is stored in special "common memory locations" by the operating system itself. What each process really is reading and writing to is a set of addresses in their own process spaces that are mapped to that special shared memory block allocated by the operating system. This is how one process' modifications are able to be propagated to all other processes instantly—since only one real copy of the data exists (the "common" block).
This concludes the analogy.
A point to note is that when a process is terminated, it needs to ensure that its mappings to the shared memory block are closed. When the shared memory block is no longer needed, the shared memory block needs to be unlinked from its name, so that the operating system can reclaim that space once all processes have disconnected from that shared memory block. If this doesn't to happen, any new attempts to bind the same name to a newly created piece of shared memory will fail (with often disastrous consequences, depending on your system you may need to reboot your computer).
Lastly, if you're reading this because you're working towards understanding the shared memory wrapper, then the wiki page on Semaphores is also a must-read because semaphores are what prevent multiple processes have accessing shared memory at the same time, and are thus vital to ensuring that these shared memory blocks do not become corrupted.
Once you have an understanding of what shared memory is and how it works, the interface for working with them is a manageable size (5 functions). Below is a brief overview of what each of these functions does; more information can be found on their man pages.
int shm_open(const char *name, int oflag, mode_t mode);
This function opens a block of shared memory with the specified name, opening options, and access mode (if the shared memory block is being created). The name of the shared memory block starts with a / and usually has -shm at the end by convention (see shm_wrapper.h for many examples). The oflag argument is the bitwise OR of various file opening flags (O_CREAT, O_RDONLY, O_WRONLY, O_RDWR, etc.) In Runtime, when the shared memory block is created, we specify O_RDWR | O_CREAT; when we're just opening it, we just have O_RDWR. If oflag contains O_CREAT, the mode argument sets the permissions on the shared memory block (usually 0660, which is read and write permissions for user and group, no permissions for other; see additional resources for more information); otherwise, the mode argument is ignored and is conventionally set to 0. The function returns a file descriptor for the newly opened shared memory block.
void *mmap(void *addr, size_t len, int prot, int flags, int fildes, off_t offset);
This function maps the memory located at a given file descriptor into the virtual address space of the calling process (essentially establishes the link between the process and, in our case, the shared memory block that we're interested in). There are a lot of arguments here, but lets break them down:
-
addr: according to themanpage, the returned address is "an implementation-defined function of theaddrparameter. Don't worry about this argument; for all intents and purposes, it will beNULL. -
len: this is the length, in bytes, of the memory-mapped region. If the shared memory block is the size of somestruct, supposestruct foo, this would be equal tosizeof(struct foo). -
prot: this indicates what the process shall be able to do to the mapped data. In Runtime, since each process needs to be able to read and write to the shared memory blocks, this is equal toPROT_READ | PROT_WRITE, indicating that the process can read and write to the shared memory block. -
flags: this indicates what kind of file is being mapped (you can map other things aside from shared memory, such as normal files and semaphores). In Runtime, since we're only using this function on shared memory, this argument is alwaysMAP_SHARED. -
offset: this indicates how many bytes past the beginning of the page in memory the memory-mapped block shall start. For all intents in purposes, this argument will be 0; you only need to specify a non-zero value for this on very niche applications or extremely small systems where every byte of memory used matters.
The return value of this function is the address of the beginning of the shared memory block in each process's memory; in other words, once the memory allocated by the operating system has been mapped onto the process' own address space, writing to the addresses between the return value and the return value + the specified len argument will cause those modifications to be propagated to the memory block allocated by the operating system, and thus to any other processes that have also opened this shared memory block.
int munmap(void *addr, size_t len);
This function unmaps the len bytes starting at the address addr from whatever it was mapped to before. This, in shared memory, is kind of the equivalent of a "disconnect" or "close".
int ftruncate(int fildes, off_t length);
This function truncates the file represented by the given file descriptor (filedes) to a length of length bytes. In Runtime, this is used to set the size of the shared memory blocks when they're created in the operating system, with filedes being equal to the file descriptor returned by shm_open().
int shm_unlink(const char *name);
This function unlinks the shared memory block with the specified name. It tells the operating system to reclaim the memory that was used by that shared memory block when all processes that have the specified shared memory block mapped and open calls munmap() on each of their own copy of the shared memory block. As a result, calling shm_unlink on the specified name results in the specified name to be able to be reused.
POSIX shared memory is the newest implementation of shared memory on most Linux systems. The older System V shared memory used randomly generated keys to open and close various blocks of shared memory, and was slower than the POSIX shared memory. Therefore, we chose to use POSIX shared memory for simplicity and reliability. It is still interesting to read about the differences between the two shared memory interfaces and attempt to program using the System V shared memory, however.
Needless to say, shared memory is used exclusively in the shared memory wrapper to manage communication between the three core Runtime processes (net_handler, dev_handler, and executor). Read more about exactly how it is used in the Shared Memory Wrapper wiki.
- Important
- Advanced/Specific