Systems Overview

Unix Concepts

Runtime uses a lot of systems concepts, which is hopefully why you're here--to learn about systems! The following is a list of some various concepts that you'll be exposed to in Runtime, along with where they are used within Runtime. Keep in mind, this is not a list of prerequisite knowledge you need to have in order to work on Runtime!

• Files and I/O
  • Files are the most fundamental way that programs interact with the computer and change the state of their environments. Most, if not all Unix systems objects can be treated as files.
  • Runtime uses files and I/O extensively to communicate within itself and with Dawn and Shepherd
• Networking
  • Used to connect Runtime to Dawn and Shepherd
  • Transmission Control Protocol (TCP)
    • This networking standard for transmitting data between two machines on a network is useful when transferring packets of data needs to be reliably sent and received
    • In Runtime, used for Runtime to transmit all data between itself and Dawn or Shepherd
  • Sockets
    • These objects are used by computers to communicate with other computers on the network. A connection over a network is made when a socket on one machine and another socket somewhere else on the network begin sending information to each other
    • In Runtime, used to make connections with Dawn and Shepherd.
• Shared Memory
  • These special files can be used for different processes to communicate messages, especially structured messages. Shared Memory is the fastest type of interprocess communication (IPC)
  • In Runtime, it is used to enable the fast transfer of most data (especially the high-throughput data) through the system
• FIFO Pipes
  • These special files can be used for different processes to communicate messages, especially unstructured messages where the order in which the messages come is important
  • In Runtime, used to transfer logs between the logger and net_handler before those logs are sent to Dawn
• Serial Communication
  • Serial communication happens whenever two devices communicate information byte-by-byte (usually over a wired connection), with no protocol or breaking up of large messages into smaller packets when necessary
  • In Runtime, dev_handler and the lowcar devices communicate over serial connections
• Semaphores
  • Semaphores are operating-system-level objects in your computer that act sort of like a "gate" for processes to access certain data. If you have some data that is being shared between multiple processes, you don't want that data to be written to at the same time. Rather, you first ensure that every process that wants to write to that data "acquires the semaphore" before it is allowed to modify that shared data. Any other process that wishes to modify the data must then wait for the first process to finish writing and "release the semaphore" before the next process can acquire it and make its change.
  • In Runtime, semaphores are used in shm_wrapper to guard the access to the various blocks of shared memory.
• Thread Management
  • Threads are essentially ways to run different sections of code within the same process simultaneously. They are very useful for breaking up the work a process needs to do into several different, well-defined "parts"
  • In Runtime, they are used by all processes to divide up their tasks appropriately and prevent logic in one section of code from crashing or blocking logic in another section
• Process Management
  • Processes can also spawn and kill other processes. Doing this is useful when you want to enforce a separation between two processes, and creating a new thread is not enough of a separation; or when you want to run a separate piece of code and ensure that everything related to that process is stopped cleanly when it exits
  • In Runtime, they are used by executor to run the student code, to ensure that nothing that is done in student code can affect the dev_handler and make it crash. They're also used by the test clients to spawn Runtime processes that are used for testing.

C and POSIX Standard Library Headers

Runtime uses the folllowing C and POSIX standard library headers, in rough order from most commonly used to least commonly used (these all do not require additional installation before they can be used on any POSIX-compliant systems):

• stdio.h: for standard input/output (ex. printf)
• stdlib.h: C standard utility functions (ex. sleep, malloc, free)
• stdint.h: C standard integer types (ex. uint16_t, uint32_t)
• string.h: for string manipulation (ex. strcpy, memset)
• errno.h: for a bunch of error constants and functions (ex. EEXIST, EWOULDBLOCK, EINTR)
• pthread.h: for threading functions (ex. pthread_create, pthread_mutex_lock, pthread_cond_signal)
• unistd.h: for miscellaneous input/output functions and constants (ex. read, write, open, close, SEEK_END)
• signal.h: for signal handlers and signal constants (ex. signal, SIGINT)
• sys/time.h and time.h: for measuring time (ex. time_t, ctime)
• fcntl.h: for constants used in opening and closing files (ex. O_RDONLY, O_WRONLY)
• sys/stat.h: for working with unconventional files and system structures (ex. mkfifo, sem_t)
• arpa/inet.h: for socket functions (ex. inet_addr, bind, listen, accept, socket)
• netinet/in.h: for socket address structures (ex. struct sockaddr, struct sockaddr_in)
• semaphore.h: for semaphore functions (ex. sem_wait, sem_post)
• sys/mman.h: for memory mapping used in shared memory (ex. mmap)
• stdarg.h: for variable-length arguments used in logger (ex. va_args)
• sys/wait.h: for wait functions used in executor and the test clients to wait for processes (ex. waitpid)