Network Handler

Table of Contents

• 1. Network Handler
  • 1.1. Structure
  • 1.2. Details
    • 1.2.1. Protobufs
    • 1.2.2. Message Types
    • 1.2.3. TCP
    • 1.2.4. Events vs Poll Thread
      • 1.2.4.1. Events Thread
      • 1.2.4.2. Poll Thread

1. Network Handler

The network handler's job is to communicate with Dawn and Shepherd through sockets and turn the network messages into shared memory updates.

1.1. Structure

• protos/ - contains the Protobuf definitions of all our network messages. This is a submodule.
• pbc_gen/ - the corresponding C code that is generated from the protobuf definitions using protobuf-c. This is not pushed to the repo.
• net_handler.c - main process file that accepts TCP connections and starts communication with Dawn/Shepherd.
• net_util.c - various helper functions and constants (ex. port numbers, message type numbers) that are used in net_handler
• connection.c - handles the TCP connections with Dawn and Shepherd. Used by net_handler.c
• message.c - helper functions used by connection.c to send and receive different messages

1.2. Details

Runtime communicates with each client (Dawn/Shepherd) using a TCP network connection. For more information on TCP connections and the OS object called a socket which is used for all networking communication, see the sockets wiki page.

1.2.1. Protobufs

To use sockets effectively, both ends need to use a common, agreed-upon messaging protocol. We use the well known Google Protobufs to convert our data into a stream of bytes. The idea is that all communication between . Then, for each , have an API to pack/unpack the messages. Since Google doesn't provide an API for C, we use the third-party protobuc-c library. Examples on how to use the library can be found in its Github wiki. The message definitions Runtime uses are defined in protos/. Each message type received will correspond to a message definition .proto file. To see all the message types and which client use them, see the message table below. This net handler planning document might also be useful.

1.2.2. Message Types

The message types are enumerated in net_util.h. The following table list the messages that are sent/received by Runtime from Dawn and Shepherd.

	RUN_MODE	START_POS	LOG	DEVICE_DATA	GAME_STATE	INPUTS
Received from Dawn (TCP)	Yes	Yes	No	No	No	Yes
Received from Shepherd (TCP)	Yes	Yes	No	No	Yes	No
Sent to Dawn (TCP)	No	No	Yes	Yes	No	No
Sent to Shepherd (TCP)	No	No	No	No	No	No

Next, we describe what data is received from Dawn/Shepherd with each of these messages, as well as what corresponding actions are taken by Runtime upon receiving messages of the given type:

• RUN_MODE: Sent by Dawn and Shepherd to put the robot into a specified run mode, such as TELEOP, AUTO, or IDLE. Runtime updates the RUN_MODE field in the robot description shared memory block by calling the shared memory wrapper function robot_desc_write()
• START_POS: Sent by Dawn and Shepherd to tell the robot which starting position the robot is in, perhaps so students can write different autonomous code for different starting position. This can change from year to year; current values that this can take on are generic "LEFT" and "RIGHT". Runtime updates the START_POS field in the robot description shared memory block by calling the shared memory wrapper function robot_desc_write()
• GAME_STATE: Sent by Shepherd to specify any game-specific states that require the robot to change its behavior when responding to student controls. The contents of this message is heavily game-dependent, and we expect this to be one of the parts of Runtime to change implementation each year. For spring 2021, it results in a call to the shared memory wrapper function robot_desc_write(), which activates some effects on the motor controls implemented by a gamestate_filter thread in executor
• INPUTS: Sent by Dawn to specify the state of the inputs the student is using to control the robot (either a Keyboard or a Gamepad). Runtime updates the input state by calling the shared memory wrapper function input_write()

Finally, we describe what data is sent to Dawn/Shepherd with each of these messages:

• LOG: Sent to Dawn for displaying in the Dawn console; these include Runtime errors, student code errors which generated Python tracebacks, as well as output from print statements in student code
• DEVICE_DATA: Sent to Dawn at a predetermined refresh rate (currently 20 Hz) so that Dawn can update their peripherals panel with the most recent reported state of the Lowcar devices currently attached to Runtime

1.2.3. TCP

TCP is a stream-based protocol, so there is no built-in way to determine when a message ends and another begins. As a result, we used a commonly used messaging scheme to delimit the messages. Each message has has the following structure:

	Message Type	Size of Payload	Payload
Size (bytes)	1	2	Determined by previous field

The message type is the enumerated integer that corresponds to the message contained in the payload. For example, a LOG message will have a message type of 3, as defined in an enum in net_util.h. The payload size is interpreted as a two-byte unsigned integer (uint16_t), and specifies the size of the remainder of the message (i.e. this size does not include the three-byte header). Two bytes is enough, since the range of uint16_t goes up to 65,535, which is much larger than any message that Runtime would need to send. Finally, the payload is the packed protobuf message.

To process the messages, we read the first byte to determine the message type. We then read two more bytes to determine how many more bytes to read to obtain the full, packed protobuf message. Then, we unpack the protobuf message using the protobuf-c unpacking function for the message type at hand. Once we have unpacked the message, we have obtained the contents of the received message, which allows Runtime to proceed to take the action called for by the received message.

To prepare messages for sending, we essentially do the above but in reverse. We pack the message using the protobuf-c packing function for the message type at hand. We then construct the three-byte header consisting of the message type in its first byte and the size of the packed message as its second and third bytes, then append the header to the packed protobuf, which is then ready to be sent on the TCP connection.

1.2.4. Events vs Poll Thread

For each client connection, we spawn an "events" thread and a "poll" thread. Each thread handles different types of messages.

1.2.4.1. Events Thread

The events thread waits on the Runtime log file descriptor and the client TCP socket file descriptor. Whenever the log file descriptor has something to read, we send a LOG to the client. Whenever the TCP socket has something to read, we unpack it and update shared memory accordingly.

1.2.4.2. Poll Thread

This thread sends messages to the client at a specified interval. For example, every so often we pack a DEVICE_DATA message and send it to the client (if it is Dawn).