Porting TCP/IP stack to embedded IoT devices

Developers generally believe that since a communication protocol stack is called a TCP/IP stack, porting it to an embedded target provides the target with all TCP/IP functionalities and performance. This is far from true.

A TCP/IP stack requires such resources as sockets and buffers to achieve its goal. These resources, however, consume RAM—a scarce resource on an embedded target. Deprived of sufficient resources, a TCP/IP stack will not work better than a RS-232 connection.

When performance is not an issue and the primary requirements are connectivity and functionality, implementing TCP/IP on a target with scarce resources (RAM and CPU) is an option. Today, however, when an Ethernet port is available on a device, expectations are that performance will be in the order of Megabits per second. This is achievable on small embedded devices, although certain design rules need to be observed.

By using a Transport Control Protocol (TCP) example, this article demonstrates design rules to be considered when porting a TCP/IP stack to an embedded device.

Network buffers
A TCP/IP stack places received packets in network buffers to be processed by the upper protocol layers and also places data to send in network buffers for transmission. Network buffers are data structure defined in RAM.

A buffer contains a header portion used by the protocol stack. This header provides information regarding the contents of the buffer. The data portion contains data that has either been received by the Network Interface Card (NIC) and thus will be processed by the stack, or data that is destined for transmission by the NIC.

The data portion of the network buffer contains the protocol data and protocol headers. For example:

The maximum network buffer size is determined by the maximum size of the data that can be transported by the networking technology used. Today, Ethernet is the ubiquitous networking technology used for Local Area Networks (LANs).

Originally, Ethernet standards defined the maximum frame size as 1518B. Removing the Ethernet, IP and TCP encapsulation data, this leaves a maximum of 1460B for the TCP segment. A segment is the data structure used to encapsulate TCP data. Carrying an Ethernet frame in one of the TCP/IP stack network buffers requires network buffers of approximately 1600B each. The difference between the Ethernet maximum frame size and the network buffer size is the space required for the network buffer metadata.

It is possible to use smaller Network buffers. For example, if the application is not streaming multimedia data but rather transferring small sensor data periodically, it is possible to use smaller network buffers than the maximum allowed.

TCP segment size is negotiated between the two devices that are establishing a logical connection. It is known as the Maximum Segment Size (MSS). An embedded system could take advantage of this protocol capability. On an embedded target with 32K RAM, when you account for the all the middleware RAM usage, there is not much left for network buffers!

Network operations
Many networking operations affect system performance. For example, network buffers are not released as soon as their task is completed. Within the TCP acknowledgment process, a TCP segment is kept until its reception is acknowledged by the receiving device. If it is not acknowledged within a certain timeframe, the segment is retransmitted and kept again.

If a system has a limited number of network buffers, network congestion (packets being dropped) will affect the usage of these buffers and the total system performance. When all the network buffers are assigned to packets (being transmitted, retransmitted or acknowledging received packets), the TCP/IP stack will slow down while it waits for available resources before resuming a specific function.