Introduction to device drivers - Part 2
A modular stack

Like the rest of Syllable, the network stack in Syllable is modular. It is comprised of four parts:

  1. The system libraries (i.e. LibC) which present a BSD sockets API to applications.
  2. The upper part of the network stack, which provided TCP, UDP, IP, ICMP and ARP functionality.
  3. The interface layer, which handles the generic code appropriate for the type of network interface in use.
  4. The device driver.

The first two of these layers are a pretty standard network stack. The third layer allows Syllable to support different types of network connection easily. For ethernet connections Syllable uses the eth_if interface driver. This handles common tasks, such as ethernet encapsulation. It then passes these ethernet frames down to the device driver to send to the hardware. It also recieves ethernet frames from the device driver and passes the IP packets to the upper layers.

As an example of another interface driver, PPP for Syllable includes a ppp_if driver, which handles communication between the network stack and pppd, the PPP deamon.

Just like device drivers, interface drivers are ELF Dynamic Shared Objects (DSOs) Unlike device drivers, they are always loaded when the kernel boots, rather than on-demand.

The full stack then, looks like this:

+-------------------+ | Sockets API | +-------------------+ | ==========Kernel========= | +-------------------+ | TCP/UDP/IP/ARP | +-------------------+ | =====Interface driver==== | +-------------------+ | Framing | +-------------------+ | ======Device driver====== | +-------------------+ | Hardware | +-------------------+ | V *_=*_=*_=*_=*_=*_=*_=*_=*_=*_=* _=* _=* _=* NETWORK _=* _=* _=* *_=*_=*_=*_=*_=*_=*_=*_=*_=*_=*
Driver interface

As we know, system libraries and applications communicate with device drivers via. the device node in /dev. For ethernet drivers this is only part of the full picture. Ethernet drivers are a special case. Because ethernet frames can and do arrive at any time and most ethernet controllers only have a small buffer to store these packets, they must be handled as quickly as possible to avoid overflowing the incoming packet buffer and losing data.

This means that the interface driver can not simply call read() to retrieve data from the driver: with a busy network connection, data would be arriving too quickly for the interface driver to keep up. Instead, the driver "pushes" data to the interface driver as each frame arrives, which ensures packets are handled as quickly as possible.

Note that the reverse is not true. Because the hardware is generally much quicker than the network stack, the interface driver can call write() to send data to the driver without any problems. This leads to one of the most confusing aspects of ethernet drivers on Syllable: the interface driver will call open(), close(), ioctl() and write() but not read(). If you are studying an ethernet driver for the first time it may not be immediatly clear how data arriving from the connection gets to the network stack.

Device nodes

How does the interface driver know which devices are available? When it is loaded by the kernel, eth_if looks for device nodes under /dev/net/eth/. The drivers create one node for every interface they have detected, in this directory. For example, on a machine with a single Intel EEPro 100 network card the device node " /dev/net/eth/eepro100-0" exists. If the machine had two Intel EEPro network devices, the driver would also create " /dev/net/eth/eepro100-1"

Packet buffers

Data to be sent and received by the driver are stored in a packet buffer. This is represented by the PacketBuf_s structure, which is defined in the system header <net/packet.h>. A single PacketBuf_s structure keeps the packet and its associated data together all the way through the network stack.

An ethernet driver can treat the PacketBuf_s data as an opaque "blob" of data. All it must deal with is the raw data inside the packet buffer, which is a complete ethernet frame.

The system header <net/net.h> provides some functions for dealing with packet buffers:

PacketBuf_s* alloc_pkt_buffer( int nSize ); void free_pkt_buffer( PacketBuf_s* psBuf );

These functions do exactly what you would expect. alloc_pkt_buffer() creates a new, empty packet buffer and free_pkt_buffer() destroys a previously allocated packet buffer and any packet data that it contains.

Each network interface also has a net queue. The system header <net/packet.h> defines the NetQueue_s structure. A net queue is used to store outgoing and incoming frames as they are passed between the interface driver and the device driver. The device driver does not need to do anything with the net queue itself other than to tell the interface driver which net queue a newly arrived frame should be placed in. We will cover this in more detail later.

Data in, data out

To understand how everything fits together, we'll now look at how data passes through a typical ethernet driver. First of all we will consider how data is sent to the network, and then how data arrives from the network.

We can ignore the work that is done by the IP stack and sockets API; from the point of view of the driver these layers are irrelevant. The only part the driver needs to know about is the interface layer.

As data works its way down through the layers it will eventually arrive at the interface layer, where it becomes individual ethernet frames. Each of these frames are stored in individual packet buffers. For each frame, the interface driver calls write() to pass the PacketBuf_s structure to the driver. The driver then copies the frame into the cards internal transmit (Tx) buffer, performs some house keeping tasks such as updating internal counters, pointers and lists, and then returns. The frame is held in the cards Tx buffer until the hardware is ready to transmit it.

When a frame arrives from the network, the hardware will store it in an internal recieve (Rx) buffer and raises an interrupt to signal to the driver that a new frame has arrived and requires attention. The drivers interrupt handler runs. The driver creates a new PacketBuf_s structure and copies the new frame into it. It must then inform the interface layer about the new frame. The system header <net/net.h> declares the following function:

void enqueue_packet( NetQueue_s* psQueue, PacketBuf_s* psBuf );

This simple function adds the new packet buffer to the net queue, which the interface layer will process afterwards. The NetQueue_s pointer is provided to the device driver by the interface layer when the interface is initialised. We will cover this in more detail in chapter 3.

Once the drivers interupt handler has queue the new frame for the interface layer, it performs some house keeping tasks such as updating internal counters, pointers and lists, and then finishes.

This is what any ethernet driver will do 95% of the time. Despite the differences in the hardware almost every single ethernet driver is structured in a very similiar way, with the aim of making sending and recieving frames as efficient as possible. Once you are familiar with how one ethernet driver works you will find it very easy to understand almost any other ethernet driver.