We currently have the following functions implemented as stubs:
Most of the functionality in the rest of these functions relies on tg3_timer() operating correctly, so this is the next logical function to port.
The first thing to note is that we must change the function declaration to take a void* argument instead of unsigned long . This is another difference between timers on Linux and Syllable. As before, we also have to change the one instance of create timer from:
To the Syllable-esque:
Nothing too complicated there, of course.
tg3_timer() only calls one unimplemented function, tg3_periodic_fetch_stats() , so we'll also port that now. You can see how the driver looks here .
Once we've added the timer handling code, our driver begins to "come alive". When we test it now, we see additional output:
We can test that the timer code is working by removing the network cable:
And plugging it back in again:
So it seems the timer code is working, mostly. We have commented out one line of code:
We'll revisit this section of code later.
The next piece of functionality we'll add is code for handling Tx (Transmit). Three of the functions we already have stubs for, tg3_tx() , tg3_start_xmit() and tg3_start_xmit_dma_bug() are related to this code path. The best thing to do is start with tg3_tx() and see where that leads us.
Because we need to make quite a few changes, our first effort is rough, with large blocks of code commented out and enclosed in #if blocks. You can see how the tg3_start_xmit() function looks here .
The biggest change is that we have removed any code that deals with fragments, or "skb frags". Transmitting fragmented packets is not done by Syllable, so the PacketBuf_s we are transmitting will always contain a single complete frame. This makes the Syllable code a little easier, but does require a few changes to the code to make it behave as if the "last" fragment has been transmitted. Linux also has a few additional macros and inline functions that are used to extract various pieces of information from an skb:
On Syllable we simply access the members in the PacketBuf_s instance directly:
You can see which members are available in the PacketBuf_s structure by looking at its definition in the system header file <net/packet.h> .
It is a little hard to test the driver at this point, with only the Tx code implemented. We can add some additional debugging output to the Tx functions such as tg3_start_xmit() and ensure that they are being called correctly. The best way to test the driver is to connect the card to a dedicated network switch, preferably with one other machine connected to the same switch. We can then configure the network interface and attempt to ping the other machine: if the Tx code is working, we will be able to see the data that is being transmitted via the activity light on the network card and the switch. If this is not an option for you, you'll have to continue to port the Rx code to the driver and test both the Tx and Rx functions together. This is more difficult to debug, however.
When we connect our machine to a switch and test it the first time, it doesn't work. From the debug output it is clear that tg3_start_xmit() is called, but no traffic is seen on the switch. We clearly need to take a closer look at tg3_start_xmit() and the other Tx code to see if we have missed anything. Our second attempt can be seen here . We've changed the way we call tg3_set_txd() and also ensured that the line:
at the bottom of the function, is outside the #if block.
When we test this iteration, the driver appears to work. Traffic can be seen by the switch, which indicates that frames must be leaving the device.
At this point there is not much more we can do to test the driver. We could connect a second machine to the switch, set it to promiscuous mode and capture the packets that our test machine is sending to ensure they are correct, but that's really not necessary.
Before we finish up, we'd better clean up the code we have. We'll remove the code in the #if blocks and lines of code that are commented out. We'll also clean up some unused variables and some unused code, and fix a compiler warning. We'll also make similar changes to the tg3_start_xmit_dma_bug() function, which is used for certain cards. The result of this cleanup is seen in here . The driver as it stands now can be seen here .
We're almost there. The last piece of code implements the Rx codepath. We'll start with the function tg3_rx() , which is currently a stub. It isn't a large function and it relies on only one extra function, tg3_recycle_rx() , which is also small.
One important difference is the way that incoming frames are placed in the net queue and notified to the interface layer. In the Linux driver you will see the functions skb_put() and netif_receive_skb() . On Syllable, skb_put() is emulated to some degree by the system header linux_compat.h . netif_receive_skb() is replaced by enqueue_packet() .
In general, you use the following code instead of a simple call to skb_put() :
The code copies the raw ethernet frame out of the cards Rx buffer ( desc ) to the a new skb. The following code then pushes the skb to the interface layer:
This code is near universal across all of the network drivers for Syllable.
In theory, we now have a complete driver which can configure the hardware and transmit and receive packets. Now we can begin to test it properly.
The first thing we'll do is attempt to ping another machine on a network. This will generate a low volume of transmits and receives; exactly what we want to do at this point.
When we try the driver, it doesn't work. We know that tg3_start_xmit() is being called and data appears to be transmitted, but nothing seems to be received by the card. There could be a number of reasons for this.
After some investigation it turns out that the driver is not receiving any interrupts, and none of the interrupt handlers are being called. The obvious first thing to do is to check that the interrupt handling code is correct, and that the driver correctly calls request_irq() . We can already see from the kernel output that the driver does indeed request IRQ 11 when it is loaded, so we can be fairly certain that it is correct. None the less, we'll add some debug output to the interrupt handlers to see if they are being called.
After some debugging, it becomes apparent that the culprit is this block of code in the function tg3_reset_hw() :
If we look at __tg3_set_coalesce() in the Linux driver we see that it mentions interrupts:
So maybe it is required after all! We'll merge the functions tg3_init_coal() and __tg3_set_coalesce() into one function. Once we've done that and test the driver again, we have more success:
So the hardware is now generating interrupts when the frame is transmitted.
However we still do not appear to be receiving interrupts for Rx events. It seems incoming data is being ignored. Even if we hardwire __tg3_set_rx_mode() to put the hardware into "promiscuous" mode (i.e. receive any and all frames from the network, even if the frame is not addressed to us) the card still does not generate Rx interrupts.
Debugging interrupt problems is difficult. Debugging Rx interrupt problems can be very difficult as there is no sure-fire way to generate incoming frames in a reliable manner.
There are a number of things that could be causing the problem:
Number 1 can be tested by connecting two other machines to the network and pinging between themselves. Number 5 can probably be discounted because interrupts are now being raised for Tx completion events. We have already tested number 2 by forcing __tg3_set_rx_mode() to place the interface in promiscuous mode. That leaves only numbers 3 and 4, both of which are going to be hard to check.
This particular bug doesn't give us a logical starting point, so we're left with little choice but to examine the code line by line, starting with device_init() and comparing our version with the original. We'll look for pieces of code that may have been changed when porting and double check them. We're also looking for places where we may have made assumptions: remember __tg3_set_coalesce() in the Tx codepath! We'll also double-check any calculations and ensure default values are correctly set.
After several thousand lines of code we come to a line in tg3_reset_hw():
It looks harmless enough, but as it deals directly with configuring the card for Rx, we give it extra attention. The obvious thing to check here is if tp->dev->mtu has a sensible value. We can add some debug output to check. When we test the driver, we find that tp->dev->mtu is 0. This could certainly cause serious problems: the maximum MTU for ethernet is just over 1500 bytes!
It turns out that the tg3 driver expects the mtu field in the net_device structure to contain a sensible value. On Linux this would be set by alloc_etherdev() , but on Syllable we simply use kmalloc() to create an instance of the net_device structure, with the allocated memory cleared to 0. Hence, the mtu field is 0. As a test, we'll hardwire the MTU value to something more suitable and test again:
Success! It seems we have found the problem, and now the device is receiving packets and generating interrupts.
Not only is the card generating interrupts, when we ping a remote machine we also receive a response: the device is correctly transmitting and receiving frames, and those frames are being sent correctly to the interface layer. We have a working driver!
You can see how the driver looks in its current state .
It's time to give the driver a proper test. We'll switch off the debugging output by commenting out the lines:
at the top of the driver and recompiling. We'll simply test the driver by using our new network connection just like any other: we'll try connecting to different servers with various protocols and ensure it works.
Part 5 : We'll clean up a little and add some "bells & whistles" in the form of power management. We'll also take a look back and recap the important points of porting a network driver to Syllable.