Syllable
Network drivers - Part 3
Device API

Our driver can be loaded by the kernel and successfully initialises the hardware. The logical next step is to add functionality to the device API in the tg3_dev_open() , tg3_dev_close() , tg3_dev_ioctl() and tg3_dev_write() functions. They are quite simple functions but will lead us onto porting the rest of the driver, including the interrupt handling code and the transmit and receive code.

The implementation of these functions is almost identical across all of the ethernet drivers. tg3_dev_write() for example, looks like this:

static int tg3_dev_write( void* pNode, void* pCookie, off_t nPosition, const void* pBuffer, size_t nSize ) { struct net_device* psNetDev = pNode; PacketBuf_s* psBuffer = alloc_pkt_buffer( nSize ); if ( psBuffer != NULL ) { memcpy( psBuffer->pb_pData, pBuffer, nSize ); psBuffer->pb_nSize = nSize; tg3_start_xmit( psBuffer, psNetDev ); } return nSize; }

All it does is to copy the raw data from the pBuffer argument into a new PacketBuf_s structure and call tg3_start_xmit() , which is currently a stub function. tg3_dev_ioctl() is similarly just as simple, requiring code for just four ioctl commands. It calls two new functions, tg3_open() and tg3_close() . Do not confuse these with the existing tg3_dev_open() and tg3_dev_close() functions. We will cover tg3_open() and tg3_close() later.

One thing to note is the use of the pNode argument in both tg3_dev_ioctl() and tg3_dev_write() . Both functions have:

struct net_device* psNetDev = pNode;

at the top of the function. The pNode argument is passed to the driver by the kernel, and comes from the pData argument we gave the kernel when we called create_device_node():

pNetDev->node_handle = create_device_node( nDeviceID, sInfo.nHandle, zNodePath, &g_sDevOps, pNetDev );

The last two functions, tg3_dev_open() and tg3_dev_close() are the simplest of the four: they don't have to do anything at all! As long as they return successfully, nothing else has to happen. The real work is done when tg3_open() or tg3_close() are called in response to the appropriate ioctl commands in tg3_dev_ioctl() .

You can see what the driver looks like now, with stubs for the three functions tg3_start_xmit() , tg3_open() and tg3_close() here .

Open, close and interrupts

We now have a driver with three stub functions. They are:

tg3_start_xmit() tg3_open() tg3_close()

Nothing much can happen until the device has been successfully opened, so we'll start with that.

One section of code that deserves a special mention is the timer code. The Linux code is:

init_timer(&tp->timer); tp->timer.expires = jiffies + tp->timer_offset; tp->timer.data = (unsigned long) tp; tp->timer.function = tg3_timer;

In Syllable, timers are managed differently so we have changed those four lines to:

tp->timer = create_timer(); start_timer(tp->timer, (timer_callback *) &tg3_timer, tp, (jiffies + tp->timer_offset)*100, true );

The effect is identical: a timer will call the function tg3_timer() after the timer period expires. We'll provide a stub for tg3_timer() for now.

With a few more tweaks, the driver compiles but we now have a whole new list of functions that need to be ported:

[user@machine:~/src]CFLAGS=-O2 make gcc -O2 -kernel -fno-PIC -c -D__ENABLE_DEBUG__ -I. tg3.c -o objs/tg3.o gcc -kernel objs/tg3.o -o objs/tg3 objs/tg3.o: In function `tg3_dev_ioctl': tg3.c:(.text+0x5604): undefined reference to `tg3_full_lock' tg3.c:(.text+0x564a): undefined reference to `tg3_full_unlock' tg3.c:(.text+0x5652): undefined reference to `tg3_alloc_consistent' tg3.c:(.text+0x56b3): undefined reference to `tg3_request_irq' tg3.c:(.text+0x56db): undefined reference to `tg3_free_consistent' tg3.c:(.text+0x56ef): undefined reference to `tg3_full_lock' tg3.c:(.text+0x56f9): undefined reference to `tg3_init_hw' tg3.c:(.text+0x57c9): undefined reference to `tg3_full_unlock' tg3.c:(.text+0x57de): undefined reference to `tg3_test_msi' tg3.c:(.text+0x5827): undefined reference to `tg3_full_lock' tg3.c:(.text+0x5840): undefined reference to `tg3_enable_ints' tg3.c:(.text+0x5848): undefined reference to `tg3_full_unlock' tg3.c:(.text+0x58a8): undefined reference to `tg3_free_rings' tg3.c:(.text+0x58b0): undefined reference to `tg3_full_unlock' tg3.c:(.text+0x58f9): undefined reference to `tg3_full_lock' tg3.c:(.text+0x592b): undefined reference to `tg3_free_rings' tg3.c:(.text+0x5933): undefined reference to `tg3_free_consistent' tg3.c:(.text+0x593b): undefined reference to `tg3_full_unlock' collect2: ld returned 1 exit status make: *** [objs/tg3] Error 1 [user@machine:~/src]

The functions tg3_full_lock() and tg3_full_unlock() look like good candidates to do first. In fact they are single-line functions that simply take a spinlock. We've modified them slightly for Syllable; the Linux version uses the function spin_lock_bh() and spin_unlock_bh() , but Syllable has no concept of "bottom halves" as Linux does so there are no special spinlock functions.

That leaves us with seven other functions to port:

tg3_alloc_consistent() tg3_request_irq() tg3_free_consistent() tg3_init_hw() tg3_test_msi() tg3_enable_ints() tg3_free_rings()

We also still have tg3_timer() , tg3_close() and tg3_xmit() . We'll go ahead and port the seven in our list above first, though.

NAPI

One place where we run into differences between Syllable and Linux is the use of the Linux NAPI functions such as netif_rx_schedule() . NAPI is quite different to how Syllable works, so we will need to change calls to these functions to act in a more compatible way.

With NAPI the driver calls netif_rx_schedule() to tell the kernel that work is required and the kernel then calls a poll() function in the driver when it is ready. We can not easily emulate this scheme in Syllable, so we will have to change it.

If you want more information, the Linux NAPI documentation describes how netif_rx_schedule() works. For Syllable, we will make the following changes:

For now, we'll provide stubs for tg3_rx() and tg3_tx() .

Our changes may require some further tweaking when we come to test the driver, but for now these simple changes should be sufficient.

Once we have all of the functions ported the driver compiles again. We've still got some code enclosed in #if blocks and some stubs (the timer handling code and the Rx and Tx functions), but we now have code to open the device and initialise the hardware and ring buffers, code to handle interrupts and code to close the device and return it to a sensible state.

We've added just over 2000 lines of code to the driver. You can see how the driver looks here .

The following functions are currently stubs:

tg3_tx() tg3_rx() tg3_start_xmit() tg3_start_xmit_dma_bug() tg3_timer() tg3_find_peer()

So we clearly do not yet have enough to be able to test our driver properly, but can at least try the load it and see what happens when the interface layer attempts to open the device. This will cause tg3_dev_open() to be called and run much of the code we have ported since we last tested the driver.

Testing

Again when we test the driver it does not crash, which would seem to indicate that most of the new code is working as we would expect. We can add some additional debugging output to check. We'll add some debugging to tg3_open() to see if the new code paths are being followed.

When we test we discover that tg3_open() is called but exits early:

0:init::init(-1) : tg3: Tigon3 [partno(BCM95751) rev 4001 PHY(5750)] (PCI Express) 10/100/1000BaseT Ethernet 0:init::init(-1) : tg3: MAC: 00:12:3f:2e:6e:80 0:init::init(-1) : tg3: RXcsums[0] LinkChgREG[1] MIirq[1] ASF[0] Split[0] WireSpeed[1] TSOcap[0] 0:init::init(-1) : tg3: dma_rwctrl[76180000] 0:init::init(-1) : tg3_probe(): Create node net/eth/tg3-0 ... 0:init::init(-1) : tg3_open ENTER 0:init::init(-1) : IRQ 11 enabled ...

tg3_open() has several places where it may return early, so we'll add debugging statements at each of those points to find out what the likely problem is:

0:init::init(-1) : tg3: Tigon3 [partno(BCM95751) rev 4001 PHY(5750)] (PCI Express) 10/100/1000BaseT Ethernet 0:init::init(-1) : tg3: MAC: 00:12:3f:2e:6e:80 0:init::init(-1) : tg3: RXcsums[0] LinkChgREG[1] MIirq[1] ASF[0] Split[0] WireSpeed[1] TSOcap[0] 0:init::init(-1) : tg3: dma_rwctrl[76180000] 0:init::init(-1) : tg3_probe(): Create node net/eth/tg3-0 ... 0:init::init(-1) : tg3_open ENTER 0:init::init(-1) : IRQ 11 enabled 0:init::init(-1) : tg3_open(): 2 ...

So tg3_open() returns at the second possible early return point, in this block of code:

err = tg3_request_irq(tp); if (err) { if (tp->tg3_flags2 & TG3_FLG2_USING_MSI) { pci_disable_msi(tp->pdev); tp->tg3_flags2 &= ~TG3_FLG2_USING_MSI; } tg3_free_consistent(tp); kerndbg( KERN_DEBUG_LOW, "%s(): 2\n", __FUNCTION__ ); return err; }

Apparently, we could not successfully claim the desired IRQ. We can also see from the kernel output that at least part of tg3_request_irq() worked:

0:init::init(-1) : IRQ 11 enabled

So we'll have to investigate a little further and try to find out what has gone wrong.

tg3_request_irq() is quite simple:

static int tg3_request_irq(struct tg3 *tp) { int (*fn)(int, void *, SysCallRegs_s *); unsigned long flags; struct net_device *dev = tp->dev; if (tp->tg3_flags2 & TG3_FLG2_USING_MSI) { fn = tg3_msi; if (tp->tg3_flags2 & TG3_FLG2_1SHOT_MSI) fn = tg3_msi_1shot; } else { fn = tg3_interrupt; if (tp->tg3_flags & TG3_FLAG_TAGGED_STATUS) fn = tg3_interrupt_tagged; flags = SA_SHIRQ; } return (request_irq(tp->dev->irq, fn, NULL, flags, dev->name, dev)); }

The problem is that we have changed the call to request_irq() but not payed attention the way it is used. On Linux request_irq() returns 0 to indicate success and a positive value to indicate failure. On Syllable request_irq() returns a positive value as a handle that is used with other kernel functions, and less than 0 to indicate failure. This is the opposite of what is expected. So we'll re-write this to fit Syllable:

static int tg3_request_irq(struct tg3 *tp) { int (*fn)(int, void *, SysCallRegs_s *); unsigned long flags; struct net_device *dev = tp->dev; if (tp->tg3_flags2 & TG3_FLG2_USING_MSI) { fn = tg3_msi; if (tp->tg3_flags2 & TG3_FLG2_1SHOT_MSI) fn = tg3_msi_1shot; } else { fn = tg3_interrupt; if (tp->tg3_flags & TG3_FLAG_TAGGED_STATUS) fn = tg3_interrupt_tagged; flags = SA_SHIRQ; } tp->dev->irq_handle = request_irq(tp->dev->irq, fn, NULL, flags, dev->name, dev); return ( tp->dev->irq_handle < 0 ? 1 : 0 ); }

We store the handle returned by request_irq() and return 0 to indicate success and 1 if request_irq() failed. This should satisfy any code in the driver that calls tg3_request_irq() .

Now tg3_open() appears to work:

1:init::init(-1) : tg3: Tigon3 [partno(BCM95751) rev 4001 PHY(5750)] (PCI Express) 10/100/1000BaseT Ethernet 1:init::init(-1) : tg3: MAC: 00:12:3f:2e:6e:80 1:init::init(-1) : tg3: RXcsums[0] LinkChgREG[1] MIirq[1] ASF[0] Split[0] WireSpeed[1] TSOcap[0] 1:init::init(-1) : tg3: dma_rwctrl[76180000] 1:init::init(-1) : tg3_probe(): Create node net/eth/tg3-0 ... 1:init::init(-1) : tg3_open ENTER 1:init::init(-1) : IRQ 11 enabled 1:init::init(-1) : tg3_open EXIT

At this point there is not much more we can do to test the driver. However, we can at least be moderately certain that the device initialisation code is now complete.

Next

Part 4: We'll port the rest of the code for the Rx and Tx code paths, and test our driver.