Dan Williams’ blog

Let’s start off with an artsy photo¹ and an obscure-but-related caption², becuase that’s the trendy thing to do these days on a blog:

^{Almost home…}

We’ve been working on NetworkManager 0.7 for almost 2 years; it’s been on the rotiesserie’s tasty-slow-cook setting for a long time.  And it’s so close to being done that your Mom is already yelling at you to stop pounding the little kids from across the street and come inside for dinner.  I’ve put up a wiki page with some work items for the NM 0.7 release. This list is by no means complete.  But in the interest of not being a black hole let’s get the stuff out in public and hey, maybe some patches will even show up on networkmanager-list@.  The driving features have been done for a few weeks now, and what’s left are a few UI things, and lots library best-practices stuff like documentation, symbol visibility, API review, and bug fixes.

But what’s not on the list?

• Bluetooth: a bunch of work, but will be a major driver of 0.7.1 or 0.7.5
• IPv6: probably won’t be ready by the time 0.7 ships
• Your broken driver: it’s in the same place as your mom, in a gimp suit in my basement.  Some things just don’t cooperate; you have to keep ‘em in the dark and learn ‘em with a whip until they stop acting up.

but don’t worry, these will get fixed up over time.

As a bonus, I ported the PPTP VPN plugin to the 0.7 API over the weekend.  You’re welcome. Enjoy.  File bugs.  Await the release of 0.7 breathlessly.

¹ To show you how cool I am because I can take photos

² To show you how cool I am because I can write witty captions

Or, how NetworkManager 0.7 transcends decadence and totally respects your distro’s persistent network configuration if you want it to.

With NetworkManager 0.7, the system settings service provides system-wide network configuration, allowing network connections at boot time and across login or fast user switches.  It also reads your distro-specific config files (there are plugins for Fedora and OpenSUSE right now, and an Ubuntu one in-progress) and thus integrates with your normal workflow doesn’t try to re-invent the wheel.  So to put it differently, NetworkManager 0.7 does not ignore your distro configuration unless you really want it to.

Full of Easy

Tambet fully anticipated the decadence of which Alberto Ruiz speaks and wrote the ‘keyfile’ system settings plugin.  We’ve said that if you want a cross-distro persistent, human-readable, text-based network configuration format for interfaces, VPN, 3G, PPP, etc, you can use the keyfile plugin instead of your distro’s format.  A certain class of users really benefits from this.  The other class can just get on with their life and not care what the backend format is, because it simply doesn’t matter to them.  Everyone is happy.

Time for Change

During the 0.7 cycle the new features we added (connection sharing, multiple active devices, 3G to name a few) were pushing the applet’s menu-based design to the limit.  It neither looks good nor behaves well to cram multiple devices and multiple connections into the menu.  Thus, back in January 2008, we asked Bryan Clark and Mike Langlie to come up with some design ideas for an nm-applet that doesn’t suck.  The most intriguing mockup was window-based, which allows for much more streamlined interaction than the menu:

It’s a mockup.  It deserves your love, not your flames.

Right away you’ll probably notice:

• Simple yet convenient: you have both a general overview of your system, but you’re not punched in the face with stuff you don’t care about.  Just like the current applet, but better.  If you want more info, it’s just a click away.
• Disconnect at will: you can already disconnect devices in NM 0.7 (the D-Bus API is there), but adding a disconnect option for every device in the current menu sucks.  Since this isn’t a GtkMenu, there’s a lot more room to play with.
• Dynamicity ™: since it’s not a  GtkMenu, it can update things like device state, signal strength, and addresses dynamically.
• Only shows what you care about: the current applet shows everything around you.  99% of the time, you care about only one or two of those networks, the ones you actually use.  The other 1% of the time, you want to connect to a network you’ve never connected to before.  Why show all 32 other networks all the time, and make you search for the one you want?  Uncool.
• More information if you want it: but not if you don’t.  Becuase there’s more space to work with, we can show stuff you might care about, like the IP address of the device, or the security features of the wifi network you’re current connected to.
• Streamlined Connection Sharing: given the larger layout and ability to tie relevant actions to a specific device, it’ll be a lot clearer to “Share this connection” than the current applet allows.

But as always, it’s a delicate balance between making the stuff you use every day prominant and easy to get to, and keeping the stuff you use only a few times a week out of your way.  I like the fact that I don’t have to care about what I’m connected to, I just want to stay connected and keep working on making stuff awesome.  I don’t want or need to know what the IP address of my VPN server is, for example, or whether my AP uses AES+CCMP for both the pairwise and group ciphers instead of AES+CCMP for the pairwise cipher and TKIP for the group cipher.  But if you really want that information, you should be able to find it within a click or two.

But windows don’t just go away when click outside them.  We could grab the pointer and close the window when you click elsewhere, but that might be weird.  It might also be weird to make this window act like and be positioned in the same place as the current popup GtkMenu, a la gnome-main-menu.  Maybe we should use effect bling to make the window genie out of the NM icon.  It’s something that needs to be prototyped and tested so we can figure out how it feels before we commit to it.  But we’ve been so busy making NetworkManager 0.7 Just Work for you that it’s taken longer than I’d like to start rewriting the applet.  Comments?  Jump #nm on freenode and discuss.

We’ve settled on #nm on freenode as the NetworkManager IRC channel.  Stop by and bitch, moan, rave, flame, suggest, request, patch, anything you like.

Pedal to the metal on the way to NetworkManager 0.7.

Connection Editor

Tambet and I have landed the last real bits of Add/Edit and gotten the pages pretty much finished. The applet and the connection editor retrieve and fill in your passwords too.

Contribute Back to the Community (or, Unmanaged Devices)

A few weeks back, I added an unmanaged devices interface to the system settings service. With 0.6.x, the most often asked question is “I’m an Ubuntu user; why can NM find my network device?”. This was for two reasons: (1) because Ubuntu ships a bunch of shady out-of-kernel wireless drivers (at76, prism2_usb, acx, madwifi, ndiswrapper) that often just don’t implement WEXT correctly and therefore won’t work well with wpa_supplicant, and therefore won’t work with NetworkManager, and (2) Ubuntu patched NetworkManager so that most devices in /etc/network/interfaces are ignored by NM, instead of helping to fix up the Debian backend to proxy that configuration so NM could have a chance to manage the device. So when anything goes wrong, the user is encouraged to configure the device in “Manual” mode instead, and it disappears from NetworkManager.

With 0.7, the system settings plugin for your distro will recognize these devices, tell NetworkManager they aren’t supposed to be managed, and the applet will make you aware of the horror of what you’ve just done

(as an aside, distros need to help push drivers and patches upstream, not stuff random bits into the kernel and hope everything is kittens and roses and puppy dogs tails and bright copper kettles and warm woolen mittens)

Network Before Login

So this time around, distros can write much more capable plugins to proxy their native config files to NetworkManager connections, and they will just show up in the menu. It also makes the connections available at boot. Static IPs, custom DNS servers, and whatever other crack you’d like to inflict on your network adapter. Both Fedora and SUSE have plugins, and Tambet just wrote a GKeyFile plugin that stores connections in a legacy-free/crack-free format too.

Other Hotness:

• Users are notified of VPN failures and what might have gone wrong
• Static WEP keys on indexes other than 1
• PPPoE
• Wired 802.1x
• Your Mom

Next up: making the serial driver code more robust, fix bugs, fix up ad-hoc Wifi, and fix more bugs. But 0.7 is already cooking MCs like a pound of bacon.

No! No! Too sexy, too sexy!! [1]

For the numerically impaired, there are four. Coming soon to Fedora 9 and an SVN server near you.

[1] yeah, it kinda looks like ass, but we’re working on that

Which one is right for you?

There’s now an addition to the HAL specification identifying mobile broadband cards, and a hot new hal-info package (20080215) that contains the necessary .fdi file for most of the cards directly supported in Linux.  This specification helps NetworkManager and other tools identify that (a) the card is really a modem instead an unconnected serial port, and (b) whether it’s a GSM or a CDMA modem.  I also updated NetworkManager SVN to work with the new spec.  Plug it in, watch it work, be happy.

Yeah, I’m talking to you happy, carefree, wireless-using, Nordic-looking prettygirl, with your Sprint Novatel S720 all up in our face.

I spent an hour or so hacking in CDMA card support on top of Tambet Ingo’s awesome 3G mobile broadband and PPP bits. And it just works. And like the GSM cards that Tambet’s got working, your VPN works over it too. Like magic. Tambet is the man.

What we do need to do, though, is standardize a method in HAL for tagging CDMA and GSM cards as “modems” (along with other things that are modems and accept AT commands). We also need to tag cards as CDMA or GSM. David Zeuthen had a few ideas on this, but I think in the end it’s going to have to be with .fdi files, because vendors just don’t follow the spec. There are separate USB protocol numbers allocated for CDMA and GSM devices, but no card I’ve been able to find uses them. And that won’t work for PCMCIA-based devices like the Sierra Wireless AirCard 860. When will hardware manufacturers learn? (hint: likely never).

The CDMA support won’t go into trunk until Tambet’s added yet more hotness, like signal strength detection and pulling out the currently connected network and showing that to the user. I’ve proved, though, that it’s trivial to add the CDMA bits when the GSM support is further along.

Rejoice, and be merry, for the wireless future is almost here.

No, this isn’t about porn, but when it’s done it’ll be even more awesome.

Tambet Ingo has been kicking mobile broadband ass recently, using it to club the NetworkManager PPP integration into shape. And that rocks. He’s been a driving force of NetworkManager for the last year, spearheading the D-Bus interface rewrite and the major refactoring for the upcoming 0.7 release. Buy him a whole keg next time you sit down at a bar and find out he’s just two seats down from you, looking all badass.

Now with 100% more NetworkManager

Interaction with 3G cards is going to work a lot like VPN connections currently do. The 3G card will show up in the applet’s menu. It will have a sub-menu of connections that you’ve created for the card. If you use more than one provider, you can create a connection for each provider and switch between the two. Eventually, you’ll be able to share that 3G connection over wifi or ethernet too. Obligatory mockup follows:

Deliciously easy 3G…

Right now the preliminary support is for GSM-based cards, most things that have the words GPRS, EDGE, UMTS, HSDPA, HSUPA, or HSPA on them. We’ll soon add support for CDMA-based 1xRTT, EVDO rev 0, and EVDO rev A cards too. It’s pretty rough at the moment, and he just committed it this morning, so don’t expect the thing to work just because you plugged it in (yet). After the PPP bits have been worked over Jack Bristo-style and are begging for mercy, we’ll start adding support for serial modem, ISDN, and Bluetooth DUN.

You ask, we provide.

It will cure cancer. It can divide by zero. It can touch M.C. Hammer. And it’ll make your life better too. Why?

Drivers Suck Less

Drivers were (and still are) the largest reason for NM flakiness. But today, they are more consistent in their support of Linux Wireless Extensions, the standard ioctl-based method of talking to wireless drivers. NetworkManager directly caused the addition of WEXT-based WPA support to Madwifi, ndiswrapper, and linux-wlan-ng. And mac80211, the new kernel 802.11 wireless stack, will provide a consistency of driver operation unseen yet in Linux. That rocks, since we’ll have one place to fix the behavior instead of 6 or 7. It just keeps getting better, even though it’s taken a while to get here. I put together a list of driver problems waaay back in 2005. People cared. Drivers got fixed.

More Bars in More Places

The biggest reason why NM 0.6.x couldn’t support some device types or WPA authentication methods was that the configuration D-Bus API wasn’t flexible enough. The config framework has been completely rewritten. It’s more flexible, better engineered, and quite a bit easier to understand. We’ll support GSM/GPRS/EDGE/UMTS/HDSPA/HSUPA and 1xRTT/EVDO-r0/EVDO-rA broadband cards via better PPP integration, which also means dialup and possibly ISDN/PPPoE/PPPoATM support too. Bluetooth DUN and PAN will be possible now too.

You’re not in Soviet Russia

The network doesn’t control you. It’s the other way ’round. A major goal of 0.7 is widening the usefulness of NetworkManager. Through static IP support (including multiple IP addresses), you could use NM on your server or stationary desktop machine if you wish. You don’t need to use DHCP. You can override specific pieces of DHCP-provided configuration (like search domains). You can set up multiple connections for a device, with static IPs when you’re at work and DHCP when you’re at home. You’ll be able to have more than one device active at one time, and share an internet connection between them. For once you’ll be able to easily share that $9.99/hr hotel WiFi connection with your friends over ethernet, or an ethernet connection with your friends over WiFi.

Your Mom is NetworkManager’s Other Ride

With NetworkManager, she won’t have to care about how she connects to the Internet to check her email or sell her stuff on Ebay when she uses Linux. Wherever she happens to be tonight. Even though 0.7 will have awesome new capabilities that you and your mom will love longtime, it will still have the same easy simplicity of use you’ve come to expect from stuff that Just Works. We’re going to keep the GNOME applet simple up-front, and punt the really nasty configuration complexity to an out-of-band connection properties tool. So you won’t have to see configuration options that 95% of people don’t have to touch anyway.

On the Technical Side

There’s been a lot of code churn. Driving wpa_supplicant via D-Bus makes everything incredibly robust and allowed us to drop threads completely. Dropping the dep on dhcdbd makes DHCP work so much better (thanks to Robert Frank for the initial patch). Tambet Ingo showed the internal structure who was boss (hint: he was), and reworked the D-Bus API adding much-needed sanity and introspection support. The configuration D-Bus interface is based on dicts, allowing a lot of future flexibility.

Christmas in October?

Probably not; Fedora 8 has been the driving force for 0.7 so far, and we’ll get something quite usable in F8 even if it’s not feature-complete enough to call 0.7.0. The first focus is feature parity with 0.6.5, then multiple active device support, and then we’ll unleash 0.7.0 upon the world.

NetworkManager 0.7 uses wpa_supplicant (over D-Bus) for everything. Why you ask? Because it saves code. We’ve been ripping code out of 0.7 faster than hugsie & co. can add code to PackageKit. Somebody must preserve the galactic balance of code. When wpa_supplicant gets notified of new scan results by the wireless driver, it echoes that notification back out over D-Bus.

Drivers Behaving Badly (duh…)

Turns out that ipw2200 and ipw2100 make extensive use of background scanning. They send out scan result notifications many times per second. Which causes wpa_supplicant to wake up, process the scan results, and emit its own scan results available signal over D-Bus. Causing NetworkManager to wake up and ask wpa_supplicant for the parsed scan results. Causing nm-applet to wake up and process updates as they come in from NetworkManager. See a pattern?

48 Hours Later…

Three patches to wireless drivers to fix asstastic scanning behavior and make your life better, your battery not hate you, and NetworkManager rock harder:

• http://marc.info/?l=linux-wireless&m=119194675024816&w=2 (orinoco)
• http://marc.info/?l=linux-wireless&m=119195246702824&w=2 (ipw2200)
• http://marc.info/?l=linux-wireless&m=119203367309826&w=2 (ipw2100)

Not huge, but little fixes keep piling up until stuff Just Works.