Archive for June, 2009

NetworkManager and ConnMan

Thursday, June 25th, 2009

Lately ofono and ConnMan have been in the news, and that’s sparked some discussion about how these two projects relate to NetworkManager.  I’ve mostly just been ignoring that discussion and focusing on making NetworkManager better.  But at some point the discussion needs to become informed and the facts need to be straightened out.

So what makes NetworkManager great?

  • Flexibility: NM’s D-Bus interface provides a ton of control and information about the network connections of your machine.  Developers and applications simply don’t take enough advantage of this.  Imagine mail automatically pulled whenever the corporate VPN is up.  Or more restrictive firewalls when connected to public networks.  Yeah, you can do that today with NetworkManager.
  • Works everywhere: from the mainframe to the power desktop to the netbook and lower.  There’s nothing stopping you from running NetworkManager on an s390 or a Palm Pre.
  • Integration: most users like NetworkManager’s distro integration, so it’s on by default (but can be turned off for running bare-metal).  NM will read your distro’s network config files: ifcfg on Fedora, /etc/network/interfaces on Debian, etc.  It doesn’t pretend the rest of the world doesn’t exist, but it can if you tell it to.
  • Connection Sharing: you can share your 3G connection to the wired or the wifi interface, or the other way around.  How you share is completely up to you.
  • VPN: it’s got plugins for Cisco (vpnc), openvpn, openconnect, and pptp.  An ipsec/openswan plugin is being written.  It’s just easy to use the VPN of your choice.
  • Makes Linux better: by not working around stupid vendor drivers or other broken components, NetworkManager drives many improvements in drivers, kernel APIs, the supplicant, and desktop applications.    Five years ago I posted a list of wifi problems, many of which got fixed because NetworkManager users complained about them.  Stuff like WPA capability fixes, hidden SSID fixes, suspend/resume improvements, Ad-Hoc mode fixes, and lots of improvements to wpa_supplicant to name just a few.  By encouraging drivers to be open, by fixing bugs in the open drivers and the stack instead of hacking around them, and by encouraging vendors to work upstream, NetworkManager makes Linux better for you.

What great stuff is coming next?

All in all, a lot of great stuff is on the plate.  NetworkManager already works well for a ton of people, but we’d like to make it work better for a lot more people.  And it will.

So what about ConnMan?

I recently came across a slide deck about ConnMan which makes both disappointing and inaccurate claims about NetworkManager.  It’s also worth emphasizing the philosophical differences between the two projects.

First, ConnMan primarily targets embedded devices, netbooks, and MIDs (slide #1).  When ConnMan was first released in early 2008, NetworkManager 0.7 was under heavy development, and NetworkManager 0.6 clearly did not meet the requirements.  But 0.7, released in November 2008, works well for a wide range of use-cases and hardware platforms.

NetworkManager scales from netbooks, MIDs, and embedded devices with custom-written UIs to desktops to large systems like IBM’s s390.  You get the best of both worlds: from phenomenal cosmic power down to itty-bitty living space.

ConnMan explicitly doesn’t try to integrate with existing distributions (slide #5), partly due to it’s requirements to be as light-weight as possible.  But NetworkManager will use your distro’s normal network config and startup scripts if you tell it to do (but you don’t have to).  Early in NetworkManager days we tried to ignore the rest of the world too.  Turns out that doesn’t work so well; users demand integration with their distribution.  But ConnMan doesn’t pretend to be general purpose, and due to its embedded focus, it can wave this issue away.

Both ConnMan and ofono reject well-established technologies like GObject (but still uses glib) in favor of re-implementing much of GObject internally anyway. This is a curious decision as GObject is not a memory hog and not a performance drag for these cases.  The NIH syndrome continues with libgppp, libgdhcp, and libgdbus, where instead of improving existing, widely-used tools like dhclient/dhcpcd, pppd, and dbus-glib, ConnMan opts to re-implement them in the name of being more “lightweight”.  With embedded projects that ConnMan targets (like Maemo and Moblin) already using GObject and dhcpcd, I don’t understand why this tradeoff was made.  Perhaps this visceral dislike of GObject and dbus-glib was one reason the project’s creators decided to write their own connection manager instead of helping to improve existing ones.

NetworkManager in contrast re-uses and helps improve components all over the Linux stack.  Because of that, more people benefit from the fixes and improvements that NetworkManager drives in projects like avahi, wpa_supplicant, the kernel, pppd, glib, dbus-glib, ModemManager, libnl, PolicyKit, udev, etc.

Taking a look at the deck

I have things to say about most of the slides, but I’ll concentrate on the most interesting and misinformed ones instead.

Supposed Words About NetworkManager

Supposed Words About NetworkManager

  • Very Complex Design: a complete strawman, because it doesn’t say anything.  NetworkManager 0.7 is a mature project with many useful features.  NM is based around a core of objects, each one performing actions based on signals and events from other objects.  It’s modular and flexible.  It’s just not a ConnMan-style box of lego blocks with a rigid plugin API and all the problems that causes.
  • Large Dependency List: NM requires things like wpa_supplicant, udev, dbus, glib, libuuid, libnl, and a crypto library.  pppd and avahi are optional. This list is certainly not large.  When you take ConnMan and its optional dependencies (most of which are needed in a useful system) the list is just about the same.
  • Too Much Decision-making in the UI: Completely bogus and frankly incomprehensible.  The core NM daemon provides a default policy which is in no way connected to the UI, and the rest is up to the user.  nm-applet contains no policy whatsoever.  If the objection is to nm-applet’s desktop-centric interaction model, then it’s important to know there is no lack of applets for different use-cases.
  • Tries to work around distro problems: this is completely a matter of perspective.  Since Intel was creating its own Linux distribution (moblin), they didn’t have to work around any existing issues; these were simply waved away.  Unfortunately NetworkManager lives in the world of reality and not some universe full of ponies.  For users that expect it, NetworkManager integrates with your distros existing network config, init scripts, and DNS resolution.  For users that don’t care, NetworkManager can run bare-metal.
  • Too much GNOME-like source code: seriously, what the hell?  I’m not sure where to begin with this one.  The NetworkManager core does not depend on GNOME.  At all. Yeah, the source-code is in the Gnome style, but is that seriously an issue?
Uninformed diagram of NetworkManager architecture

Uninformed diagram of NetworkManager architecture

(Misinformation shaded blue for your protection)

The User Settings service is contained in the applet, and it’s completely optional.  The System Settings service has been merged back into the NetworkManager core daemon and is no longer a separate process.  That same commit ported NM from HAL to udev; thus HAL is no longer required.  NetworkManager always used HAL/udev for device detection instead of RTNL (ie, netlink).  NetworkManager also hasn’t used WEXT for a long time; wpa_supplicant handles kernel wireless configuration.  NetworkManager uses distro networking scripts only for service control, as does ConnMan.  The rest of the slide is quite petty and just splits hairs.

Where to?

It’s unlikely that either NetworkManager or ConnMan will disappear in the near future.  That means we’ll all have to live with two mutually exclusive connection managers and two completely different network configuration systems.  I think that’s pretty pointless, but I don’t get the last word anyway, since that’s not how Open Source works.  The users will decide which solution works best for them.  And that means NetworkManager will keep getting better, keep getting more useful, and will continue to be the easiest network management solution around.

Mobile Broadband Assistant makes it Easy

Monday, June 22nd, 2009
Yay!  Mobile broadband with NetworkManager is so simple!

Yay! Mobile broadband with NetworkManager is so simple!

(credit mandolin davis)

Easier than your… well, you probably know where I was going with that.  It’s a great leap forward for NetworkManager usability.  Other operating systems either don’t have one, or your network operator gives you the software so of course you don’t have to configure it.  On Linux, we like to work for everyone, so we get to make it easy to get connected to the operator of your choice.

Antti does the base

Antti Kaijanmäki did some work last summer (2008) to put together the mobile broadband provider database and write a library and assistant to use that data.  That was a great start, and Ubuntu started shipping it as a patch in 8.10.  Seems to have worked fairly well there, but since we were deep in the middle of getting the NM 0.7 release out at that time, it wasn’t possible to integrate then.  Antti’s patch didn’t get committed to 0.7.1 for mostly licensing and scope-related reasons, but he built the database which the assistant that just hit git uses, and he proved that it was something users wanted.

Tambet the wrote a compatibly-licensed library to parse the database for network-manager-netbook, which means I didn’t have to, which was nice.

Implemented with Máirín-induced goodness

So a few weeks ago I started rewriting the pitiful GSM/CDMA chooser dialog in network-manager-applet into a full GtkAssistant-based helper.  I’m not an interaction expert, so I tricked Máirín Duffy into helping me get the flow and design planned out.  Then we iterated over my implementation and fixed what sucked, and came out with something that works pretty well.  Starting from the user’s perspective is incredibly important, and that’s what we did with the mobile broadband assistant.

Why do you want this?  (or, WTF is an APN?)

Because you probably have no idea what a GPRS APN is, or why you need the right one to make things work.  Nor should you have to.  At least CDMA got this right by not having one, they are an interaction nightmare.  Your provider knows exactly what you’re paying for, so they know exactly what to bill you for when you use various services they offer.  But when connecting to GPRS data services, you need to tell your phone or device what APN you’d like to use when connecting which in turn tells the provider how you’d like to be billed for it.  But this sort of access control is simply at the wrong level, and having it a the GSM level instead of the application level sucks for users.

There are different APNs for everything; for example T-Mobile USA splits it up as follows:

  • wap.voicestream.com – for T-Zones, the WAP-based walled-garden for dumbphones ($6/mo)
  • internet2.voicestream.com – Unblocked access to anything using a NAT-ed IP address ($20/mo)
  • internet3.voicestream.com – Unblocked access to anythign using a public, routable IP address for certain VPN clients (also $20/mo)
  • epc.tmobile.com – new, nobody’s quite sure what its for

Some providers have a separate APN that downsamples JPEGs to save data costs, others have separate APNs for pay-as-you-go versus contract (they already know whether your IMSI is contract or not, so this baffles me), others have separate APNs per region they serve (BSNL India).  It’s a freaking mess.

Sanity through NetworkManager

APNs don’t really change that often, so it’s easy to build up a crowdsourced database of current providers and their APNs.  Which is what Antti did, and that worked out really well.  Máirín decided to loosely map the APN to a provider’s billing plan, which usually maps to brand name or service that users actually care about.  So I reorganized the mobile broadband provider database to allow multiple “services” (ie, APN or CDMA) for each provider.  This almost gave me carpal tunnel since its not easily scriptable.

Second, I added all the MCC/MNCs (network identification numbers) that I could find, so that in the future we can read you IMSI off your SIM card and automatically suggest your provider when you plug in your phone or data card the first time.  That’s pretty hot.

Hot Pics

When you first insert your card, it shows up in the applet menu.  Ubuntu has a patch that will nag you with a notification that you’ve just plugged in new 3G hardware; they’re welcome to port that to the new code and submit.  For now, it looks like this:

hotplug

When you click that, you’ll get the Assistant’s intro page:

intro

This page explains some of the information you’ll need.  Hopefully you know what provider you signed up for, but if you don’t, you seriously need to stop getting drunk before 3 in the afternoon.  You probably also know what country you’re in, if not,seriously, get a GPS.  I can’t help you with that.

country

Now it gets a little tougher, but since you’re filling a wheelbarrow with your money and dumping it on your provider’s doorstep, you’ll probably also know what provider you signed up with.  But maybe you got shanhai-ed into signing a contract, I don’t know.

provider

But if your provider isn’t listed, we need your help. File a bug in Gnome Bugzilla, tell us your provider name, your country, the common name of your plan, and the APN you use.  We’ll update the provider database with that information, and thank you profusely for making life easier for everyone else too.  We could, in the future, allow users to automatically send their manually entered settings to a server somewhere, and make the provider database update process less manual.  Patches for that greatly appreciated.  Now you get to choose your plan:

plan

Again, if your plan (ie, APN) isn’t listed, file a bug.  But since you signed up, and they probably have some sort of FAQ for that sort of thing, you’ll probably be OK.  Lastly, we have:

summary

and if all looks well, you hit Apply, and NetworkManager will activate the connection you just set up.  You can also change the APN easily through the connection editor.  Much rejoicing was heard.

Future Improvements

There’s a few things we can do in the future…  If the connection fails for some reason, re-display the Assistant.  We can autodetect your provider based on the first 5 or 6 digits of your IMSI on your SIM, skip the Country page, and automatically select that provider in the Provider page, saving you a step or two.  Unfortunately, we can’t autodetect the plan/APN because that’s not stored anywhere (well it is usually preloaded into your phone, but all the APNs are, and there’s no indication of which one is the one you really want).  So there’s room to make it even more awesome.

Your help is required

Again, the mobile broadband provider database is incomplete.  Help fix it up for your country and your provider by filing a bug with your information.  Include your provider name, your country, your plan’s marketing name if you know it, and of course the APN you’re using for data.  If you have a CDMA provider, just tell us your country and provider name.  Username and password are generally ignored by the network and the device, so they aren’t useful.  It’s your help that makes this effort work better for everyone.

I voted

Tuesday, June 16th, 2009

voted

You should too.  That’s how this whole community thing works.

Fedora 11 is out!!!!1111

Thursday, June 11th, 2009

There’s waaaay too much awesome going on in the Fedora community right now.  I mean it.  A TON.  So much it hurts.

If you don't run Fedora 11 he'll eat you...

If you don't run Fedora 11 he'll eat you...

Fedora 11 released, community rejoices

It’s out, no secret.  Install it. Packed with over 50 rocking features like faster startup, automatic font installation, DeviceKit, lickable kernel modesetting for Intel, ATI, and NVIDIA, Firefox 3.5, Gnome 2.26, ext4, KDE 4.2… THE AWESOME DOES NOT STOP.

So much hard work from the Fedora community went into Fedora 11 to make your life better.  Mad props to the Fedora Artwork team too, the graphics in F11 are stunning.  I started at Red Hat in 2003 during the Fedora Core 1 cycle, and with every Fedora release it’s been a great pleasure to watch how the community continues to grow rapidly and contributes so much more each release.  Fedora has truly been a community project for years now, and Fedora 11 shows just how great the community can be when everyone pulls together.  People are awesome.  Which brings us to the next stage…

fedora-community-plainPackaging²

If you have the right tools, tools that help you do what you came to do and don’t get in your way.  And that’s what Fedora Community is; it’s the next step in helping people make a better Fedora. Building better software like Fedora only gets you so far; to keep getting better you need to make the people that make the software better.  That means giving the community the tools it needs to be more efficient, turn ideas into features, and collaborate more effectively.  Fedora Community helps fill that need.  It only gets better from here.

Time for a Stalinist purge

Time for a Stalinist purge

HAL is dead; all hail udev

Over the past few days I’ve exorcised HAL from NetworkManager’s ‘master’ branch.  Instead, we go bare-metal with libgudev.  cgit’s diffstat lies, here are the real numbers:

 86 files changed, 5244 insertions(+), 6755 deletions(-)

Net loss of 1511 lines of code.  Not bad for a few days’ work.  Besides killing HAL, this patch merges nm-system-settings into NetworkManager.  Why do you care?  Here’s why: fewer running processes, less latency, and cleaner internal code.  We just keep scaling up from here.  Next up: nm-applet and ModemManager.