Recently Dave Täht wrote a blog post investigating latency and WiFi scanning and came across NetworkManager’s periodic scan behavior. When a WiFi device scans it obviously must change from its current radio channel to other channels and wait for a short amount of time listening for beacons from access points. That means it’s not passing your traffic.
With a bad driver it can sometimes take 20+ seconds and all your traffic gets dropped on the floor.
With a good driver scanning takes only a few seconds and the driver breaks the scan into chunks, returning to the associated access point’s channel periodically to handle pending traffic. Even with a good driver, latency-critical applications like VOIP or gaming will clearly suffer while the WiFi device is listening on another channel.
So why does NetworkManager periodically scan for WiFi access points?
Roaming
Whenever your WiFi network has multiple access points with the same SSID (or a dual-band AP with a single SSID) you need roaming to maintain optimal connectivity and speed. Jumping to a better AP requires that the device know what access points are available, which means doing a periodic scan like NetworkManager does every 2 minutes. Without periodic scans, the driver must scan at precisely the worst moment: when the signal quality is bad, and data rates are low, and the risk of disconnecting is higher.
Enterprise WiFi setups make the roaming problem much worse because they often have tens or hundreds of access points in the network and because they typically use high-security 802.1x authentication with EAP. Roaming with 802.1x introduces many more steps to the roaming process, each of which can fail the roaming attempt. Strategies like pre-authentication and periodic scanning greatly reduce roaming errors and latency.
User responsiveness and Location awareness
The second reason for periodic scanning is to maintain a list of access points around you for presentation in user interfaces and for geolocation in browsers that support it. Up until a couple years ago, most Linux WiFi applets displayed a drop-down list of access points that you could click on at any time. Waiting for 5 to 15 seconds for a menu to populate or ‘nmcli dev wifi list’ to return would be annoying.
But with the proliferation of WiFi (often more than 30 or 40 if you live in a flat) those lists became less and less useful, so UIs like GNOME Shell moved to a separate window for WiFi lists. This reduces the need for a constantly up-to-date WiFi list and thus for periodic scanning.
To help support these interaction models and click-to-scan behaviors like Mac OS X or Maemo, NetworkManager long ago added a D-Bus API method to request an out-of-band WiFi scan. While it’s pretty trivial to use this API to initiate geolocation or to refresh the WiFi list based on specific user actions, I’m not aware of any clients using it well. GNOME Shell only requests scans when the network list is empty and plasma-nm only does so when the user clicks a button. Instead, UIs should simply request scans periodically while the WiFi list is shown, removing the need for yet another click.
WHAT TO DO
If you don’t care about roaming, and I’m assuming David doesn’t, then NetworkManager offers a simple solution: lock your WiFi connection profile to the BSSID of your access point. When you do this, NetworkManager understands that you do not want to roam and will disable the periodic scanning behavior. Explicitly requested scans are still allowed.
You can also advocate that your favorite WiFi interface add support for NetworkManager’s RequestScan() API method and begin requesting periodic scans when WiFi lists are shown or when your browser uses geolocation. When most do this, perhaps NetworkManager could be less aggressive with its own periodic scans, or perhaps remove them altogether in favor of a more general solution.
That general solution might involve disabling periodic roaming when the signal strength is extremely good and start scanning more aggressively when signal strength drops over a threshold. But signal strength drops for many reasons like turning on a microwave, closing doors, turning on Bluetooth, or even walking to the next room, and triggering a scan then still interrupts your VOIP call or low ping headshot. This also doesn’t help people who aren’t close to their access point, leading to the same scanning problem David talks about if you’re in the basement but not if you’re in the bedroom.
Another idea would be to disable periodic scanning when latency critical applications are active, but this requires that these applications consistently set the IPv4 TOS field or use the SO_PRIORITY socket option. Few do so. This also requires visibility into kernel mac80211 queue depths and would not work for proprietary or non-mac80211-based drivers. But if all the pieces fell into place on the kernel side, NetworkManager could definitely do this while waiting for applications and drivers to catch up.
If you’ve got other ideas, feel free to propose them.
You don’t need an Uber, you don’t need a cab (via Casey Bisson CC BY-NC-SA 2.0)
NetworkManager 1.2 was released yesterday, and it’s already built for Fedora (24 and rawhide), a release candidate is in Ubuntu 16.04, and it should appear in other distros soon too. Lubo wrote a great post on many of the new features, but there’s too many to highlight in one post for our ADD social media 140-character tap-tap generation to handle. Ready for more?
indicator menus
Wayland is coming, and it doesn’t support the XEmbed status icons like nm-applet creates. Desktop environments also want more control over how these status menus appear. While KDE and GNOME both provide their own network status menus Ubuntu, XFCE, and LXDE use nm-applet. How do they deal with lack of XEmbed and status icons?
Ubuntu has long patched nm-applet to add App Indicator support, which exposes the applet’s menu structure as D-Bus objects to allow the desktop environment to draw the menu just like it wants. We enhanced the GTK3 support in libdbusmenu-gtk to handle nm-applet’s icons and then added an indicator mode to nm-applet based off Ubuntu’s work. We’ve made packager’s lives easier by building both modes into the applet simultaneously and allowing them to be switched at runtime.
IP reconfiguration
Want to add a second IP address or change your DNS servers right away? With NetworkManager 1.2 you can now change the IP configuration of a device through the D-Bus interface or nmcli without triggering a reconnect. This lets the network UIs like KDE or GNOME control-center apply changes you make to network configuration immediately without interrupting your network connection. That might take a cycle or two to show up in your favorite desktop environment, but the basis is there.
802.1x/WPA Enterprise authentication
An oft-requested feature was the ability to use certificate domain suffix checking to validate an authentication server. While NetworkManager has supported certificate subject checking for years, this has limitations and isn’t as secure as domain suffix checking. Both these options help prevent man-in-the-middle attacks where a rogue access point could masquerade as as your normal secure network. 802.1x authentication is still too complicated, and we hope to greatly simplify it in upcoming releases.
Interface stacking
While NM has always been architected to allow bridges-on-bonds-on-VLANs, there were some internal issues that prevented these more complicated configurations from working. We’ve fixed those bugs, so now layer-cake network setups work in a flash! Hopefully somebody will come up with a fancy drag-n-drop UI based off Minecraft or CandyCrush with arbitrary interface trees. Maybe it’ll even have trophies when you finally get a Level 48 active-backup bond.
Old Stable Series
Now that 1.2 is out, the 1.0 series is in maintenance mode. We’ll fix bugs and any security issues that come up, but typically don’t add new features. Backporting from 1.2 to 1.0 will be even more difficult due to the removal of dbus-glib, a major feature in 1.2 release. If you’re on 1.0, 0.9.10, or (gasp!) 0.9.8 I’d urge you to upgrade, and I think you’ll like what you see!
dbus-glib debuted in 2002 and was the first usable D-Bus client library for GLib-based applications. NetworkManager used it since the earliest commits in mid-2004.
Why do you care?
Apps and services built on it and enabled the modern desktop. If you’ve ever tried to code directly to libdbus to add a D-Bus service or talk to one you know what a pain that is. dbus-glib made that process infinitely easier but had a ton of quirks that didn’t fit well with an evolving GLib. But in the past 5 years the GLib authors recognized that D-Bus needed first-class support and they wrote GDBus with much better GLib integration and a modern asynchronous API. dbus-glib has been on life-support for years. It was time to move on.
The upcoming NetworkManager 1.2 release will finally remove use of dbus-glib for the daemon, command-line/TUI utilities, nm-applet/editor, and VPN plugins if you build with legacy components disabled. Almost nobody noticed this in-flight rebuild of the airplane and that’s exactly how we like it. I cannot understate how much work this was and how much careful planning we (well, mostly Dan Winship) did to ensure we didn’t break backwards compatibility of either the utility libraries or the D-Bus interface.
To be fair, this is a slight lie; we can’t maintain C API/ABI compatibility in the legacy client libraries by removing dbus-glib and we cannot drop them yet either. But we did write the new libnm which much more closely matches GLib coding practices and API, including GCancellables and asynchronous functions. Its API is close enough to the old libnm-util and libnm-glib that porting to libnm is not a huge task, and porting will actually remove code from your application too. Seriously: the libnm port for nm-connection-editor was +772/-1203 lines of code and nm-applet was +697/-990. Killing code feels great.
We’ve already ported over all the NetworkManager tools like nmcli, nmtui, nm-applet, nm-connection-editor, and all the VPN plugins. In the process, we cleaned up a ton of code, created better client APIs, wrote more testcases, reduced the dependency chain, and paid down most of our technical debt. When kdbus (or bus1 or whatever it’s called) eventually hits, NetworkManager will be ready.
Almost since the beginning of time, NetworkManager kept an internal list of access points found in the last 3 scans. Since the background scan were triggered at least every two minutes, an access point could stay in the list for up to 6 minutes. This was a compromise between mobility, unreliable drivers, and an unreliable medium (eg, air). Even when you’re not moving the closest access point may not show up in every scan. So NetworkManager attempted to compensate by keeping access points around for a longer time.
Obviously that approach has problems if you’re driving, on a train, or on a bus. You can end up with a huge list of access points that are obviously no longer in range. If you turn off an access point, it could stay in the list a long time.
Ubuntu contributed a patch that exposes the “last seen time” for each access point, which allows the user-interface to decide for itself which access points to show. A location service (like Firefox or Geoclue) may want a more complete list of access points than the live Wi-Fi network list does, for example, which is why NetworkManager keeps the list in the first place instead of only showing the results of the most recent (and potentially unreliable) scan.
But in the end this behavior needed to change, and with recent versions of wpa_supplicant it was possible to make NetworkManager’s scanning behavior better. The supplicant also contains a scan list from which NetworkManager built it’s. Wouldn’t it be great if there was one list instead of two?
So we threw away the internal NetworkManager list and just followed the supplicant’s list. When the supplicant decides that an access point is no longer visible, NetworkManager removes it too. This works better because the supplicant has more information than NetworkManager does and can make smarter decisions. NetworkManager tweaks the supplicant’s behavior through the BSSExpireAge and BSSExpireCount properties so that any access point seen more than 4 minutes ago, or not seen in the past two scans, will be dropped.
When scans happen more often, like when a Wi-Fi network list is displayed, the two-scan limit removes access points after 20 or 30 seconds in the best case. The supplicant performs backgrounds scans to facilitate faster roaming, which can be triggered on signal strength, which also helps remove old access points when they are out of range.
Tracking the Current BSS
Along with the scanning cleanup, NetworkManager delegates tracking the access point you’re currently associated with to wpa_supplicant’s CurrentBSS property. Previously NetworkManager periodically asked the Wi-Fi driver what the current access point was, but this was inconsistently implemented between drivers and required hacky workarounds to smooth out intermittent results.
The supplicant’s CurrentBSS property tracks the access point the supplicant wants to be associated with, not what the driver currently is associated with, but these are almost always the same thing, and there’s no point in telling the user that they are momentarily disconnected from their access point during a scan when there is no actual interruption in traffic due to 802.11 protocol mechanisms like powersave buffering. This was another huge cleanup in the NetworkManager codebase.
Death of dbus-glib
Finally, along with these changes all communication with wpa_supplicant was switched to use GDBus instead of the old, deprecated, and unmaintained dbus-glib library. This advances our goal of removing all use of dbus-glib from NetworkManager, which was one of the first major users of the library in 2004, and is likely the last one too. GDBus provides much better integration with glib and the GIO model, is fully supported, and has an saner API.
And even better, through the painstaking work of Dan Winship, Jirka Klimes, Thomas Haller, Lubomir Rintel, me, and others, all of NetworkManager 1.2 was ported to GDBus without changing our public D-Bus API. Welcome to the future!
Just a quick note that we’ve released the latest stable NetworkManager, version 1.0.4. This is mainly a bugfix release, though it does have a couple new features and larger changes. Some of those are:
Some configuration options can now be changed without restarting the NM daemon. Those include the ‘dns’, ‘connectivity’, and ‘ignore-carrier’ settings.
Devices that have only an IPv6 link-local address are no longer assumed to be connected; by default whenever an interface is set “up” the kernel will assign an IPv6 link-local address which could theoretically be used for communication. NetworkManager used to interpret this as the interface being available for network communication, while this is rarely what users want or expect.
Correct routing is now maintained when two interfaces of the same priority are connected to the same network.
udev rules can now be used to tell NetworkManager to manage or unmanage specific devices.
Connections with children (bridge, bond, team, etc) can now optionally bring up their slave interfaces when the master is started.
Many, many bugs and crashers have also been fixed in the core daemon, the libraries, and nmcli.
We’re well into the development cycle of NetworkManager 1.2 as well, where among other things, we’ll finally be moving to GDBus instead of dbus-glib. We’ll also have support for structured logging with journald, indicating that certain connections are metered, advertising LLDP-discovered information, built-in IPv4 link-local addressing to get rid of avahi-autoipd, improvements to Wi-Fi Access Point scanning, less verbose logging by default, improvements to DNS handling, and more. Look for it later this year!
Just this morning Lubomir released NetworkManager 1.0.2, the latest of the 1.0 stable series. It’s a great cleanup and bugfix release with contributions from lots of community members in many different areas of the project!
Some highlights of new functionality and fixes:
Wi-Fi device band capability indications, requested by the GNOME Shell team
Devices set to ignore carrier that use DHCP configurations will now wait a period of time for the carrier to appear, instead of failing immediately
Startup optimizations allow networking-dependent services to be started much earlier by systemd
Memory usage reductions through many memory leak fixes and optimizations
teamd interface management is now more robust and teamd is respawned when it terminates
dnsmasq is now respawned when it terminates in the local caching nameserver configuration
Fixes for an IPv6 DoS issue CVE-2015-2924, similar to one fixed recently in the kernel
IPv6 Dynamic DNS updates sent through DHCP now work more reliably (and require a fully qualified name, per the RFCs)
An IPv6 router solicitation loop due to a non-responsive IPv6 router has been fixed
While the list of generally interesting enhancements may be short, it masks 373 git commits and over 50 bugzilla issues fixed. It’s a great release and we recommend that everyone upgrade.
Next up is NetworkManager 1.2, with DNS improvements, Wi-Fi scanning and AP list fixes for mobile uses, NM-in-containers improvements (no udev required!), even less dependence on the obsolete dbus-glib, less logging noise, device management fixes, continuing removal of external dependencies (like avahi-autoipd), configuration reload-ability, and much more!
NetworkManager is a system service that manages network interfaces and connections based on user or automatic configuration. It supports Ethernet, Bridge, Bond, VLAN, team, InfiniBand, Wi-Fi, mobile broadband (WWAN), PPPoE and other devices, and supports a variety of different VPN services. You can manage it a couple different ways, from config files to a rich command-line client, a curses-like client for non-GUI systems, graphical clients for the major desktop environments, and even web-based management consoles like Cockpit.
There’s an old perception that NetworkManager is only useful on laptops for controlling Wi-Fi, but nothing could be further from the truth. No laptop I know of has InfiniBand ports. We recently released NetworkManager 1.0 with a whole load of improvements for workstations, servers, containers, and tiny systems from embedded to RaspberryPi. In the spirit of making double-plus sure that everyone knows how capable and useful NetworkManager is, let’s take a magical journey into Administrator-land and start at the very bottom…
Daemon Configuration Files
Basic configuration is stored in /etc/NetworkManager/NetworkManager.conf in a standard key/value ini-style format. The sections and values are well-described by ‘man NetworkManager.conf’. A standard default configuration looks like this:
[main]
plugins=ifcfg-rh
You can override default configuration through either command-line switches or by dropping “configuration snippets” into /etc/NetworkManager/conf.d. These snippets use the same configuration options from ‘man NetworkManager.conf’ but are much easier to distribute among larger numbers of machines though packages or tools like Puppet, or even just to install features through your favorite package manager. For example, in Fedora, there is a NetworkManager-config-connectivity-fedora RPM package that installs a snippet that enables connectivity checking to Fedora Project servers. If you don’t care about connectivity checking, you simply ‘rpm -e NetworkManager-config-connectivity-fedora’ instead of tracking down and deleting /etc/NetworkManager/conf.d/20-connectivity-fedora.conf.
Just for kicks, let’s take a walk through the various configuration options, what they do, and why you might care about them in a server, datacenter, or minimal environment…
Configuration Snippets
First, each configuration “snippet” in /etc/NetworkManager/conf.d can override values set in earlier snippets, or even the default configuration (but not command-line options). So the same option specified in 50-foobar.conf will override that option specified in 10-barfoo.conf. Many options also support the “+” modifier, which allows their value to be added to earlier ones instead of replacing. So “plugins+=something-else” will add “something-else” to the list, instead of overwriting any earlier values. You’ll see why this is quite useful in a minute…
This option enables or disables certain settings plugins, which are small loadable libraries that read and write distribution-specific network configuration. For example, Fedora/RHEL would specify ‘plugins=ifcfg-rh’ for reading and writing the ifcfg file format, while Debian/Ubuntu would use ‘plugins=ifupdown’ for reading /etc/network/interfaces, and Gentoo would use ‘plugins=ifnet’. If you know your distro’s config format like the back of your hand, NetworkManager doesn’t make you change it.
There is one default plugin though, ‘keyfile’, which NetworkManager uses to read and write configurations that the distro-specific plugins can’t handle. These files go into /etc/NetworkManager/system-connections and are standard .ini-style key/value files. If you’re interested in the key and value definitions, you can check out ‘man nm-settings’ and ‘man nm-settings-keyfiles’, or even look at some examples.
[main]
monitor-connection-files=yes | no (default no)
By popular demand, NetworkManager no longer watches configuration files for changes. Instead, you make all the changes you want, and then explicitly tell NetworkManager when you’re done with “nmcli con reload” or “nmcli con load <filename>”. This prevents reading partial configuration and allows you to double-check that everything is correct before making the configuration update. Note that changes made through the D-Bus interface (instead of the filesystem) always happen immediately.
However, if you want the old behavior back, you can set this option to “yes”.
[main]
auth-polkit=yes | no (default yes)
If built with support for it, NetworkManager uses PolicyKit for fine-grained authorization of network actions. This will be the subject of another article in this series, but the TLDR is that PolicyKit easily allows user A the permission to use WiFi while denying user B WiFi but allowing WWAN. These things can be done with Unix groups, but that quickly gets unwieldy and isn’t fine-grained enough for some organizations. In any case, PolicyKit is often unecessary on small, single-user systems or in datacenters with controlled access. So even if your distribution builds NetworkManager with PolicyKit enabled, you can turn it off for simpler root-only operation.
[main]
dhcp=dhclient | dhcpcd | internal (default determined at build time, dhclient preferred if enabled)
With NetworkManager 1.0 we’ve added a new internal DHCP client (based off systemd code which was based off ConnMan code) which is smaller, faster, and lighter than dhclient or dhcpcd. It doesn’t do DHCPv6 yet, but we’re working on that. We think you’ll like it, and it’s certainly much less of a resource hog than a dhclient process for every interface. To use it, set this option to “internal” and restart NetworkManager.
If NetworkManager was built with support for dhclient or dhcpcd, you can use either of these clients by setting this option to the client’s name. Note that if you enable both dhclient and dhcpcd, dhclient will be preferred for maximum compatibility.
[main]
no-auto-default= (default empty)
By default, NetworkManager will create an in-memory DHCP connection for every Ethernet interface on your system, which ensures that you have connectivity when bringing a new system up or booting a live DVD. But that’s not ideal on large systems with many NICs, or on systems where you’d like to control initial network bring-up yourself. In that case, you should set this option to “*” to disable the auto-Ethernet behavior for all interfaces, indicating that you’d like to create explicit configuration instead. You can also use MAC addresses or interface names here too! On Fedora we’ve created a package called NetworkManager-config-server that sets this option to “*” by default.
[main]
ignore-carrier= (default empty)
Trip over a cable? Want to make sure a critical interface stays configured if the switch port goes down? This option is for you! Setting it to “*” (all interfaces) or using MAC addresses or interface names here will tell NetworkManager to ignore carrier events after the interface is configured. For DHCP connections a carrier is obviously required for initial configuration, while static connections can start regardless of carrier status. After that, feel free to unplug the cable every time Apple sells an iPhone!
[main]
configure-and-quit=yes | no (default no)
New with 1.0 is the “configure and quit” mode where NetworkManager configures interfaces (including, if desired, blocking startup until networking is active) and then quits, spawning small helpers to maintain DHCP leases and IPv6 address lifetimes if required. In a datacenter or cloud where cycles are money, this can save you some cash and deliver a more stable setup with known behavior.
Want to control DNS yourself? NetworkManager makes it easy! Don’t want to? NetworkManager makes that easy too! When you set this option to ‘dnsmasq’ NetworkManager will configure dnsmasq as a local caching nameserver, including split DNS for VPN tunnels. If you set it to ‘none’ then NetworkManager won’t touch /etc/resolv.conf and you can use dispatcher scripts that NetworkManager calls at various points to set up DNS any way you choose.
Leaving the option empty or setting it to “default” asks NetworkManager to own resolv.conf, updating system DNS with any information from your explicit network settings or those received from automatic means like DHCP.
In the upcoming NetworkManager 1.2, DNS information is written to /var/lib/NetworkManager/resolv.conf and, if NM is allowed to manage /etc/resolv.conf, that file will be a symlink to the one in /var similar to systemd-resolvd. This makes it easier for external tools to incorporate the DNS information that NetworkManager combines from multiple sources like DHCP, PPP, IPv6, VPNs, and more.
[keyfile]
unmanaged-devices= (default empty)
Want to keep NetworkManager’s hands off a specific device? That’s what this option is for, where you can use “interface-name:eth0” or “mac:00:22:68:1c:59:b1” to prevent automatic management of a device. While there are some situations that require this, by default NetworkManager doesn’t touch virtual interfaces that it didn’t create, like bridges, bonds, VLANs, teams, macvlan, tun, tap, etc. So while it’s unusual to need this option, we realize that NetworkManager can be used in concert with other tools, so it’s here if you do.
Connectivity checking helps users log into captive ports and hotspots, while also providing information about whether or not the Internet is reachable. When NetworkManager connects a network interface, it sends an HTTP request to the given URI and waits for the specified response. If you’re connected to the Internet and the connectivity server isn’t down, the response should match and NetworkManager will change state from CONNECTED_SITE to CONNECTED. It will also check connectivity every ‘interval’ seconds so that clients can report status to the user.
If you’re instead connected to a WiFi hotspot or some kind of captive portal like a hotel network, your DNS will be hijacked and the request will be redirected to an authentication server. The response will be unexpected and NetworkManager will know that you’re behind a captive portal. Clients like GNOME Shell will then indicate that you must authenticate before you can access the real Internet, and could provide an embedded web browser for this purpose.
Upstream connectivity checking is disabled by default, but some distribution variants (like Fedora Workstation) are now enabling it for desktops, laptops, and workstations. On a server or embedded system, or where traffic costs a lot of money, you probably don’t want this feature enabled. To turn it off you can either remove your distro-provided connectivity package (which just drops a file in /etc/NetworkManager/conf.d) or you can remove the options from NetworkManager.conf.
Special NetworkManager data files
In the normal course of network management sometimes non-configuration data needs to persist. NetworkManager does this in the /var/lib/NetworkManager directory, which contains a few different files of interest:
seen-bssids
This file contains the BSSIDs (MAC addresses) of WiFi access points that NetworkManager has connected to for each configured WiFi network. NetworkManager doesn’t do this to spy on you (and the file is readable only by root), but instead to automatically connect to WiFi networks that do not broadcast their SSID. You almost never need to touch this file, but if you are concerned about privacy feel free to delete this file periodically.
timestamps
Each time you connect to a network, whether wired, WiFi, etc, NetworkManager updates the timestamp in this file. This allows NetworkManager to determine which network you last used, which can be used to automatically connect you to more preferred networks. NetworkManager also uses the timestamp as an indicator that you have successfully connected to the network before, which it uses when deciding whether or not to ask for your WiFi password when you get randomly disconnected or the driver fails.
NetworkManager.state
This file stores persistent user-determined state for Airplane mode for each technology like WiFi, WWAN, and WiMAX. Normally this is controlled by hardware buttons, but some systems don’t have hardware buttons or the drivers don’t work, plus that state is not persistent across boots. So NetworkManager stores a user-defined state for each radio type and will ensure the radio stays in that state across reboots too.
DHCP lease and configuration files
When you obtain a DHCP lease, that lease may last longer than your connection to that network. To ensure that you receive a nominally stable IP address the next time you connect, or to ensure that your TCP sessions are not broken if there is a network hiccup, NetworkManager stores the DHCP lease and attempts to acquire the same lease again. These files are stored per-connection to ensure that a lease acquired on your home WiFi or ethernet network is not used for work or Starbucks. Temporary DHCP configuration files are also stored here, which are constructed based on your preferences and on generic DHCP configuration files in /etc for each supported DHCP client. If you want to wipe the DHCP slate clean, feel free to remove any of the lease or configuration files.
And that’s it for this time, stay tuned for the next part in this series!
Right before the 2014 holidays, and more than 10 years after the first line of NetworkManager was typed, we released version 1.0. A huge milestone on the way to making NetworkManager more cooperative, more flexible, more configurable, and more useful than ever before.
How you ask?
1: libnm: the new GLib client library
For all the GLib/GObject users out there, we’ve rebuilt libnm-util and libnm-glib from the ground up into a new single library called libnm. It uses GDBus instead of dbus-glib. It provides GIO-style asynchronous methods. It also exposes IP addresses, MAC addresses, and other properties as strings instead of byte arrays, and combines the old NMClient and NMRemoteSettings objects into a single NMClient object, among other things.
from gi.repository import GLib, NM
for dev in NM.Client.new(None).get_devices():
ipcfg = dev.get_ip4_config()
if ipcfg:
for addr in ipcfg.get_addresses():
print "(%s) %s/%d" % (dev.get_iface(), addr.get_address(), addr.get_prefix())
2: a smaller, faster DHCP client
While it doesn’t do DHCPv6 (yet!) this internal client (based off systemd/connman code) is much faster than dhclient and dhcpcd, and doesn’t consume huge amounts of memory like dhclient. Use the ‘dhcp=internal’ option in NetworkManager.conf to enable it and let us know how it works. We’ll be adding DHCPv6 support and enhancing the recognized options in the near future.
3: configure and quit
Have a more static configuration and still want to use NetworkManager configuration and API to manage it? The ‘configure-and-quit=yes’ option in NetworkManager.conf will configure your interfaces and quit the NM process, spawning small helpers to preserve DHCP and IPv6 addresses. This saves cycles (and therefore money) and is simpler to manage.
4: more cooperative
Continuing the trend, NetworkManager 1.0 does a much better job of leaving externally configured interfaces alone until you tell it to do something. In addition to improvements for IPv6 sysctl recognition and user-added route preservation, externally created virtual interfaces are no longer automatically set IFF_UP, and NetworkManager handles external master/slave relationship changes more smoothly.
5: more powerful nmcli
We’ve added PolicyKit and interactive password support to nmcli, allowing full command-line-only operation for most network connections, even for less privileged users. There’s a new ‘nmcli dev connect’ command that brings up an interface using the best available connection. You can also delete virtual interfaces directly through nmcli.
6: improved IPv6
We’ve ensured that if network interfaces are supposed to be down and unconfigured, that the kernel doesn’t assign a link-local address to them, to prevent potential security issues when you think networking is down. We’ve also added support for IPv6 WWAN connections and fixes to respect router-delivered MTUs.
7: Bluetooth DUN support
Bluez5 changed API for Dial-Up-Networking functionality, which broke the NetworkManager support. At long last we’ve added that support back, no thanks to Bluez. Happy mobile networking!
8: more flexible and cooperative routing
Every interface that can have a default route now gets one, and NetworkManager manages the priorities to ensure they don’t conflict. Plus, if you need to, you can manually manage priorities on a per-connection basis to prefer WiFi over WWAN or WWAN over ethernet, or whatever you need.
9: fewer dependencies
We’ve also removed some direct dependencies (PolicyKit), slimmed down code, and split functionality into selectable plugins, leading to easier installs on limited systems and better configurability.
That’s just the tip of the iceberg; we’ve improved almost every part of NetworkManager and we’re not stopping there. We’re planning improvements to container use-cases, WiFi, VPNs, power savings, client APIs, and much more. 2015 is gonna be a great year, and not just because the version number is greater than 1!
(Note: the final 0.9.10 will be out later this week… read on for the awesome that it will contain)
Hey wouldn’t it be great if NetworkManager did X and made my life so awesome I could retire to a private island surrounded by things I love? Like kittens and teddy bears and bright copper kettles and Domaine Leflaive Montrachet Grand Cru?
If only your dream was reality… Oh wait! NetworkManager 0.9.10 will be your genie from Aladdin, granting every wish you dream of, except this time you can wish for more wishes. But you still can’t bring awful networking back from the dead because it’s just not pretty. Don’t do it.
Tons of new features, yet somehow smaller and nimbler! (via cuxclipper, CC BY 2.0)
What is pretty is NetworkManager 0.9.10; it’s like the lightning-quick racing yacht that Larry Ellison doesn’t have and really, really wants, but which somehow also adds a Triple-E-Class-worth of new features just for you. Somebody (maybe you!) wished for every single thing you’re about to see. And then a magic genie showed up, snapped its fingers, and gave it to them.
nmtui
We found a usability gap between full-fledged CLI tools like nmcli and GUI-based ones, and thus nmtui was born. Sometimes you don’t want to remember esoteric commands and options, but you also don’t want to run X. Boom, first wish granted: a curses-based tool for configuring and managing your network, no X involved:
nmcli
The command-line still rules with divine mandate, and we’re here to please so nmcli was a huge focus for this release. We’ve added interactive editing support, single-command editing, detailed help,tab completion, and enhanced bash completion. You really need to check this out; almost anything you can do with GUI tools can now be done with nmcli, and there’s even some stuff nmcli can do that the GUI tools can’t. If you’re comfortable with terminals, NetworkManager 0.9.10 is right up your alley.
Size Does Matter
Continuing on the quest to be more nimble and streamlined, we’ve split Wi-Fi, WWAN, Bluetooth, ADSL, and WiMAX device support into plugins which you don’t need to install if you like a minimal system. Distributions should package these separately so they can be added/removed independently of NetworkManager itself, which reduces disk usage, runtime memory usage, and packaging dependency chains. We’ve also spent time slimming down and optimizing the code. The core NetworkManager daemon is now just over 1MB in size!
dbus-daemon is also no longer required for root-only or early-boot operation, with communication using a private root-only Unix socket. Similarly, PolicyKit is no longer used for root operation, though it could always be disabled at build-time anyway.
To facilitate remote and SSH-based management, the “at_console” D-Bus permission has been removed, which also helpfully harmonizes authorization settings between Fedora and Debian-based distributions. All permissions authorization now happens through PolicyKit instead.
When you Absolutely Positively MUST have your ethernet frames delivered on-time and without loss you turn to Data Center Bridging. DCB provides the reliability and robustness that iSCSI and FibreChannel over Ethernet (FCoE) need so you don’t have to keep shovelling money into a proprietary SAN. Since users requested it, we snapped our fingers and added support to NetworkManager 0.9.10 for configuring DCB on your ethernet interfaces.
We’ve also upped our game with IP-level configuration support for many more software interfaces like GRE, macvlan, macvtap, tun, tap, veth, and vxlan. And when you have services that aren’t yet network-aware, the NetworkManager-wait-online systemd service is more reliable to ensure your legacy services start up with the resources they require.
Customization Galore
You dreamed, we listened. Creepy, no? Yeah, we know what you want. And top of the list was more flexible configuration:
Connection configuration files are no longer watched for changes by default, which used to cause problems with backups, filesystem copies, half-configured connections, etc. If you want that behavior you can turn it back on (monitor-connection-files=true), but instead, edit them as much as you want and when you’re done, “nmcli con reload“.
Connections can now be locked to interface names instead of just MAC addresses
A new “ignore-carrier” option is available to ensure your critical app doesn’t fail just because you got drunk on Captain Morgan + Coke, and tripped over a cable
Want to manage /etc/resolv.conf yourself? You can! “dns=none” is your new best friend.
Configuration file snippets can be dropped into /etc/NetworkManager/conf.d to change smaller sets of configuration options
The NetworkManager dispatcher got some enhancements too. It now has a “pre-up” event that allow scripts to execute before NetworkManager announces connectivity to applications. We also added a “pre-down” event that lets network filesystems flush data before the interface is actually disconnected from the network.
Do you love /sbin/ip? ifconfig? brctl? vconfig? Keep using them! Changes you make outside of NetworkManager get picked up, respected, and reflected in the D-Bus API. NetworkManager 0.9.10 also goes to great lengths to read the existing configuration of interfaces and not touch them. Most network interfaces known to the kernel are now exposed in the D-Bus API, and you can even change their IP configuration right from NetworkManager. There’s more work to do here but we hope you’ll appreciate the new situational awareness as much as we do.
Get Your VPN On
We’ve improved support for routing-only VPNs like Openswan/Libreswan/Strongswan. We’ve added full details of the VPN’s IP configuration to the D-Bus API. And best yet, VPN plugins can now request additional passwords during the connection process if the ones you previously gave them are wrong or changed.
All the Rest
For clients, more properties are exposed in the D-Bus API. We’ve added support for custom IP ranges to the Internet Connection Sharing functionality. We’ve added WWAN autoconnect support and more reliable airplane mode behavior. Fatal connection errors now more reliably block reconnect, which means better handling of wrong Wi-Fi passwords and access point failures. Captive portal/hotspot support is moving forward, as are DNSSEC enhancements.
Geez, are we done yet?
Not even close! Seriously, there’s more but I’m kinda tired of typing. Try it out (the final release will be out later this week) and tell us what you think. Then tell us what you want. Don’t be afraid to dream a little bigger, darling!
Recently we posted a Fedora 21 update delivering the huge new awesome that is NetworkManager 0.9.10-beta1. Among a literal Triple E Class boatload of enhancements and fixes, this update continues our fine tradition of making the core of NetworkManager smaller and more flexible by splitting out Wi-Fi support into the NetworkManager-wifi package. If you don’t have or don’t use Wi-Fi on your system, you don’t need to install stuff for it and you can save some disk space and RAM.
The Problem
To ensure upgrades work correctly and people didn’t unexpectedly lose Wi-Fi support we set RPM Obsoletes to ensure that when upgrading the new package was installed even though it didn’t exist before. Unfortunately this didn’t work for those using DNF instead of yum for package management.
It turns out that DNF treats RPM obsoletes differently than yum, and it’s unclear right now how package splitting is actually supposed to work with DNF. You can track the issue here.
The Workaround
If you suddenly find yourself without Wi-Fi, find a wired network connection and:
and harmony will return to the Universe. We understand the pain, will continue to monitor the situation, and will update the Fedora NetworkManager packages when DNF has a solution.
Wayland is coming, and it doesn’t support the XEmbed status icons like nm-applet creates. Desktop environments also want more control over how these status menus appear. While KDE and GNOME both provide their own network status menus Ubuntu, XFCE, and LXDE use nm-applet. How do they deal with lack of XEmbed and status icons?
