Year: 2011

Gtk+ work on windows

The last few weeks I have been working on the Gtk+ win32 port. Since the client side windows work landed in Gtk+ 2.18 the Windows port has been a bit broken, but now I finally sat down and fixed the remaining issues. So, the newly released Gtk+ 2.24.8 is now officially the best ever Gtk+ 2.x release on windows.

Then I forward ported all the work to the current Gtk 3 tree. It was mostly trivial, but one thing that changed a lot in Gtk3 is theming. The old windows theme was mostly a custom themeing engine, but in Gtk+ 3 we want to use engines less in favour of CSS. So, I had to completely redo the windows theme using CSS.

I added a few CSS extensions that access the win32 theming APIs, so you can get theme backgrounds, theme colors and theme sizes. Then the rest is just traditional CSS to bind the things together.

Here is an image of the current state:

Widget Factory on Windows

There are clearly still some issues that need fixing, but it works impressively well for just being some CSS. Check it out.

Glick2 code availible

I spent some time this week cleaning up and polishing the glick2 codebase. It is now available on the net here.

I also changed the checksumming scheme a bit from my last post, now each file in a bundle is indexed by sha1 checksum, so we can share any files that are identical between two bundles in use.

There is no release yet, and very little documentation on how to use it, but interested parties can play with the code.

Rethinking the Linux distibution

Recently I’ve been thinking about how Linux desktop distributions work, and how applications are deployed. I have some ideas for how this could work in a completely different way.

I want to start with a small screencast showing how bundles work for an end user before getting into the technical details:

[youtube]http://www.youtube.com/watch?v=qpRjSAD_3wU[/youtube]

Note how easy it is to download and install apps? Thats just one of the benefits of bundles. But before we start with bundles I want to take a step back and look at what the problem is with the current Linux distribution models.

Desktop distributions like Fedora or Ubuntu work remarkably well, and have a lot of applications packaged. However, they are not as reliable as you would like. Most Linux users have experienced some package update that broke their system, or made their app stop working. Typically this happens at the worst times. Linux users quickly learn to disable upgrades before leaving for some important presentation or meeting.

Its easy to blame this on lack of testing and too many updates, but I think there are some deeper issues here that affect testability in general:

  • Every package installs into a single large “system” where everything interacts in unpredictable ways. For example, upgrading a library to fix one app might affect other applications.
  • Everyone is running a different set of bits:
    • The package set for each user is different, and per the above all packages interact which can cause problems
    • Package installation modify the system at runtime, including running scripts on the users machine. This can give different results due to different package set, install order, hardware, etc.

Also, while it is very easy to install the latest packaged version of an application, other things are not so easy:

  • Installing applications not packaged for your distribution
  • Installing a newer version of an application that requires newer dependencies than what is in your current repositories
  • Keeping multiple versions of the same app installed
  • Keeping older versions of applications running as you update your overall system

So, how can we make this better? First we make everyone run the same bits. (Note: From here we start to get pretty technical)

I imagine a system where the OS is a well defined set of non-optional core libraries, services and apps. The OS is shipped as a read-only image that gets loopback mounted at / during early boot. So, not only does everyone have the same files, they are using (and testing) *exactly* the same bits. We can do semi-regular updates by replacing the image (keeping the old one for easy rollback), and we can do security hot-fixes by bind-mounting over individual files.

The core OS is separated into two distinct parts. Lets call it the platform and the desktop. The platform is a small set of highly ABI stable and reliable core packages. It would have things like libc, coreutils, libz, libX11, libGL, dbus, libpng, Gtk+, Qt, and bash. Enough unix to run typical scripts and some core libraries that are supportable and that lots of apps need.

The desktop part is a runtime that lets you work with the computer. It has the services needed to be able to start and log into a desktop UI, including things like login manager, window manager, desktop shell, and the core desktop utilities. By necessity there will some libraries needed in the desktop that are not in the platform, these are considered internal details and we don’t ship with header files for them or support third party binaries using them.

Secondly, we untangle the application interactions.

All applications are shipped as bundles, single files that contain everything (libraries, files, tools, etc) the application depends on. Except they can (optionally) depend on things from the OS platform. Bundles are self-contained, so they don’t interact with other bundles that are installed. This means that if a bundle works once it will always keep working, as long as the platform is ABI stable as guaranteed. Running new apps is as easy as downloading and clicking a file. Installing them is as easy as dropping them in a known directory.

I’ve started writing a new bundle system, called Glick 2, replacing an old system I did called Glick. Here is how the core works:

When a bundle is started, it creates a new mount namespace, a kernel feature that lets different processes see different sets of mounts. Then the bundle file itself is mounted as a fuse filesystem in a well known prefix, say /opt/bundle. This mount is only visible to the bundle process and its children. Then an executable from the bundle is started, which is compiled to read all its data and libraries from /opt/bundle. Another kernel feature called shared subtrees is used to make the new mount namespace share all non-bundle mounts in the system, so that if a USB stick is inserted after the bundle is started it will still be visible in the bundle.

There are some problematic aspects of bundles:

  • Its a lot of work to create a bundle, as you have to build all the dependencies of your app yourself
  • Shared libraries used by several apps are not shared, leading to higher memory use and more disk i/o
  • Its hard for bundles to interact with the system, for instance to expose icons and desktop files to the desktop, or add a new mimetype

In Glick 2, all bundles are composed of a set of slices. When the bundle is mounted we see the union of all the slices as the file tree, but in the file itself they are distinct bits of data. When creating a bundle you build just your application, and then pick existing library bundles for the dependencies and combine them into an final application  bundle that the user sees.

With this approach one can easily imagine a whole echo-system of library bundles for free software, maintained similarly to distro repositories (ideally maintained by upstream). This way it becomes pretty easy to package applications in bundles.

Additionally, with a set of shared slices like this used by applications it becomes increasingly likely that an up-to-date set of apps will be using the same build of some of its dependencies. Glick 2 takes advantage of this by using a checksum of each slice, and keeping track of all the slices in use globally on the desktop. If any two bundles use the same slice, only one copy of the slice on disk will be used, and the files in the two bundle mount mounts will use the same inode. This means we read the data from disk only once, and that we share the memory for the library in the page cache. In other words, they work like traditional shared libraries.

Interaction with the system is handled by allowing bundle installation. This really just means dropping the bundle file in a known directory, like ~/Apps or some system directory. The session then tracks files being added to this directory, and whenever a bundle is added we look at it for slices marked as exported. All the exported slices of all the installed bundles are then made visible in a desktop-wide instance of /opt/bundle (and to process-private instances).

This means that bundles can mark things like desktop files, icons, dbus service files, mimetypes, etc as exported and have them globally visible (so that other apps and the desktop can see them). Additionally we expose symlinks to the intstalled bundles themselves in a well known location like /opt/bundle/.bundles/<bundle-id> so that e.g. the desktop file can reference the application binary in an absolute fashion.

There is nothing that prohibits bundles from running in regular distributions too, as long as the base set of platform dependencies are installed, via for instance distro metapackages. So, bundles can also be used as a way to create binaries for cross-distro deployment.

The current codebase is of prototype quality. It works, but requires some handholding, and lacks some features I want. I hope to clean it up and publish it in the near future.

A contacts update, in which robots take over the world

After a vacation and an excellent week in Berlin at the Desktop Summit I’m now back working on Gnome Contacts.

I’m slowly getting most of the UI mockups implemented in preparation for the UI freeze next week, and the evolution-data-server backend in libfolks has landed and been made the default. We also have support in evolution-data-server for the new Online Accounts system. This means we’re currently in a pretty good shape for Gnome 3.2, although there is still a lot of work to do, like getting contacts linking working.

However, today I’d like to talk about something else that I’ve been working on. As you can see in my last update Gnome Contacts relies a great deal on avatars in the UI, and its often the case that your address book contains a lot of people that don’t have one. All these default avatar icons makes it hard to scan the contact lists.

To solve this I’ve been talking to the guy behind the awesome robohash site. It turns out that he’s actually somewhat of a Gnome fan, even helping out with news.gnome.org a long time ago. He’s agreed to let us use the source images and code (in exchange for pimping robohash.org, which you should use! It’s awesome!). I’ve been working on turning the code and images into a library that can be used locally by gnome-contacts. It’s still a work in progress, but here is a screenshot:

This is still a work in progress, but I’m very hopeful that we can land this for Gnome 3.2.

Announcing gnome-contacts

For the last few weeks we’ve been working on the new Gnome contacts applications. Allan day has done most of the design for this and blogged about this recently. Today I did the first release of the code for the Gnome 3.1.2 release.

The code is still at an early state, but most non-editing functionality works, even if its not fully polished. Here are some screenshots to show the current state:

While Allans post showed how the UI looks I’d like to go a bit deeper into the technical side.

Gnome contacts uses libfolks for all access to contacts information. Folks then aggregates multiple sources of contact data, linking pieces of contacts into a whole. For instance, via telepathy it gets IM contacts and information about them, including presence status. Traditional addressbook information is taken from evolution-data-server (currently in progress) It can also connect to social websites such as facebook or twitter via libsocialweb.

The social website integration at first seemed like a very interesting source of information for contacts, but it turns out that there are some non-technical problems with it. All these sites require an application specific key for API access that is generally not distributable, which is not very compatible with open source. Furthermore they have really harsh terms of service that limit what you can do with the service data and how you are able to present it (for instance twitter recently did some very harsh changes for 3rd party apps).

So, while libsocialweb support is still available in Gnome 3.2 if you manage to set it up we’re primary focusing on IM integration (telepathy) and regular vcard style contacts.

The default contacts store for libfolks will be evolution-data-server, which supports several backends including local databases and google contacts. These two will be the primary focus initially so that people can use local databases or easy sync with e.g. android devices via google contacts.

Tomorrow I’m leaving for the IM, social and contacts hackfest in Cambridge where I hope to continue working on the contacts application and the frameworks its using. Also, we want to work on integrating contacts with other applications, including the gnome shell. For instance, it would be very nice if contacts appeared in the gnome-shell overview search.

Broadway update 3

This week saw some new updates of the broadway backend. We now have a in-browser window manager for the non-toplevel window mode, and the backend now support a bunch more features.

I don’t want to bore you with technical mumbo jumbo though. Lets see some video instead! (Original source availible here)

[youtube width=”512″ height=”384″]http://www.youtube.com/watch?v=AO-qca9ddqg[/youtube]
This will be the last update in a while, as I need to spend time on other things. The code is in a pretty good shape though. There are still things to do, but most things work.

Update: Tested this with safari on OSX, and it worked too. Also we now have nicer browser-side window decorations.

Broadway update 2

With Gnome 3.0 being released I had some time to spend on the broadway Gtk+ backend.

I managed to get rid of all roundtrips, which should make remote access snappier. I also fixed a bunch of bugs and added some missing features (including in-process cut and paste).

All that stuff  is nice, but not really all that interesting to show off.

However, I also implemented a cool idea I’ve had for a while about using chromeless browser windows with a canvas inside to get real toplevel windows. It turns out that this works pretty well. Although you have to disable the dom.disable_window_feature_open.location config option to get rid of the location bar at the top of each window.

Here is a short video showing how this looks: (original webm source here)

[vimeo width=”512″ height=”384″]http://vimeo.com/22092069[/vimeo]

Cool, eh? Its all in git, go play with it.

PS: I have no idea why google thinks I’m in france…

Gnome 3.0 released!

I can’t believe we finally released 3.0! The last months have been crazy with energy. The whole project feels revitalized.

Also, I’ve been using gnome-shell now for some time, and I’m really liking it. Going back to gnome 2 now feels clumsy and painful. You should all try it out and make up your own mind, but please try to avoid immediately tweaking it to your old ways, instead try the new defaults for a while, you might like it more than you expect.

Make sure to read the cheat sheet to learn all the new tricks. I really like the new alt-<key over tab> feature!

I am GNOME

Multimonitor support in gnome shell

I recently switched to using gnome-shell in my day to day work, and so far I like it a lot. However, I use a dual monitor setup, and the gnome-shell support for multihead was very rudimentary.

Here is an example desktop with a 1920×1080 primary monitor and a 1280×1024 monitor on the left:

And the corresponding overview:

There are a number of issues here:

  • All previews are on a single monitor. So, while we have a lot of space for normal windows with two monitors we need to cram them all into part of a single monitor in overview mode. This makes the windows way to small to be usable.
  • Due to internal issues the part of the overview that shows the windows is always the same aspect ratio as the screen (i.e. the total size of all monitors). This is fine in the case of one monitor, but with two monitors it doesn’t match the monitor aspect ratio making windows even smaller.
  • During the overview switch the windows on the extra monitor fly a long way, making the animation not work really well, as well as breaking the natural grouping of related windows being on the same monitor.
  • The extra monitor space is unused
  • The thumbnails that normally slide in from the side in a slick way instead overlap the other monitor and its hard to make slide in since the mouse enters the other monitor instead.
  • The thumbnails include the “dead” area due to the different sized monitors and generally look weird.

Additionally, but not visible here:

  • Its hard to hit the hot corner on the primary monitor if there is a monitor to the left. (And to hit the menu in the right corner if there is a monitor to the right.)
  • If the top-left monitor is not the primary there is no hot corner in the easy to reach corner, instead you have to mouse into the primary and then try to hit the corner.
  • If another monitor is taller than the primary, its very hard to hit the message tray on the bottom of the primary monitor, as the pointer passed over to the dead area.
  • In most typical cases the external monitor is used to show something static, like always visible information, a presentation on a projector, or a movie on a TV. However, the extra monitors are connected to the same virtual workspaces as the primary, so when you switch workspace the apps on the extra monitors switch around too.

The last few weeks I’ve been working on this, and these changes have now landed in git.

Here is how it looks now:

And the overview:

We can see here:

  • There is a window overview on each monitor, showing the windows on that monitor.
  • The window overview area is not artificially constrained by aspect ratios but as large as it can be, so the windows are larger in the overview.
  • The thumbnails are always slided in if there is a monitor to the right of the primary monitor.
  • The thumbnails only show the primary monitor. This is because the whole workspace feature only affects the primary monitor. Windows on extra monitors stay in place when switching workspaces, only the primary monitor changes.

Additionally some things we can’t see:

  • Every monitor with a natural top-left corner (i.e. the top left monitor, or a monitor to the right that is taller but aligned at the bottom) gets a hot corner that activates the overview.
  • The ever-rocking ajax has added pointer barrier support to the Xserver for us. If this is available (he backported it to Fedora 15, yay!) we add one-sided barriers in the panel and the message tray so that its easier to hit these, even if there are monitors on the sides.
  • Additionally, as part of the pointer barrier support the Xserver now automatically blocks the mouse from entering the dead areas that are not shown on any monitor. This is great if monitors have different resolutions.

These are not enormous changes, but the difference in day to day use in a multimonitor setup is like day and night. These should be in the next release which is out soon, so if you’re a multimonitor user, do try it out!