gnome3 – Form and Function

Sandboxed applications for GNOME

Application sandboxing is a subject that I am passionate about. In recent months I have been involved in a design initiative to plan out how sandboxed applications would work on GNOME, and I gave a talk on this subject at GNOME.Asia early this year, and I’ve been meaning to blog about it ever since.

There’s quite a lot to be said about sandboxed applications, so this is the first of two posts on the subject. In this post, I want to talk about why application sandboxing is important, and it importance across a range of areas. The second post will deal with the actual user experience of using a sandboxed application.

A brief definition

Application sandboxing can mean a number of things. I’m using a fairly loose, abstract, definition, and I don’t want to get too bogged down in technical details. For the purposes of this post, a sandboxed application is an app that:

Only has access to a limited set of system API.
Has managed access to additional system API and to user data – so that the application must ask permission for access.
Can be effectively tracked and managed by the host operating system as it runs.
Can be installed and updated independently of the host operating system – new dependencies are not required, and the operating system components do not need to be updated.
Can include any dependencies that are not provided by the operating system.

There are two related aspects to this definition: the first is application distribution, installation and update. The second is a privacy and security framework. Sandboxed applications are containerised, and those containers can include bundled libraries. They have only limited access to the underlying system, and they must ask for user permission to use certain hardware devices (like cameras, microphones or GPS), or to access user data.

Why sandboxing matters

I genuinely think that application sandboxing represents the future for the GNOME project. We have never had a rigourous definition of what an application actually is in the GNOME world. That has led to all kinds of negative consequences, both in terms of the user experience that we provide, and the ability for developers to distribute applications. Conversely, sandboxing infrastructure for GNOME promises to have all kinds of positive consequences.

The following are what I consider to be the most important consequences of application sandboxing.

Value proposition

Security and privacy, I think, are core beliefs for Free Software. Users should be able to trust us to have their interests at heart, and should be able to have more faith in our products than proprietary alternatives. Ironically, though, the Free Software desktop world hasn’t done a great job at security. It is actually pretty scary what a malicious desktop application could do if it wants to. We rely on transparency and good faith to ensure that applications do not infringe on user privacy, rather than robust technical architecture.

If we want to be able to make a strong case for the value of Free Software desktops, we need to be able to make serious claims about the robustness of our security and privacy features. It is time to stand up for what we believe in, and to make our technologies reflect our beliefs and commitments. Otherwise, it is quite difficult for us to tell a coherent story about why we or our products matter.

Application developer experience

Sandboxed applications are an essential part of an effective, easy-to-use application development platform. By definition, a sandboxed application has access to less API than one that isn’t sandboxed. This is actually a good thing – it makes it clear to application developers which API is supported by the platform. It gives them less to deal with, and reduces uncertainty about how they should implement their application.

It is also easier for us to develop our application development platform if it works through sandboxing – since we will have a clearer, more tightly defined set of platform functionality to worry about. Sandboxing makes it easier to look at the platform as a whole and identify missing functionality, or functionality that can be improved.

Application distribution

This is an important issue for application developers, as well as the GNOME application ecosystem. Right now it is far too difficult for developers to get their applications into the hands of users. Application distribution is fragmented, slow, and fraught with organisational hurdles.

An application development platform built on sandboxing and containerisation will provide a stable base which application authors can target. This will allow application authors to target specific versions of the platform, to easily distribute and update their applications, and to have greater confidence in how their application will look and perform in the hands of users. It will make it easier for developers to independently distribute their applications, should they want to.

OS definition

The primary purpose of any operating system is to host applications. While there are certain additional features that these operating systems need to provide, wherever possible, the base operating system should be as minimal as possible, and only provide functionality that is needed by the applications it hosts.

Right now, we don’t have a clearly defined set of API that is exposed to applications. As a result, it becomes rather difficult to decide which functionality a desktop operating system should provide and, without a clearly defined application platform, we lack the ability to rigorously define what functionality and components the host operating system should include.

Application sandboxing helps with this problem of operating system definition. The API that is exposed to applications becomes the primary reference point for determining which components should make up the operating system. Addressing this issue of OS definition has a number of other beneficial consequences: once you can clearly define an operating system, testing becomes a lot easier. Writing documentation gets easier. Providing support gets easier.

OS user experience

Application sandboxing and containerisation enables a far superior user experience. I’ve already alluded to some of the ways that it does this – protecting privacy is a user experience issue, for example. Likewise, application installation and update also have user experience consequences.

There are other ways that sandboxing helps to improve user experience though. It ensures that applications behave consistently, as well as preventing them from misbehaving. It also enables the operating system to effectively manage system resources. Applications that are running in the background can be throttled to ensure that the system and whatever you are interacting with remains fast and responsive. Power management can be improved. Network bandwidth can be portioned out more effectively.

Enabling the operating system to effectively manage applications is a key way that we can make the user experience consistent, smooth, and predictable. It is 2014. Your system should not be laggy and slow because some application is chewing up all your CPU.

Sounds rather good, doesn’t it?

Application definition, integration and management is a fundamental aspect of defining an operating system, a user experience, and an application development platform. As a result, application sandboxing promises to have positive effects across a wide range of areas. I hope that this post has illustrated where application sandboxing will have a positive impact, and why it is so important.

In the GNOME project, we have been discussing and planning how sandboxed applications could work for some time. Several major technical initiatives – like Wayland and KDBus – are now reaching a stage where they can provide the necessary infrastracture for us to make them a reality. As Christian has posted today, concrete plans now exist for the next stage of development.

In my next post on this subject, I will describe the user experience design that Jon, Jakub and me have been working on in this area. This should give a better idea of what it will be like to use a sandboxed application. It should also give an idea of the outstanding work necessary to make them a reality.

In praise of Jim Hall

At the beginning of this year, Jon was contacted by someone who was interested in doing user testing on GNOME. This person said that he wanted to promote usability in open source software, and had research experience. His name was Jim Hall.

Fast forward to the present day, and Jim has conducted a set of user tests on GNOME 3.10 and 3.12, which he has analysed and presented to Jakub and me. I have started filing bugs so we can fix the usability issues he discovered. More bug reports are on their way, and we’re pushing to use Jim’s testing data to increase GNOME’s usability for the next release. (Check out the bugs if you’re interested in helping out with this!)

The best thing about working with Jim has been how keen he has been to work us in the GNOME project throughout the research process. We were able to tell him about the kinds of things we were interested in for user testing, were able to give feedback and comments on his test plans, and were able to discuss his results with him shortly after the tests were conducted. This kind of direct access to the research process was fantastic, and ensured that the tests were relevant to our ongoing work.

Jim will be presenting his test results at GUADEC next month.

A notifications update

A couple of weeks ago I posted about ongoing work to redesign notifications in GNOME 3. Since then, the redesign has moved on a fair bit, so I thought that an update was in order. My previous post also generated a fair amount comment, and I’ve been wanting to respond to the questions and comments we received.

What’s happening now

Previously, I described a work-in-progress design that we have been pursuing in GNOME design. Since that post, the process has diversified, and we are exploring several variations on the original design. These different options are in a state of evolution, and we are developing and evaluating them in parallel. To help with this, Jasper has created a couple of rough prototypes that we’ve been testing.

I’m not going to go into the details of these different designs here – they’re in a state of flux, and it would take a fair amount of time and effort to go through each one. However, if you are interested, there’s plenty of activity in the usual places: check out the wiki, our mockups repository, or hang out on the #gnome-design or #gnome-shell IRC channels (note that details are necessarily incomplete as the design is in a state of flux).

Each of the designs we are considering have some basic things in common, like using a time ordered list for the Message Tray. They also share some goals, like simplifying the UI and code base. This has meant that the design project has been able to move forward in some areas, and our plans for the technical architecture have been moving forward.

Comments and questions

I read every comment that is made on my blog, as well as the responses I get on social media. Since there was a fair amount of feedback on my last post, I wanted to take the time to respond in a more structured manner than usual.

The first thing to say here is thanks for all the positive comments and encouragement. Most people seemed to think that the new designs were a step in the right direction, and I’m really happy about that. There were concerns and suggestions of course. These are the most popular ones that I saw:

Lock screen privacy concerns

A number of people raised privacy concerns about displaying notifications on the lock screen. This was largely the result of a misunderstanding (my fault!): the mockup I posted in my previous post featured detailed notification banners on the lock screen, and this led people to think that this was going to be a feature of the new design.

The current behaviour is to show notifications on the lock screen, but to only indicate the application which sent the notification – we don’t include the content of notifications on the lock screen by default. The new design didn’t involve us changing this policy, even though the mockup alone suggested otherwise.

That said, the comments about this did prompt us to re-examine how the current lock screen notifications design performs when notification content isn’t displayed, and we might well make some changes here.

Notifications getting in the way

The comments contained multiple reports of the existing toaster-style notifications getting in the way of what users are doing in applications. This was observed to be a particular issue when using chat clients, IRC or terminals.

We’ve been aware of this issue for some time, and it is something that we’ve attempted to resolve in the past. The designs that I presented last time around didn’t fix the issue, and one of the reasons why we’ve made the decision to investigate other options is to see if we can do better.

It should be noted that most notification systems involve popups appearing over applications, and an alternative design will likely have its own quirks. This is something we need to take seriously when considering major design changes.

Missed notifications

A number of people told us that they miss notifications with the existing GNOME 3 notifications, and this observation often prompted the suggestion that the new notifications design should include a permanent visual indication of pending notifications.

The issue of missed notifications is definitely an important factor that we are attempting to address, by changing the position, size and timing of notification banners. We are also hoping to introduce a priority setting for notifications, so that more important popups (like for chat messages) can be made more noticable. We are also considering an indicator for pending or missed notifications as a part of the design.

What about the top bar?

There were various requests that the top bar should be used for notifications. This is a complex question that touches on a fundamental part of the notifications design: if you give notifications a presence in the top bar, it is my view that notification banners need to be located in the same area of the screen, and this raises a whole other set of issues and questions.

GNOME 3 has historically had its notifications at the bottom of the screen (I mentioned some reasons for this in my previous post): it allowed notification banners to be expanded so that actions could be taken on them. In particular, this arrangement was designed to facilitate the chat integration features we see in GNOME 3 today.

Having a way to access notifications from the top bar would have a number of advantages. It would give you a discoverable click target, and the top screen edge has ergonomic advantages. It would also locate notifications amongst other system functionality. However, it would be a big change, which would require an even bigger redesign than the one I have already described. Nevertheless, it is something that we’re seriously looking at.

Status icons

Status icons are another outstanding issue that we received feedback about. There are known interaction issues associated with them in GNOME 3, and this is something that we want to resolve. One of the major issues here is the problematic nature of the status icon API – if we are going to deal with the problem properly, we need to establish a new API and give applications time to port over to it. We are currently working on a plan for this.

What next?

Notifications are complex, and we are taking the time to make sure that any design changes that we make are a genuine improvement. As I mentioned, we are currently working with a number of designs. Once we’ve developed one of them to a state where we are happy with it, I’ll try to post another update with details.

Notify me

Over the past several GNOME releases, we have been aiming to stabilise GNOME Shell as much as possible. We have been largely successful in this: the last major UI change was in 3.10, when we introduced the combined system status area, and the main improvements in the recent 3.12 release were for performance and bug fixing. This is a good thing. At the same time, there is one area where a number of us still feel that bigger changes are needed. This is notifications, particularly the Message Tray.

In this post, I’m going to present a new set of designs for notifications and the Message Tray, which we’re hoping to implement for the next GNOME release. As ever, these aren’t set in stone and are in a state of evolution. The aim of publicising the designs is to get feedback so we can improve them.

A bit of history

The original notifications design for GNOME 3 was introduced with 3.0. One of the main features of that design was being able to take actions directly from notifications. There was also an emphasis on integrated chat – you could reply to messages directly from a notification, and you could (and still can) see your conversations in the Message Tray and post to them from there. This feature influenced the basic design of the tray, which was structured as a set of “message sources” that you can interact with from the bottom of the screen.

The first iteration of the Message Tray implementation had some UI bugs with it, which we sought to address with a design update in 3.8. Despite these changes, we have continued to encounter a number of issues with the design of the Message Tray, and I think it’s fair to say that the tray hasn’t been performing as well as we’d hoped. Reasons for this include:

The layout of the tray as a strip of icons means that it doesn’t communicate a lot of information when it is first opened. It also makes it hard work to use, since each notification source has to be individually opened to get more information.
The tray is too difficult to open with a pointer, as it requires a lot of downward pressure against the bottom screen-edge.
There isn’t a way to quickly see how many notifications are in the tray, and there’s no reminder provided about waiting notifications.

Addressing these issues requires that we rethink the overall design of the Message Tray. This is also an opportunity to address a range of smaller interaction issues with the existing design.

This said, the original vision for notifications in GNOME 3 has some very positive aspects, which we want to preserve. Being able to directly act on notifications is a powerful feature, for example. The combination of a minimal message banner (the initial notification popup) which can be expanded is also good, as is the concept of having a tray where outstanding notifications can be viewed and interacted with.

Design goals

Before I get into the details of the design itself, it’s worth talking a little bit about the goals and principles behind it. As the new design has evolved, a number of themes and objectives have emerged, and reviewing these helps to clarify what the design aims to achieve. These goals include:

Be immediately useful. We want the tray to instantly provide useful information, and we want it to be possible to act on notifications straight away.
Make use of physical affordances. In particular, we want to make greater use of the screen edge when using the pointer. This should make interacting with notifications and the tray easier.
Sequential ordering. Notifications occur in time. Reflecting this in the layout of the Message Tray means that it will provide a more accurate representation of your notifications.
Advertise interactivity. Some interactive parts of the existing design often haven’t been noticed. We’ll be making these much more obvious with the new design.
Consistency and simplification. The existing design has been a bit burdened by its complexity, particularly because notifications are organised into different message sources. With the new design, we are pushing to simplify the design as much as possible. This should help to reduce the number of bugs, as well as improve usability, since we are aiming to ensure that notifications have a consistent appearance and layout.

The final goal is one that was at the core of the original design, and which is central to the design of GNOME 3 as a whole: that is, to be noticable and useful without being distracting. Wherever possible with GNOME 3, we have tried to produce a distraction-free experience which helps you concentrate on the task in hand. This requires a fine balancing act, which can be tricky to get right. With the new designs, we want to change that balance slightly, by making notifications a bit more noticable and by providing more effective reminders, but we still want to retain the emphasis on avoiding distraction.

A new design

The new design that we’re hoping to implement for GNOME 3.14 has been evolving for a while. There’s been a lot of experimentation in the design space, and quite a few concepts have been evaluated and thrown out. The design that is described below is the current state of that process. Since notifications can be quite complex, there are a lot of details involved. If you’re interested, there is plenty of information on the wiki.

The Message Tray

The most striking change with the new design is the introduction of a completely new Message Tray. The previous Message Tray design presented message sources as a set of icons in a strip along the bottom of the screen. With the new design, the tray contains a list of notifications, and slides up from the same bottom screen edge. This allows simplification, since the notifications are no longer reorganised into a set of message sources. The list is time-ordered, reflecting their sequential nature. It is also immediately useful, since you can see information about each notification straight away.

https://www.youtube.com/watch?v=C4TzqY0Ct90

The new design also changes the way that the tray is opened. Hovering or pushing on the screen edge will cause the tray to peek up, along with an indication of the number of notifications in the tray. Clicking the visible portion of the tray will cause it to open and a second click will cause it to hide again. The peeking tray, along with the notification count, will always be visible in the Activities Overview, in order to provide a reminding function.

This aspect of the design is intended to provide a quick way to check if you have missed any notifications, as well as to quickly inspect the contents of the tray itself. We also want to make the action of opening the tray much easier than it is now.

Lock Screen

One of the primary goals of the lock screen was to show missed notifications when you return to the computer. Designs for expanding the role of lock screen notifications have existed for some time, particularly with regards to being able to activate notifications from the lock screen. I’m not sure whether we’ll get chance to implement this aspect of the design for 3.14, but it should be said that the new design makes an effort to make the Lock Screen and Message Tray notifications as consistent as possible.

Notification banners

The existing formula for notification banners remains the same with the new design. Some aspects of the layout have changed, however: this enables them to have a consistent layout, irrespective of whether they are presented as a popup, are in the Message Tray, or are on the Lock Screen. The banners are bigger, which makes them more noticable.

Expanded banners also remain fundamentally the same, albeit with a modified layout. Buttons are now flush against the screen edge, making them easier to target with a pointer. We also plan to use hover states more readily, in order to communicate interactivity.

Next steps

Everything I have presented here is currently being treated as experimental. Jasper has an initial implementation in the works, which we are already testing, and are going to continue evaluating as it is developed. As usual, the design might change as we try things out, and we’ll only merge this into 3.14 if and when we feel that it is worthwhile.

You can help us to develop the designs with your questions, comments and feedback.

Looking Forward to 3.12

I usually do a review of what is coming in the run up to a release. However, there have been so many blog posts about 3.12 already that I don’t feel I need to go over individual features. If you haven’t read Planet GNOME in a while, now is a good time to check it out: there’s lots of great content on there right now.

It is worth looking at what the individual features in 3.12 add up to though. A release is more than the sum of its parts, and this is especially true of 3.12.

One important thing you will see in 3.12 is that, more and more, GNOME’s core applications are coming together. Videos will look and behave like a GNOME 3 app: it will let you browse your content, and it offers a modern, streamlined viewing experience. gedit has also had the GNOME 3 treatment. It has retained all its existing functionality, but in a more compact interface [1]. Many of the other apps have also matured of course, Software and Web in particular.

The other big news for 3.12 is that a number of significant gaps have been filled in. For a long time people have wanted to be able to manually organise their apps: now they can with the new apps folder feature. We’ve also added functionality to make installing sofware updates easier and more convenient, as well as the addition of wired networking controls to the system status area.

There are also major developments in the developer space, with the new notifications API, new GTK+ widgets, new capabilities for launching processes, and improved documentation. I think that 3.12 is probably our strongest for developers in a long time.

Finally, and for me perhaps most significantly, 3.12 looks set to be the best quality release so far. Signs of ongoing improvements are everywhere. There are performance gains for startup and (hopefully) memory usage, the theme and animations in the shell has been refined in quite a few subtle ways, high-resolution display support has been extended, and a great many bugs have been fixed. As each release comes and goes, GNOME 3 gets better and better, and 3.12 is no exception.

There’s plenty more that I could mention about this release, of course, and the release notes will provide full details, but what is important is the progress that GNOME is making. 3.12 feels like another significant upgrade, and is another release where it feels like things are coming together more and more.

[1] The other day I did a quick comparison, and found that the chrome in the new version is around 60 pixels shorter than before. That’s an impressive space saving, and makes the app much more focused on what you are editing.