It’s been a while since we last blogged about the GNOME Shell design work that’s been happening. While we might not have blogged in a bit, there’s been a lot going on behind the scenes, particularly on the research side, and it’s about time that we told everyone about what we’ve been up to.
As a side note: a great team has developed around this initiative. The existing design team of Jakub, Tobias and myself has been joined by Maria Komarova from System76. Maria has a particularly strong research background and was immensely helpful in running interviews. The development side has also been fully engaged with the process, particularly through Georges and Florian.
Research and goals
In addition to thinking and discussing the design goals that we outlined previously (here and here), the design team has conducted two research exercises over the past six months. The main research exercise was a series of interviews with existing GNOME users. We also ran a small survey, to get some baseline numbers on user behaviour. I’ll mostly be talking about the interviews here.
The goals of the interviews was to a) get general information on how the shell gets used, b) evaluate how the existing shell design is performing, and c) identify ways in which the shell could be improved for our users. In addition to these fairly specific goals, we also wanted to develop a deeper understanding of our users: how they use their computers, their concerns, predispositions, interests and attitudes.
Interview-based research is a classic research approach for answering these kinds of questions, which are particularly appropriate to ask early on in a design process, when the emphasis is on exploration, insight and understanding. (Some readers might be more familiar with usability testing as the means through which data-driven design gets done, and that is a useful approach, but it typically happens later in the process.)
The survey
The survey element of our research was a smaller exercise than the interviews, so I won’t go into it in much detail. It was solely conducted within the confines of Red Hat’s Desktop Team, purely for convenience’s sake, and was able to provide some initial numbers for use by the design team.
The survey was an opportunity to ask users about their number of running apps, open windows, workspace usage, and a number of other behavioural factors. The key goal of the exercise was to identify typical workloads as well as the extremes, in order to inform our design requirements around the shell and the overview.
For example, through this exercise, we found out that the modal number of open windows was 8, and the highest number was 28. We also found that most people (63%) were using a single workspace, and that the largest number of workspaces in use was 10.
Clearly, such a limited exercise has to be treated with a high degree of caution, but some quick preliminary numbers were a useful frame of reference and are something that we hope to build upon in the future.
The interviews
The rest of this post will be devoted to the interview exercise that we conducted. Here’s what we did…
We conducted seven interviews in total, each conducted remotely. Interviewees were recruited from through the team’s professional contacts. We avoided talking to anyone who is directly involved in desktop development, and we tried to recruit research participants with a range of roles and technical backgrounds. We had some developers, but we also had people who work in sales, marketing and support.
During each interview, we got general background information about the interviewee and their computing activities. Then we conducted a screen sharing exercise in which the interviewee showed us how their desktop was set up. Then we ran through some exercises to see how they performed basic tasks.
We also used the interviews to ask evaluative questions, to see what the interviewee thought about the current shell design, and how it compares to other desktops.
We took notes on each interview. We mapped out the responses we got. We identified themes and patterns.
Research results
Let’s start by talking about the user behaviours that we observed. A quick summary:
- The number of apps in use was generally low, 3-5 being the common range, both in the interviews and the survey.
- Search was prominent as a way of launching apps, but we also observed occasional pointer-driven app launching (curiously, this seemed common when launching some apps and not others).
- Engagement with the app grid was very low.
- About half the interviewees had customised the favourites in the dash.
- Workspace usage was quite mixed: some used workspaces extensively, while others never touched them [1, 2].
These findings have direct relevance to the design areas that the team are investigating. For example, they imply that our designs need to cater both to users who use workspaces extensively, and those who don’t.
The results also posed some conundrums. For instance, is lack of engagement with the current app grid an indication that its design needs to be improved, or that the feature should be demoted or relegated, somehow?
Likes and dislikes
While much of the research findings related to observed behaviours, the interviews also allowed us to ask evaluative research questions. We wanted to know how the existing GNOME Shell design is performing, and how it can be improved.
The main thing that came across here is that overall, the existing design appeared to be working well for the users we spoke to. With one exception, all the interviewees thought that the design compared favourably with other desktops they’d used. (Of course, this insight needs to be taken with a touch of salt [3].)
Most of the interviewees identified things that they like about the current shell design. They seemed to really like the search feature, in particular its speed, but also that it’s just a single keystroke away. The overview also received regular praise (one interviewee described it as “fantastic”).
The other thing that repeatedly came up as a positive aspect of the existing design is its none-distracting nature. Interviewees paid the shell compliments by saying things like “it’s not cluttered” and “it gets out of my way”.
The main issues with the current design that came out of the research was the frequent lack of engagement with the dash and app grid. We also got a hint that onboarding might be a struggle for some, and we got a clear message that touchpad gestures need to be improved (confirming one of our initial hypotheses).
What’s next?
We are already planning additional rounds of research (which we will endeavour to blog about in a more timely fashion!) Some of these will be open to wider participation so, if you are interested, we would love to have you involved!
The design team is also exploring various concepts for the shell, which we hope to present in the not too distant future. Indeed, one of the reasons for all this research work is to help us be confident about the designs before we present them to the world! Watch this space.
Footnotes
[1] There was also behavioural variation within the group of workspace users: some had developed systems for themselves, with certain workspaces dedicated to particular activities, while others allowed their workspace usage to evolve organically as they worked.
[2] Workspace usage did not appear to be positively related to technical expertise. If anything, the opposite was true, with the less technically-expert making more use of workspaces rather than less.
[3] Some notes on: representativeness and generalisability. How can only seven users provide useful results? Won’t we have missed critical user types? Won’t the sample be unrepresentative?
The first thing to say here is that small-N studies (those with a small number of participants) are valuable and can deliver significant insights, and have important advantages over large-N surveys. In particular, they allow accurate observations to be made, without making assumptions about what’s going to be seen.
Second, small-N studies like this allow relationships to be identified, and these can be used to infer qualities of a wider population. For example, if you observe that technically expert users tend to use the keyboard more, and you know that you have a lot of technically expert users, you can make a guess that keyboard use is important for the population as a whole.
Third and finally, experience tells us that small research studies can and do pick up on general trends and behaviours with remarkable speed (such as with the rule of five in user testing).
That said, we certainly do need to acknowledge the non-representative nature of our interview sample. We did include some relatively non-technical users, but the sample as a whole was skewed towards the more technical/professional end of the spectrum, and was taken from organisations that are invested in the Linux/GNOME desktop to varying degrees.
This doesn’t mean that the results we got are invalid. We are confident that our research findings are accurate with regards to the people we spoke to, and that those people are indicative of a larger section of the GNOME user population. The only question is is whether there are other types of user who we didn’t include.