Birds in flight

If you follow Planet GNOME, you’ll know about Jim Hall’s fantastic usability testing work. For years Jim has spearheaded usability testing on GNOME, both by running tests himself and mentoring usability testing internships offered through Outreachy.

This Autumn, Jim will once again be mentoring usability testing internships. However, this time round, we’re planning on running the internships a bit differently.

In previous rounds of usability testing, the tests have typically been performed on released software: that is, apps and features that are already in the hands of users. This is great and has flagged up issues that we’ve gone on to fix, but it has some drawbacks.

Most obviously, it means that users are exposed to the software prior to usability testing takes place. However, it also means that test results can take a long time to be corrected: active development of the software in question might have been paused by the time the tests are conducted, and it can sometimes take a while until a developer is able to correct any usability issues that have been identified.

Therefore, for this round of the Outreachy internships, we are only going to test UX changes that are actively being worked on. Instead of testing finished features, the tests will be on two things:

1. Mockups or prototypes of changes that we hope to implement soon (this can include static mockups and paper or software prototypes)
2. Features or UI changes that are being actively worked on, but haven’t been released to users yet

One goal is to increase the number of cycles of data-driven iteration that our UX work goes through. Ideally there should be multiple rounds of testing and design changes prior to coding even taking place! This will reduce the amount of UI changes that have to be made, and in turn reduce the amount of work for our developers.

Organising the tests in this way, we’re drawing on ideas from agile and lean. The plan is to have a predefined schedule of tests. When test day rolls around, we’ll figure out what we want to test. This will force a routine to our practice and ensure that we keep the exercise light and iterative.

There’s lots of work in progress UX work in GNOME right now, all of which would benefit from testing. This includes the new menu arrangements that are replacing app menus, new sound settings, new design patterns for lists, new application permission settings, the new lock screen design, and more.

One thing I’d actually love to see is design initiatives rejected outright because of testing feedback.

This approach to testing is an experiment and we’ll have to see how well it works in practice. However, if it does go well, I’m hopeful that we can incorporate it into our design and development practice more generally.

Jim and the rest of the design team will be looking for help from the rest of the GNOME community as we approach the test days. If anyone wants to help make prototypes or make sure that development branches can be easily run by our interns, your help would be extremely welcome. Likewise, we’d love to hear from anyone who has development work that they would like to have tested.