On Problems with Vala

If you’re going to be writing a new application based on GNOME technologies and targeting the GNOME ecosystem, then you should seriously consider writing it in the Vala programming language.

That’s a pretty controversial statement! Emmanuele just told us that Vala is dying and that you should find an alternative. So, if I’m recommending that you start writing new applications in Vala, clearly I disagree with him at least somewhat. Even so, I actually think pretty much all of Emmanuele’s points are correct. Vala really is dying! The status quo really is pretty bad. Using a dying programming language to write your application is rarely a good idea. You should think twice before doing so.

Still, I wouldn’t be so quick to write off Vala. For one thing, it’s a pleasure to use. The design of the language is very good. The integration with GObject and the GNOME ecosystem, from GObject signals and properties to native support for D-Bus and composite GTK+ widget templates, is second to none, and will probably never be surpassed by another language. It’s hard to understate how good the syntax of the language is, and how tailored it is for GNOME programming. People like Vala for good reasons.

Emmanuele says that it’s time to look at alternatives to Vala, but the alternatives we have to Vala right now have big problems too. If I were to start writing a new GNOME app today, Vala is still the language I would use. So now I have to try to convince you of that! First, let’s look at current problems with Vala in more detail. Then, let’s look into the alternatives we have available.

The problems with Vala are real and very serious, so I can only give it that qualified recommendation. The Vala community is slowly dying, and I would not recommend starting a big, complex application in Vala today, given the risk that the compiler might be completely unmaintained in a few years. But most GNOME applications are fairly small — only a few, like Builder or Evolution or Epiphany, are big and complex — and I think most will probably do well enough in the long run with Vala even if the Vala compiler stops improving.

Problems with Vala

Yeah, I’m afraid this is not going to be a short post. Let’s take this in two: first, common complaints that I don’t think are actually serious problems, and second, the actual serious problems.

Minor Problems with Vala

Let me start off by pointing out a couple things that I don’t consider to be serious problems with Vala: bindings issues and tooling issues.

People often complain that there are bugs with the bindings. And there are! Debugging bindings bugs is not fun at all. But I get the impression that bindings complaints are generally about the state of the bindings five years ago. The bindings situation is a lot better now than it was then, and it is constantly improving. Vala bindings actually are well-maintained (thanks mostly to Rico Tzschichholz); it’s only the compiler itself that is having maintenance problems. And most of the bindings problems are caused by introspection issues in the upstream libraries. That means that if you’re hitting a bindings problem in Vala, it’s probably a problem in every other language you might want to use as well… except C and C++, of course. And bindings issues are actually arguably far easier to debug in Vala than they would be in Python or JavaScript, since you can look for errors in the generated C code. Fixing bindings is generally easy, and you can work around the problems using a drop-in vapi file if you can’t wait to get the fix upstreamed. Adding new bindings is work if the library is not introspectable, but much easier than it is in other languages. No doubt programming would be nicer if bindings were not necessary, but unless you want to write everything in C or C++, bindings are a fact of life that won’t go away, and Vala’s are pretty darn good.

As far as tooling: it’s true that the Vala ecosystem does not have great tools, but I don’t think this is really a horrible problem. The most common complaint I see is that debugging requires looking at generated C code, and there’s no special Vala debugger. Now, in the case of crashes, it usually does indeed require looking at the generated code. But, in my experience, crashes are much, much rarer in applications that are written in Vala, so we’re going to be spending a lot less time in the debugger than we would when working on C applications anyway. And debugging the generated code with gdb isn’t so horrible. It’s hardly a great experience, but you get used to it. Be sure that you’re using vala -g to emit Vala line numbers into the generated code, otherwise you’re just making your life unnecessarily difficult. At any rate, gdb plus line numbers is the way to go here. Vala debugging is never going to be as simple as C or C++ debugging, but you’ll have to do less of it than you would in C or C++, and that’s a reasonable trade-off.

Another problem with Vala is that it suffers from the same safety issues as C and C++. You will make mistakes, and your mistakes will allow remote attackers to take control of your users’ computers. Vala doesn’t do anything to avoid buffer overflows, for instance. That’s pretty bad. But you will at least make fewer mistakes than you would in C or C++. For instance, I believe the language makes refcounting errors an order of magnitude less likely, drastically reducing the number of use-after-free vulnerabilities in your code. Compared to Rust or Python or JavaScript, this is not very good at all, but compared to C or C++, it’s excellent.

Major Problems with Vala

I see two serious problems with Vala. The first is that the compiler has bugs, and debugging compiler bugs is very unpleasant. The second is that the compiler is not well-maintained. Like Emmanuele says, the Vala community is dying, or, if you want to be generous, at least not in a very healthy state. So when you report compiler bugs, probably nobody is going to fix those bugs. This can be very frustrating.

Vala Bugs

I can confidently say that Vala has more bugs than any other programming language you might be considering using for GNOME development. It’s sad, but true. Most of the bugs are just small annoyances; for instance, bugs in which the Vala compiler emits C code that does not actually compile. These are usually easy to work around, but that can be pretty annoying. Other bugs are more serious. For instance, see signal handler spuriously runs when signal is emitted by object not connected to once every 98 emissions (which was fixed a few years ago, but a good example of how Vala bugs can cause runtime problems) or Incorrect choice of signal marshaller causes crash when promoting a pawn in GNOME Chess when built with Fedora or Debian hardening flags (still broken).

Of course, all bugs are fragile if there is an active community of developers fixing them. But, as Emmanuele has already pointed out, that is not going so well.

Vala Maintainership and Community

Vala’s greatest strength — its focus on GNOME — is also its greatest weakness. Vala is not very interesting to anyone outside the GNOME and GTK+ development communities. Accordingly, the community of Vala developers and maintainers is several orders of magnitude smaller than other programming language communities.

Relative to the fairly small size of the GNOME ecosystem, there are actually a very large number of Vala applications in existence. (All of elementary’s applications use Vala, for example.) So there is a relatively large number of Vala application maintainers with a stake in the success of the Vala project. But they’re mostly focused on developing their applications, not Vala itself. A programming language is probably not the greatest tool for any job if it requires that you participate in maintaining the compiler, after all. And the barrier for entry to Vala compiler development is high. For starters, compilers are difficult and complicated; working on a compiler is far more difficult than working on desktop applications. Moreover, of the people who are motivated to contribute to the compiler and submit a patch, most probably get discouraged pretty quickly, because most patches posted on Bugzilla do not get reviewed. There are currently 179 unreviewed patches in Vala’s request queue. The oldest patch there is 2,556 days old, so we know that it’s been seven years since anyone has cared for the outstanding patches. Any of those discouraged contributors might have eventually turned into Vala maintainers if only their patches were reviewed. Of course, most would not have, but if only one or two of the people who submitted patches was an active Vala maintainer today, the project would be in a significantly better state. And I see patches there from a large number of different developers.

But who can review the patches? Vala needs more maintainers. Rico is taking good care of the bindings and appears to be committing patches to the compiler as well, but he’s just one person and can’t do it alone. Vala stakeholders need to increase investment in the compiler. But this is a familiar problem: the majority of our modules need more maintainers. Maintainers do not grow on trees. Ideally a company will step in to support Vala compiler development, but few companies seem to have taken an interest in Vala, so this doesn’t seem likely. This is unfortunate.

I frankly expect that Emmanuele’s prediction will prove true, and that the Vala situation will only get worse in the next five years. It’s more likely than not. But I’m not very confident in that guess! Several people have contributed significant patches to the Vala compiler recently. (Carlos Garnacho, you have earned much beer.) The future is still uncertain. I very much hope that my pessimistic expectation is proven wrong, and that the maintainership situation will improve soon.

But while the Vala compiler may stagnate, it’s probably not going to get worse. I think it’s good enough for writing GNOME applications today, and I expect it will still be good enough in five years.

Alternatives to Vala

So Vala is not in great state. What else can we use to write GNOME applications? The only serious programming languages in the GNOME ecosystem are C, C++, Vala, Python (using PyGObject), and JavaScript (using gjs). No, I did not miss any options. If your favorite language isn’t listed there, it’s either because (a) it doesn’t have decent GObject bindings, or (b) the language is not popular at all. To my knowledge, all GNOME software is written in one of those five languages, except for a couple old applications that use C#. And the state of C# in GNOME makes Vala look like an active vibrant language. If you want to start writing a GTK+ 2 app in 2017, go ahead and use C#. The rest of us will restrict our search to C, C++, Vala, Python, and JavaScript.

(Tangent time! Rust is trendy, but I’m told it needs more help to improve the GObject bindings before we start using it in applications. I’m hoping that it will emerge as the superior option in the not so distant future, but it’s definitely not ready for use in GNOME yet. It has to have better GObject integration. It has to have some degree of ABI stability, even if it’s limited. Dynamic linking has to be the default. It’s not going to be successful in the GNOME community otherwise. You should join the Rust folks and help out!)

Let’s start with C. C is undoubtedly the most popular language used in GNOME programming, but it would be flatly irresponsible to choose it for writing new applications. I enjoy writing C, but like everyone else, I make mistakes, and I think it would be desirable if my programming mistakes did not allow attackers to execute arbitrary code on your computer. It’s also extremely verbose, requiring far more lines of code to do simple things than the other programming languages that we’re considering do. C is not a reasonable option for new applications in 2017, even if it is the language you are most familiar with. I wouldn’t go so far as to say that our existing applications need to be rewritten in a safer language, because rewriting applications is hard and our developer community is small, but I certainly would not want to start writing any new applications in C. We need a C migration plan.

Modern C++ is a bit safer and much more pleasant to use than C, but that’s really not saying all that much. Footguns abound. You have to know all sorts of arcane rules to use it properly. The barrier for entry to new contributors is much higher than it is with C. Developers still make lots of mistakes, and those mistakes still allow remote attackers to take control of your users’ computers. So C++ is not a good choice for new applications either.

Python… OK, I suppose Python is pretty good, if you’re willing to give up compiler errors and static typing. I prefer to use a compiled language for writing serious software, because I make a lot of mistakes, and I’d rather the compiler catch those mistakes when possible than find out about them at runtime. So I would still prefer Vala. But if you prefer scripting languages, then Python is just fine, and doesn’t suffer from any of the disadvantages of Vala, and you should use it for your new app. Some developers have mentioned that there are some gotchas and interoperability issues with moving between Python APIs and GNOME APIs, but no programming environment is ever going to be perfect. PyGObject is good enough, and I’m pretty sure we’re going to be using it for a long time.

The last option is JavaScript. With all due respect to the gjs folks — and Philip Chimento in particular, who has been working hard at Endless to improve the JavaScript experience for GNOME developers — there’s no way to change the reality that JavaScript is a terrible language. It has close to zero redeeming features, and many confusing ones. You use it in web browsers because you have to, but for a desktop application, I have no clue why you would choose to use this over Python. We have to maintain gjs forever (for some value of “forever”) because GNOME Shell uses it, and it’s also being used by a couple apps like GNOME Weather and GNOME Documents. But it should be your last choice for a desktop application. Do not use JavaScript for new projects.

Another disadvantage of using JavaScript is that there is a huge barrier to entry for newcomers. But wait, lots of web developers are familiar with JavaScript; wasn’t the whole point of using it to lower the barrier of entry to newcomers? Well look how well that worked out for us! We have approximately zero new developers flocking to work on our JavaScript applications. The only documentation currently available online is over three years old, covers only a subset of the introspectable libraries that you want to use, and is frankly pretty bad. Unless opening gir files in a text editor and reading internal gjs unit tests to figure out how to call functions sounds like a good newcomer experience to you, then we need to steer far clear of JavaScript. The documentation situation is a fixable problem — Philip has much improved documentation that’s just waiting for hosting to materialize — but there’s no momentum to fix it right now, and the defects of the language can’t ever be fixed.

So all of the alternatives to Vala have big problems too, except maybe for Python, which is not a compiled language, which many of us would consider a serious disadvantage in itself. If you don’t want to use Vala, you have to pick one of the alternatives. So which will it be? I have no doubt that many or even most of our community places different weight on the various advantages and disadvantages of the languages. I actually expect mine is a minority opinion. But at the very least, I think I’ve shown why Vala still seems attractive to many developers.

(Note that the above analysis does not apply to libraries. You cannot write a system library in Python or JavaScript. You can do so with Vala or C++, but it requires special care. GNOME platform libraries must have a C API in order to be introspectable and useful.)

Conclusion

If you ignore its bugs and its maintainership status, Vala is by far the best language for writing GNOME applications. But those are pretty big things to ignore. I’d still use it anyway. It’s hard to understate how pleasant it is to develop with. The most frequent complaints I see are about problems that I don’t actually consider very serious. I don’t know. I also don’t know what the language of GNOME’s future is, but I do know that we need to stop writing new applications in C, and until GObject integration for Rust is ready, Vala still seems like our best shot at that.

Who Maintains That Stuff?

If you use GNOME or Ubuntu, then GNOME Disks is probably what you rely on if you ever need to do any disk management operations, so it’s a relatively important piece of software for GNOME and Ubuntu users. Now if you’re a command line geek, you might handle disk management via command line, and that’s fine, but most users don’t know how to do that. Or if you’re living in the past like Ubuntu and not yet using Wayland, you might prefer GParted (which does not work under Wayland because it requires root permissions, while we intentionally will not allow applications to run as root in Wayland). But for anyone else, you’re probably using GNOME Disks. So it would be good for it to work reliably, and for it to be relatively free of bugs.

I regularly receive new bug reports against GNOME Disks. Sometimes they’re not very well-constructed or based on some misunderstanding of how partitioning works, in which case I’ll close them, but most of them are good and valid. So who fixes bug reports against GNOME Disks? The answer is: nobody! Unless it’s really, really easy — in which case I might allocate five minutes for it — nobody is going to fix the bug that you reported. What a shame!

Who is the maintainer? In this case, it’s me, but I don’t actually know much anything about the application and certainly don’t have time to fix things; I just check Bugzilla to see if anybody has posted a patch, so that contributors’ patches (which are rare) don’t get totally neglected, and make new releases every once in a while, and only because I didn’t want to see such a critical piece of software go completely unmaintained.

If you’re a software developer with an interest in both GNOME and disk management, GNOME Disks would be a great place to help out. A great place to start would be to search through GNOME Bugzilla for issues to work on, and submit patches for them.

Of course, Disks is far from the only unmaintained or undermaintained software in GNOME. Last year, Sébastien set up a wiki page to track unmaintained and undermaintained apps. It has had some success: in that time, GNOME Calculator, Shotwell, Gtranslator, and Geary have all found maintainers and been removed from the list of unmaintained modules. (Geary is still listed as undermaintained, and no doubt it would be nice to have more Geary maintainers, but the current maintainer seems to be quite active, so I would hesitate to list it as undermaintained. Epiphany would love to have a second maintainer as well. No doubt most GNOME apps would.)

But we still have a few apps that are listed as unmaintained:

  • Bijiben (GNOME Notes)
  • Empathy
  • GNOME Disks

No doubt there are more GNOME modules that should be listed. If you know of some, please add them or leave a comment here.

Help would be very much welcome with any of these. In particular, Empathy and Bijiben are both slated to be removed from Fedora beginning with Fedora 27 due to their unacceptable dependencies on an old, insecure version of WebKitGTK+ that is about to be removed from the distribution. Most of the work to port these applications to modern WebKitGTK+ is already done (and, in the case of Empathy, I’ve already committed the port to git), but an active maintainer is required to finish the job and get things to a releasable state. Last I checked, Bijiben also still needed to be ported to GTK+ 3.20. If nobody is interested in helping out, these apps are going to disappear sooner rather than later.

Disks, fortunately, is not going to disappear anytime soon. But the bugs aren’t going to fix themselves.

P.S. This blog is not the right place to complain about no longer allowing applications to run as root. Such applications can and should use Polkit to move privileged operations out of the GUI and into a helper process. This should have been done roughly a decade ago. Such applications might themselves be unmaintained or undermaintained; can you help them out?

An Update on WebKit Security Updates

One year ago, I wrote a blog post about WebKit security updates that attracted a fair amount of attention at the time. For a full understanding of the situation, you really have to read the whole thing, but the most important point was that, while WebKitGTK+ — one of the two WebKit ports present in Linux distributions — was regularly releasing upstream security updates, most Linux distributions were ignoring the updates, leaving users vulnerable to various security bugs, mainly of the remote code execution variety. At the time of that blog post, only Arch Linux and Fedora were regularly releasing WebKitGTK+ updates, and Fedora had only very recently begun doing so comprehensively.

Progress report!

So how have things changed in the past year? The best way to see this is to look at the versions of WebKitGTK+ in currently-supported distributions. The latest version of WebKitGTK+ is 2.14.3, which fixes 13 known security issues present in 2.14.2. Do users of the most popular Linux operating systems have the fixes?

  • Fedora users are good. Both Fedora 24 and Fedora 25 have the latest version, 2.14.3.
  • If you use Arch, you know you always have the latest stuff.
  • Ubuntu users rejoice: 2.14.3 updates have been released to users of both Ubuntu 16.04 and 16.10. I’m very  pleased that Ubuntu has decided to take my advice and make an exception to its usual stable release update policy to ensure its users have a secure version of WebKit. I can’t give Ubuntu an A grade here because the updates tend to lag behind upstream by several months, but slow updates are much better than no updates, so this is undoubtedly a huge improvement. (Anyway, it’s hardly a bad idea to be cautious when releasing a big update with high regression potential, as is unfortunately the case with even stable WebKit updates.) But if you use the still-supported Ubuntu 14.04 or 12.04, be aware that these versions of Ubuntu cannot ever update WebKit, as it would require a switch to WebKit2, a major API change.
  • Debian does not update WebKit as a matter of policy. The latest release, Debian 8.7, is still shipping WebKitGTK+ 2.6.2. I count 184 known vulnerabilities affecting it, though that’s an overcount as we did not exclude some Mac-specific security issues from the 2015 security advisories. (Shipping ancient WebKit is not just a security problem, but a user experience problem too. Actually attempting to browse the web with WebKitGTK+ 2.6.2 is quite painful due to bugs that were fixed years ago, so please don’t try to pretend it’s “stable.”) Note that a secure version of WebKitGTK+ is available for those in the know via the backports repository, but this does no good for users who trust Debian to provide them with security updates by default without requiring difficult configuration. Debian testing users also currently have the latest 2.14.3, but you will need to switch to Debian unstable to get security updates for the foreseeable future, as testing is about to freeze.
  • For openSUSE users, only Tumbleweed has the latest version of WebKit. The current stable release, Leap 42.2, ships with WebKitGTK+ 2.12.5, which is coincidentally affected by exactly 42 known vulnerabilities. (I swear I am not making this up.) The previous stable release, Leap 42.1, originally released with WebKitGTK+ 2.8.5 and later updated to 2.10.7, but never past that. It is affected by 65 known vulnerabilities. (Note: I have to disclose that I told openSUSE I’d try to help out with that update, but never actually did. Sorry!) openSUSE has it a bit harder than other distros because it has decided to use SUSE Linux Enterprise as the source for its GCC package, meaning it’s stuck on GCC 4.8 for the foreseeable future, while WebKit requires GCC 4.9. Still, this is only a build-time requirement; it’s not as if it would be impossible to build with Clang instead, or a custom version of GCC. I would expect WebKit updates to be provided to both currently-supported Leap releases.
  • Gentoo has the latest version of WebKitGTK+, but only in testing. The latest version marked stable is 2.12.5, so this is a serious problem if you’re following Gentoo’s stable channel.
  • Mageia has been updating WebKit and released a couple security advisories for Mageia 5, but it seems to be stuck on 2.12.4, which is disappointing, especially since 2.12.5 is a fairly small update. The problem here does not seem to be lack of upstream release monitoring, but rather lack of manpower to prepare the updates, which is a typical problem for small distros.
  • The enterprise distros from Red Hat, Oracle, and SUSE do not provide any WebKit security updates. They suffer from the same problem as Ubuntu’s old LTS releases: the WebKit2 API change  makes updating impossible. See my previous blog post if you want to learn more about that. (SUSE actually does have WebKitGTK+ 2.12.5 as well, but… yeah, 42.)

So results are clearly mixed. Some distros are clearly doing well, and others are struggling, and Debian is Debian. Still, the situation on the whole seems to be much better than it was one year ago. Most importantly, Ubuntu’s decision to start updating WebKitGTK+ means the vast majority of Linux users are now receiving updates. Thanks Ubuntu!

To arrive at the above vulnerability totals, I just counted up the CVEs listed in WebKitGTK+ Security Advisories, so please do double-check my counting if you want. The upstream security advisories themselves are worth mentioning, as we have only been releasing these for two years now, and the first year was pretty rough when we lost our original security contact at Apple shortly after releasing the first advisory: you can see there were only two advisories in all of 2015, and the second one was huge as a result of that. But 2016 seems to have gone decently well. WebKitGTK+ has normally been releasing most security fixes even before Apple does, though the actual advisories and a few remaining fixes normally lag behind Apple by roughly a month or so. Big thanks to my colleagues at Igalia who handle this work.

Challenges ahead

There are still some pretty big problems remaining!

First of all, the distributions that still aren’t releasing regular WebKit updates should start doing so.

Next, we have to do something about QtWebKit, the other big WebKit port for Linux, which stopped receiving security updates in 2013 after the Qt developers decided to abandon the project. The good news is that Konstantin Tokarev has been working on a QtWebKit fork based on WebKitGTK+ 2.12, which is almost (but not quite yet) ready for use in distributions. I hope we are able to switch to use his project as the new upstream for QtWebKit in Fedora 26, and I’d encourage other distros to follow along. WebKitGTK+ 2.12 does still suffer from those 42 vulnerabilities, but this will be a big improvement nevertheless and an important stepping stone for a subsequent release based on the latest version of WebKitGTK+. (Yes, QtWebKit will be a downstream of WebKitGTK+. No, it will not use GTK+. It will work out fine!)

It’s also time to get rid of the old WebKitGTK+ 2.4 (“WebKit1”), which all distributions currently parallel-install alongside modern WebKitGTK+ (“WebKit2”). It’s very unfortunate that a large number of applications still depend on WebKitGTK+ 2.4 — I count 41 such packages in Fedora — but this old version of WebKit is affected by over 200 known vulnerabilities and really has to go sooner rather than later. We’ve agreed to remove WebKitGTK+ 2.4 and its dependencies from Fedora rawhide right after Fedora 26 is branched next month, so they will no longer be present in Fedora 27 (targeted for release in November). That’s bad for you if you use any of the affected applications, but fortunately most of the remaining unported applications are not very important or well-known; the most notable ones that are unlikely to be ported in time are GnuCash (which won’t make our deadline) and Empathy (which is ported in git master, but is not currently in a  releasable state; help wanted!). I encourage other distributions to follow our lead here in setting a deadline for removal. The alternative is to leave WebKitGTK+ 2.4 around until no more applications are using it. Distros that opt for this approach should be prepared to be stuck with it for the next 10 years or so, as the remaining applications are realistically not likely to be ported so long as zombie WebKitGTK+ 2.4 remains available.

These are surmountable problems, but they require action by downstream distributions. No doubt some distributions will be more successful than others, but hopefully many distributions will be able to fix these problems in 2017. We shall see!

On Epiphany Security Updates and Stable Branches

One of the advantages of maintaining a web browser based on WebKit, like Epiphany, is that the vast majority of complexity is contained within WebKit. Epiphany itself doesn’t have any code for HTML parsing or rendering, multimedia playback, or JavaScript execution, or anything else that’s actually related to displaying web pages: all of the hard stuff is handled by WebKit. That means almost all of the security problems exist in WebKit’s code and not Epiphany’s code. While WebKit has been affected by over 200 CVEs in the past two years, and those issues do affect Epiphany, I believe nobody has reported a security issue in Epiphany’s code during that time. I’m sure a large part of that is simply because only the bad guys are looking, but the attack surface really is much, much smaller than that of WebKit. To my knowledge, the last time we fixed a security issue that affected a stable version of Epiphany was 2014.

Well that streak has unfortunately ended; you need to make sure to update to Epiphany 3.22.6, 3.20.7, or 3.18.11 as soon as possible (or Epiphany 3.23.5 if you’re testing our unstable series). If your distribution is not already preparing an update, insist that it do so. I’m not planning to discuss the embarrassing issue here — you can check the bug report if you’re interested — but rather on why I made new releases on three different branches. That’s quite unlike how we handle WebKitGTK+ updates! Distributions must always update to the very latest version of WebKitGTK+, as it is not practical to backport dozens of WebKit security fixes to older versions of WebKit. This is rarely a problem, because WebKitGTK+ has a strict policy to dictate when it’s acceptable to require new versions of runtime dependencies, designed to ensure roughly three years of WebKit updates without the need to upgrade any of its dependencies. But new major versions of Epiphany are usually incompatible with older releases of system libraries like GTK+, so it’s not practical or expected for distributions to update to new major versions.

My current working policy is to support three stable branches at once: the latest stable release (currently Epiphany 3.22), the previous stable release (currently Epiphany 3.20), and an LTS branch defined by whatever’s currently in Ubuntu LTS and elementary OS (currently Epiphany 3.18). It was nice of elementary OS to make Epiphany its default web browser, and I would hardly want to make it difficult for its users to receive updates.

Three branches can be annoying at times, and it’s a lot more than is typical for a GNOME application, but a web browser is not a typical application. For better or for worse, the majority of our users are going to be stuck on Epiphany 3.18 for a long time, and it would be a shame to leave them completely without updates. That said, the 3.18 and 3.20 branches are very stable and only getting bugfixes and occasional releases for the most serious issues. In contrast, I try to backport all significant bugfixes to the 3.22 branch and do a new release every month or thereabouts.

So that’s why I just released another update for Epiphany 3.18, which was originally released in September 2015. Compare this to the long-term support policies of Chrome (which supports only the latest version of the browser, and only for six weeks) or Firefox (which provides nine months of support for an ESR release), and I think we compare quite favorably. (A stable WebKit series like 2.14 is only supported for six months, but that’s comparable to Firefox.) Not bad?

On Ignorance, Intolerance, and Bigotry

It seems incredible that lawful permanent residents of the United States are stranded abroad, prohibited from boarding flights home, for such a capricious reason as being unfortunate enough to be traveling at the wrong time. (This is not even to mention the plight of millions of innocent refugees fleeing violence and terror, who are no less deserving of justice.) And yet, here we are.

Who do you know who is affected by Friday’s executive order?

One of my friends in college was of Iranian descent. Years ago, he joined the US army and risked his life fighting for our country in Iraq. Later, he visited his extended family in Iran, fearful that the government would imprison him if it discovered he had served in our army. Now he cannot go back, due to Iran’s entirely-justified reciprocal ban on Americans. When will he be able to see his family again? Will this really only last 90 days?

Who do you know?

I should not have had to detail my friend’s military service or present him as a sympathetic character. It should not matter. Equality is supposed to be one of the uniting principles of our country. We have a long history of failing in this regard, but it has mostly been a history of progress in the right direction. Clearly, that is no longer the case.

So who do you know? If you do not know anyone affected by yesterday’s executive action, perhaps you should think twice before voting for ignorance, irrational fear, hate, and bigotry. Of course I mean that you should think twice before voting for the Republican Party. If you still, after this weekend, do not believe that is what the party now stands for, then you are long overdue for a reality check.

The great irony of the just and tolerant society is that it must refuse to tolerate intolerance. At this, we have failed.

I have never before today been so ashamed of my country. It’s not like we didn’t know this was coming. We have brought it on ourselves via a legitimate democratic election (of which, absurdly, only the winner contends was marred by massive fraud). Donald Trump campaigned on his Muslim ban, and he is only delivering as promised.

Things are going to get much, much worse before they get better, but at least we have some reason for hope. The United States is fortunate in that it has a strong, independent judiciary. It is nothing short of amazing that lawyers representing victims detained at US airports have been able to win multiple injunctions barring their deportation in just one day. (If you’re not already a proud supporter of the ACLU like me, you should fix that right now.) That strong judiciary also protects our First Amendment rights (which do not, by the way, extend to my personal blog; hateful comments here will not be approved). As we enter the post-truth society where Republicans believe a separate set of “alternative facts,” it remains to be seen what all speech can still accomplish, but now is surely the time to find out. Do not remain silent. If you use social media or have a blog, you have a duty now to express your dissent. Do your part to move the needle of public opinion.

You know, you don’t need to know anyone to see that this is wrong.

git commit –amend –date

I accidentally committed something with git with the wrong timezone set in my VM. How to fix it? The help message for git commit says:

--date <date>         override date for commit

OK that’s great, but I don’t really want to figure out the input format for feeding in a date. I blindly tried:

git commit --amend --date now

And blindly typing “now” actually worked as intended. git is smart. Thanks git developers!

Epiphany Icon Refresh

We have a nice new app icon for Epiphany 3.24, thanks to Jakub Steiner (Update: and also Lapo Calamandrei):

new-icon
Our new icon. Ignore the version numbers, it’s for 3.24.

Wow pretty!

The old icon was not actually specific to Epiphany, but was taken from the system, so it could be totally different depending on your icon theme. Here’s the icon currently used in GNOME, for comparison:

old-icon
The old icon, for comparison

You can view the new icon it in its full 512×512 glory by navigating to about:web:

big-icon
It’s big (click for full size)

(The old GNOME icon was a mere 256×256.)

Thanks Jakub!

GNOME 3.22 core apps

GNOME 3.22 is scheduled to be released today. Along with this release come brand new recommendations for distributions on which applications should be installed by default, and which applications should not. I’ve been steadily working on these since joining the release team earlier this year, and I’m quite pleased with the result.

When a user installs a distribution and boots it for the first time, his or her first impression of the system will be influenced by the quality of the applications that are installed by default. Selecting the right set of default applications is critical to achieving a quality user experience. Installing redundant or overly technical applications by default can leave users confused and frustrated with the distribution. Historically, distributions have selected wildly different sets of default applications. There’s nothing inherently wrong with this, but it’s clear that some distributions have done a much better job of this than others. For instance, a default install of Debian 8 with the GNOME desktop includes two different chess applications, GNOME Chess and XBoard. Debian fails here: these applications are redundant, for starters, and the later app looks like an ancient Windows 95 application that’s clearly out of place with the rest of the system. It’s pretty clear that nobody is auditing the set of default applications here, as I doubt anyone would have intentionally included Xboard; it turns out that XBoard gets pulled in by Recommends via an obscure chess engine that’s pulled in by another Recommends from GNOME Chess, so I presume this is just an accident that nobody has ever cared to fix. Debian is far from the only offender here; you can find similar issues in most distributions. This is the motivation for providing the new default app recommendations.

Most distributions will probably ignore these, continue to select default apps on their own, and continue to do so badly. However, many distributions also strive to provide a pure, vanilla GNOME experience out-of-the-box. Such distributions are the target audience for the new default app guidelines. Fedora Workstation has already adopted them as the basis for selecting which apps will be present by default, and the result is a cleaner out-of-the-box experience.

Update: I want to be clear that these guidelines are not appropriate for all distros. Most distros are not interested in providing a “pure GNOME experience.” Distros should judge for themselves if these guidelines are relevant to them.

Classifications

The canonical source of these classifications is maintained in JHBuild, but that’s not very readable, so I’ll list them out here. The guidelines are as follows:

  • Applications classified as core are intended to be installed by default. Distributions should only claim to provide a vanilla GNOME experience if all such applications are included out-of-the-box.
  • Applications classified as extra are NOT intended to be installed by default. Distributions should not claim to provide a vanilla GNOME experience if any such applications are included out-of-the-box.
  • Applications classified as Incubator are somewhere in between. Incubator is a classification for applications that are designed to be core apps, but which have not yet reached a high enough level of quality that we can move them to core and recommend they be installed by default. If you’re looking for somewhere to help out in GNOME, the apps marked Incubator would be good places to start.

Core apps

Distributions that want to provide a pure GNOME experience MUST include all of the following apps by default:

  • Archive Manager (File Roller)
  • Boxes
  • Calculator
  • Calendar
  • Characters (gnome-characters, not gucharmap)
  • Cheese
  • Clocks
  • Contacts
  • Disk Usage Analyzer (Baobab)
  • Disks
  • Document Viewer (Evince)
  • Documents
  • Files (Nautilus)
  • Font Viewer
  • Help (Yelp)
  • Image Viewer (Eye of GNOME)
  • Logs (gnome-logs, not gnome-system-log)
  • Maps
  • Photos
  • Screenshot
  • Software
  • System Monitor
  • Terminal
  • Text Editor (gedit)
  • Videos (Totem)
  • Weather
  • Web (Epiphany)

Notice that all core apps present generic names (though it’s somewhat debatable if Cheese qualifies as a generic name, I think it sounds better than alternatives like Photo Booth). They all also (more or less) follow the GNOME Human Interface Guidelines.

The list of core apps is not set in stone. For example, if Photos or Documents eventually learn to provide good file previews, we wouldn’t need Image Viewer or Document Viewer anymore. And now that Files has native support for compressed archives (new in 3.22!), we may not need Archive Manager much longer.

Currently, about half of these applications are arbitrarily marked as “system” applications in Software, and are impossible to remove. We’ve received complaints about this and are mostly agreed that it should be possible to remove all but the most critical core applications (e.g. allowing users to remove Software itself would clearly be problematic). Unfortunately this didn’t get fixed in time for GNOME 3.22, so we will need to work on improving this situation for GNOME 3.24.

Incubator

Distributions that want to provide a pure GNOME experience REALLY SHOULD NOT include any of the following apps by default:

  • Dictionary
  • Music
  • Notes (Bijiben)
  • Passwords and Keys (Seahorse)

We think these apps are generally useful and should be in core; they’re just not good enough yet. Please help us improve them.

These are not the only apps that we would like to include in core, but they are the only ones that both (a) actually exist and (b) have actual releases. Take a look at our designs for core apps if you’re interested in working on something new.

Extra apps

Distributions that want to provide a pure GNOME experience REALLY SHOULD NOT include any of the following apps by default:

  • Accerciser
  • Builder
  • dconf Editor
  • Devhelp
  • Empathy
  • Evolution
  • Hex Editor (ghex)
  • gitg
  • Glade
  • Multi Writer
  • Nemiver
  • Network Tools (gnome-nettool)
  • Polari
  • Sound Recorder
  • To Do
  • Tweak Tool
  • Vinagre

Not listed are Shotwell, Rhythmbox, or other applications hosted on git.gnome.org that are not (or are no longer) part of official GNOME releases. These applications REALLY SHOULD NOT be included either.

Note that the inclusion of applications in core versus extra is not a quality judgment: that’s what Incubator is for. Rather, we  classify apps as extra when we do not believe they would be beneficial to the out-of-the-box user experience. For instance, even though Evolution is (in my opinion) the highest-quality desktop mail client that exists today, it can be very difficult to configure, the user interface is large and unintuitive, and most users would probably be better served by webmail. Some applications listed here are special purpose tools that are probably not generally useful to the typical user (like Sound Recorder). Other applications, like Builder, are here because they are developer tools, and developer tools are inherently extremely confusing to nontechnical users. (Update: I originally used Polari instead of Builder as the developer tool example in the previous sentence. It was a bad example.)

Games

What about games? It’s OK to install a couple of the higher-quality GNOME games by default, but none are necessary, and it doesn’t make sense to include too many, since they vary in quality. For instance, Fedora Workstation does not include any games, but Ubuntu installs GNOME Mahjongg, GNOME Mines, and GNOME Sudoku. This is harmless, and it seems like a good list. I might add GNOME Chess, or perhaps GNOME Taquin. I’ve omitted games from the list of extra apps up above, as they’re not my focus here.

Third party applications

It’s OK to include a few third-party, non-GNOME applications by default, but they should be kept to a reasonable minimum. For example Fedora Workstation includes Firefox (instead of Epiphany), Problem Reporting (ABRT), SELinux Troubleshooter, Shotwell (instead of GNOME Photos), Rhythmbox, and LibreOffice Calc, Draw, Impress, and Writer. Note that LibreOffice Base is not included here, because it’s not reasonable to include a database management tool on systems designed for nontechnical users. The LibreOffice start center is also not included, because it’s not an application.

Summing up

Distributions, consider following our recommendations when deciding what should be installed by default. Other distributions should feel encouraged to use these classifications as the basis for downstream package groups. At the very least, distributions should audit their set of default applications and decide for themselves if they are appropriate. A few distributions have some horrendous technical stuff visible in the overview by default; Fedora Workstation shows it does not have to be this way.

Epiphany 3.22 (and a couple new stable releases too!)

It’s that time of year again! A new major release of Epiphany is out now, representing another six months of incremental progress. That’s a fancy way of saying that not too much has changed (so how did this blog post get so long?). It’s not for lack of development effort, though. There’s actually lot of action in git master and on sidebranches right now, most of it thanks to my awesome Google Summer of Code students, Gabriel Ivascu and Iulian Radu. However, I decided that most of the exciting changes we’re working on would be deferred to Epiphany 3.24, to give them more time to mature and to ensure quality. And since this is a blog post about Epiphany 3.22, that means you’ll have to wait until next time if you want details about the return of the traditional address bar, the brand-new user interface for bookmarks, the new support for syncing data between Epiphany browsers on different computers with Firefox Sync, or Prism source code view, all features that are brewing for 3.24. This blog also does not cover the cool new stuff in WebKitGTK+ 2.14, like new support for copy/paste and accelerated compositing in Wayland.

New stuff

So, what’s new in 3.22?

  • A new Paste and Go context menu option in the address bar, implemented by Iulian. It’s so simple, but it’s also the greatest thing ever. Why did nobody implement this earlier?
  • A new Duplicate Tab context menu option on tabs, implemented by Gabriel. It’s not something I use myself, but it seems some folks who use it in other browsers were disappointed it was missing in Epiphany.
  • A new keyboard shortcuts dialog is available in the app menu, implemented by Gabriel.

Gabriel also redesigned all the error pages. My favorite one is the new TLS error page, based on a mockup from Jakub Steiner:

Web app improvements

Pivoting to web apps, Daniel Aleksandersen turned his attention to the algorithm we use to pick a desktop icon for newly-created web apps. It was, to say the least, subpar; in Epiphany 3.20, it normally always fell back to using the website’s 16×16 favicon, which doesn’t look so great in a desktop environment where all app icons are expected to be at least 256×256. Epiphany 3.22 will try to pick better icons when websites make it possible. Read more on Daniel’s blog, which goes into detail on how to pick good web app icons.

Also new is support for system-installed web apps. Previously, Epiphany could only handle web apps installed in home directories, which meant it was impossible to package a web app in an RPM or Debian package. That limitation has now been removed. (Update: I had forgotten that limitation was actually removed for GNOME 3.20, but the web apps only worked when running in GNOME and not in other desktops, so it wasn’t really usable. That’s fixed now in 3.22.) This was needed to support packaging Fedora Developer Portal, but of course it can be used to package up any website. It’s probably only interesting to distributions that ship Epiphany by default, though. (Epiphany is installed by default in Fedora Workstation as it’s needed by GNOME Software to run web apps, it’s just hidden from the shell overview unless you “install” it.) At least one media outlet has amusingly reported this as Epiphany attempting to compete generally with Electron, something I did write in a commit message, but which is only true in the specific case where you need to just show a website with absolutely no changes in the GNOME desktop. So if you were expecting to see Visual Studio running in Epiphany: haha, no.

Shortcut woes

On another note, I’m pleased to announce that we managed to accidentally stomp on both shortcuts for opening the GTK+ inspector this cycle, by mapping Duplicate Tab to Ctrl+Shift+D, and by adding a new Ctrl+Shift+I shortcut to open the WebKit web inspector (in addition to F12). Go team! We caught the problem with Ctrl+Shift+D and removed the shortcut in time for the release, so at least you can still use that to open the GTK+ inspector, but I didn’t notice the issue with the web inspector until it was too late, and Ctrl+Shift+I will no longer work as expected in GTK+ apps. Suggestions welcome for whether we should leave the clashing Ctrl+Shift+I shortcut or get rid of it. I am leaning towards removing it, because we normally match Epiphany behavior with GTK+, and only match other browsers when it doesn’t conflict with GTK+. That’s called desktop integration, and it’s worked well for us so far. But a case can be made for matching other browsers, too.

Stable releases

On top of Epiphany 3.22, I’ve also rolled new stable releases 3.20.4 and 3.18.8. I don’t normally blog about stable releases since they only include bugfixes and are usually boring, so why are these worth mentioning here? Two reasons. First, one of the fixes in these releases is quite significant: I discovered that a few important features were broken when multiple tabs share the same web process behind the scenes (a somewhat unusual condition): the load anyway button on the unacceptable TLS certificate error page, password storage with GNOME keyring, removing pages from the new tab overview, and deleting web applications. It was one subtle bug that was to blame for breaking all of those features in this odd corner case, which finally explains some difficult-to-reproduce complaints we’d been getting, so it’s good to put out that bug of the way. Of course, that’s also fixed in Epiphany 3.22, but new stable releases ensure users don’t need a full distribution upgrade to pick up a simple bugfix.

Additionally, the new stable releases are compatible with WebKitGTK+ 2.14 (to be released later this week). The Epiphany 3.20.4 and 3.18.8 releases will intentionally no longer build with older versions of WebKitGTK+, as new WebKitGTK+ releases are important and all distributions must upgrade. But wait, if WebKitGTK+ is kept API and ABI stable in order to encourage distributions to release updates, then why is the new release incompatible with older versions of Epiphany? Well, in addition to stable API, there’s also an unstable DOM API that changes willy-nilly without any soname bumps; we don’t normally notice when it changes, since it’s autogenerated from web IDL files. Sounds terrible, right? In practice, no application has (to my knowledge) ever been affected by an unstable DOM API break before now, but that has changed with WebKitGTK+ 2.14, and an Epiphany update is required. Most applications don’t have to worry about this, though; the unstable API is totally undocumented and not available unless you #define a macro to make it visible, so applications that use it know to expect breakage. But unannounced ABI changes without soname bumps are obviously a big a problem for distributions, which is why we’re fixing this problem once and for all in WebKitGTK+ 2.16. Look out for a future blog post about that, probably from Carlos Garcia.

elementary OS

Lastly, I’m pleased to note that elementary OS Loki is out now. elementary is kinda (totally) competing with us GNOME folks, but it’s cool too, and the default browser has changed from Midori to Epiphany in this release due to unfixed security problems with Midori. They’ve shipped Epiphany 3.18.5, so if there are any elementary fans in the audience, it’s worth asking them to upgrade to 3.18.8. elementary does have some downstream patches to improve desktop integration with their OS — notably, they’ve jumped ahead of us in bringing back the traditional address bar — but desktop integration is kinda the whole point of Epiphany, so I can’t complain. Check it out! (But be sure to complain if they are not releasing WebKit security updates when advised to do so.)

A WebKit Update for Ubuntu

I’m pleased to learn that Ubuntu has just updated WebKitGTK+ from 2.10.9 to 2.12.5 in Ubuntu 16.04. To my knowledge, this is the first time Ubuntu has released a major WebKit update. It includes fixes for 16 security vulnerabilities detailed in WSA-2016-0004 and WSA-2016-0005.

This is really great. Of course, it would have been better if it didn’t take three and a half months to respond to WSA-2016-0004, and the week before WebKitGTK+ 2.12 becomes obsolete was not the greatest timing, but late security updates are much better than no security updates. It remains to be seen if Ubuntu will keep up with WebKit updates in the future, but I think I can tentatively stop complaining about Ubuntu for now. Debian is looking increasingly isolated in not offering WebKit security updates to its users.

Thanks, Ubuntu!