With GTK+ 4 in development, it is a good time to reflect about some best-practices to handle API breaks in a library, and providing a smooth transition for the developers who will need to port their code.
But this is not just about one library breaking its API. It’s about a set of related libraries all breaking their APIs at the same time. Like what will happen with (at least a subset of) the GNOME libraries in addition to GTK+.
Smooth transition, you say?
What am I implying by “smooth transition”, exactly? If you know the principles behind code refactoring, the goal should be obvious: doing small changes in the code, one step at a time, and – more importantly – being able to compile and test the code after each step. Not in one huge commit or a branch with a lot of un-testable commits.
So, how to achieve that?
Reducing API breaks to the minimum
When developing a non-trivial feature in a library, designing a good API is a hard problem. So often, once an API is released and marked as stable, we see some possible improvements several years later. So what is usually done is to add a new API (e.g. a new function), and deprecating an old one. For a new major version of the library, all the deprecated APIs are removed, to simplify the code. So far so good.
Note that the deprecated APIs still need to work as advertised. In a lot of cases, we can just leave the code as-is. But in some other cases, the deprecated API needs to be re-implemented in terms of the new API, usually for a stateful API where the state is stored only wrt the new API.
And this is one case where library developers may be tempted to introduce the new API only in a new major version of the library, removing at the same time the old API to avoid the need to adapt the old API implementation. But please, if possible, don’t do that! Because an application would be forced to migrate to the new API at the same time as dealing with other API breaks, which we want to avoid.
So, ideally, a new major version of a library should only remove the deprecated APIs, not doing other API breaks. Or, at least, reducing to the minimum the list of the other, “real” API breaks.
Let’s look at another example: what if you want to change the signature of a function? For example adding or removing a parameter. This is an API break, right? So you might be tempted to defer that API break for the next major version. But there is another solution! Just add a new function, with a different name, and deprecate the first one. Coming up with a good name for the new function can be hard, but it should just be seen as the function “version 2”. So why not just add a “2” at the end of the function name? Like some Linux system calls: umount() -> umount2() or renameat() -> renameat2(), etc. I admit such names are a little ugly, but a developer can port a piece of code to the new function with one (or several) small, testable commit(s). The new major version of the library can rename the v2 function to the original name, since the function with the original name was deprecated and thus removed. It’s a small API break, but trivial to handle, it’s just renaming a function (a git grep or the compiler is your friend).
GTK+ timing and relation to other GNOME libraries
GTK+ 3.22 as the latest GTK+ 3 version came up a little as a surprise. It was announced quite late during the GTK+/GNOME 3.20 -> 3.22 development cycle. Other GTK+-based libraries will need to be ported to GTK+ 4, which will require a fair amount of code changes, and might force API breaks in those higher-level libraries. So what will happen is that for those libraries, a new major version will also be released, removing their own deprecated APIs, and doing other API breaks.
If you are a maintainer of one of those higher-level libraries, you might have a list of things you want to improve in the API, some corners that you find a little ugly but you never took the time to add a better API. So you think, “now is a good time” since you’ll release a new major version.
Let’s say you released libfoo 3.22 in September. If you follow the new GTK+ numbering scheme, you’ll release libfoo 3.90 in March (if everything goes well). But remember, porting an application to libfoo 3.90/4.0 should be as smooth as possible. So instead of introducing the new APIs directly in libfoo 3.90 (and removing the old, ugly APIs at the same time), you should release one more version based on GTK+ 3: libfoo 3.24. To reduce the API delta between libfoo-3 and libfoo-4.
So the unusual thing about this development cycle is that, for some libraries, there will be two new versions in March (excluding the micro/patch versions). Or, alternatively, one new version released in the middle of the development cycle. That’s what will be done for GtkSourceView, at least (the first option), and I encourage other library developers to do the same if they are in the same situation (wanting to get rid of APIs which were not yet marked as deprecated in GNOME 3.22).
Porting, one library at a time
If each library maintainer has reduced to the minimum the real API breaks, this eases greatly the work to port an application (or higher-level library).
But in the case where (1) multiple libraries all break their APIs at the same time, and (2) they are all based on the same main library (in our case GTK+), and (3) the new major version of those other libraries all depend on the new major version of the main library (in our case, libfoo 3.90/4.0 can be used only with GTK+ 3.90/4.0, not with GTK+ 3.22). Then… it’s again the mess to port an application – except with the following good practice that I will just describe!
The problem is easy but must be done in a well-defined order. So imagine that libfoo 3.24 is ready to be released (you can either release it directly, or create a branch and wait March to do the release, to follow the GNOME release schedule). What are the next steps?
- Do not port libfoo to GTK+ 3.89/3.90 directly, stay at GTK+ 3.22.
- Bump the major version of libfoo, making it parallel-installable with previous major versions.
- Remove the deprecated APIs and then release libfoo 3.89.1 (development version). With a git tag and a tarball.
- Do the (hopefully few) other API breaks and then release libfoo 3.89.2. If there are many API breaks, more than one release can be done for this step.
- Port to GTK+ 3.89/3.90 for the subsequent releases (which may force other API breaks in libfoo).
The same for libbar.
Then, to port an application:
- Make sure that the application doesn’t use any deprecated APIs (look at compilation warnings).
- Test against libfoo 3.89.1.
- Port to libfoo 3.89.2.
- Test against libbar 3.89.1.
- Port to libbar 3.89.2.
- Port to GTK+ 3.89/3.90/…/4.0.
This results in smaller and testable commits. You can compile the code, run the unit tests, run other small interactive/GUI tests, and run the final executable. All of that, in finer-grained steps. It is not hard to do, provided that each library maintainer has followed the above steps in the good order, with the git tags and tarballs so that application developers can compile the intermediate versions. Alongside a comprehensive (and comprehensible) porting guide, of course.
And you, what is your list of library development best-practices when it comes to API breaks?