At this year’s GUADEC, I made a proposal: that the GNOME project should migrate its documentation from wiki.gnome.org to other locations, and then disable its wiki. This blog post describes the proposal in more detail, in order to receive feedback from the wider community.
Background
For many years, the GNOME wiki (a MoinMoin instance) has been the main source of internal project documentation. It contains key project information, including guidance for newcomers, the release schedule, pages for the various teams, and much more.
Today, much of the content on the wiki is out of date, and a large portion of it isn’t actively updated. Many activities have moved from the wiki to other platforms (particularly Gitlab), leaving the related wiki content to rot. There have also been problems with spam, which have meant that edit rights have to be manually granted.
This situation inevitably causes issues, particularly for new contributors who are trying to learn how to get involved.
The proposal
The proposal is for us to replace the wiki with a combination of alternative platforms. The main alternatives would be:
- gnome.org, for small amounts of important high-level information
- developer.gnome.org, for technical documentation
- apps.gnome.org, for application home pages
- In-tree documentation, for individual project docs
- Gitlab issues in special-purpose projects, for more transient documentation
- A new project handbook site, for general project documentation (contribution guide, code of conduct, governance, release process, and so on)
The new handbook site would be hosted using Gitlab pages and generated with Sphinx. We already do this for developer.gnome.org and the HIG, so the new site would not be unique in GNOME.
To see what the new handbook site might look like, you can visit a prototype that I’ve quickly created, by copying content from the wiki to a new Sphinx project.
About Sphinx
We started using Sphinx for docs in GNOME after evaluating a variety of alternative solutions. It’s not the newest or trendiest docs tool, but it has some key features which made it a good choice for us:
- It’s a drop in tool which is both mature and well-maintained
- It generates static HTML, which makes deployment and hosting easy with CI and Gitlab pages
- Docs content is written using a light weight markup language
- We can easily restyle the docs sites to match GNOME branding
Our experience so far has been positive: Sphnix-based sites are easy to set up, deploy, host and edit.
Sphinx’s primary markup language is restructured text, which leads to the obvious question: can’t we use Markdown? Markdown is supported, so we could decide to use that. However, with the HIG, we found that restructured text gave a better experience overall ¹, and was light weight enough to not be burdensome ².
In terms of editing experience, you can make quick changes using Gitlab’s web-based editing tools or, for a richer experience, VS Code will give you live previews and error checking.
Transition process
My suggestion for retiring the wiki would be to have a managed transition, which would look something like this:
- Create a migration guide for how and where to move existing wiki content
- Announce the transition period (including an end date)
- Transition! I’d finish the handbook, and other teams and individuals would be responsible for migrating their own content
- Finish the transition: archive the wiki, and update any links to point to the new docs locations
Archiving the wiki could take various forms depending on what is most practical. We could put it in read-only mode and add a banner, like we did with Bugzilla, or we could create a static archive and take it offline completely. I’ll leave that question to our sysadmins.
Pros and cons
I’m definitely not suggesting that this proposal is perfect. Nor am I suggesting that there aren’t other alternatives to consider (it would be great to hear any other ideas)! So, in the interests of transparency, here are the main pros and cons of the proposal.
Advantages of the new handbook site:
- It’s a nice website with good styling, clear navigation, and search
- Having a clear scope and purpose as the “GNOME Project Handbook” is very helpful for both documentation users and authors
- Having the content be maintained would (hopefully) result in higher overall quality
- The site would use a contribution workflow that is consistent with other GNOME Gitlab projects
And the disadvantages of the general proposal:
- There is more overhead required to make changes to docs stored in Git, compared to a wiki
- Making changes via Git requires more technical knowledge (though we can potentially minimize this with the Gitlab web IDE and good documentation)
- The new handbook site would run into problems if it were to become unmaintained
- Docs that aren’t hosted on the handbook site could be fragmented and hard to find
Feedback wanted!
This proposal is just that: a proposal. We can and should consider alternatives. It also needs to be elaborated to cover all the requirements GNOME has today. If you have any concerns, suggestions, or uncertainties, please let me know.
Thanks!
¹ With restructured text, you get more powerful linking features, as well as internal link validation.
² All you really need to know is the format for an internal and external link, and you’re good to go.
Ok, now I have much clearer understanding about your proposal than right after your presentation.
I feel this is moving cost of keeping documentation updated from maintaining wiki system (most likely quite a lot of work for operations team) to developers side a bit. However if it is given clear instructions where to add things and how, it might even bring better results than now?
Because showing it all into wiki is not right answer, but we never had suitable alternatives. This feels like good start of one.
Also statistically generated pages have nice feature of being easily shipped and moved around without huge problems.
“Gitlab issues in special-purpose projects, for more transient documentation”
It’s not OK to put documentation in issue reports. Better to use GitLab wikis instead. Every GitLab project has a wiki; might as well use it?
One more thing, regarding the Sphinx handbook: not many people know how to use rst, whereas almost everybody who would be editing the handbook already knows how to use Markdown. I’m really skeptical that any advantages of rst somehow outweigh the ubiquity of Markdown.
> It’s not OK to put documentation in issue reports
I think it depends on what you mean by “documentation”. In the past, the design team used to have a wiki page for the design of every app and feature. The problem was that we could never keep them all up to date.
Nowadays we instead use issues to “document” our designs, but only for the lifetime of a design project. Once it is complete, the issue is closed.
You could imagine a similar approach to hackfests, or the schedule of each release – things that are more ephemeral.
This is just an idea of course. We don’t have to use issues in this way and there are alternatives.
> Every GitLab project has a wiki; might as well use it?
I’m nervous about going down this road – I don’t want us to end up with the same problems we have now, just with a different platform. We could end up with a lot of unmaintained content, particularly if projects feel that they need to have a wiki for consistency.
Also, my experience with Gitlab wikis have been a bit limited when I’ve tried them in the past (for one, I remember there not being a visible change history).
I think there is merit to considering the importing the wiki into one or more GitLab Wikis as both a transition and archival strategy; it means the MoinMoin service can be retired sooner and the Gitlab marked as archived/etc without the content disappearing offline. There appear to be tools that can do it for us – https://github.com/nigelm/moin2gitwiki
It’s also then much easier (as the structure is reflected in MD files in a Git repo) to see everything we have and then lift subtrees into other (living) Wiki repos if that’s the right thing to do.
I think there are probably some corners of the Wiki; for example the Board’s internal documentation that lives on the Wiki, and some public info about the Foundation/Board such as committees, officers, public minutes, etc, which would suit a Wiki presentation as they are either mutable or forever growing info which seem out of place in a “handbook” format.
I find rst really annoying to use. Most likely because I’m not familiar with it.
If the goal is to encourage contributions and maintenance then Markdown is probably the best choice? It’s used in Hedgedoc, development, gitlab wikis, GNOME apps, Matrix, Discourse, …
None of those websites share the same superset of Markdown, though…
Death to the wiki!
I think one of the reasons why the wiki is rotting with old information is because the editing permissions are restricted to a small group of people, as you said in your post, to reduce spam. Maybe we should try using some wiki software with better spam protection, like MediaWiki combined with relaxed permissions to allow more people to contribute. If I could edit and contribute to the wiki by updating information I would, but I can’t, and moving the docs to git seams like it would just make it harder to contribute, especially when the docs aren’t tied to code. With things like GitLab wikis the permissions are restricted to people with write access to the repo, which means I have to do things like message a diff to the maintainer to edit the wiki and if the maintainer goes away then there might be people that still want to update the information but can’t because of permissions. The entire point of a wiki is that because it is edible by a wide group of people mistakes and old info are more likely to be found and fixed, and I feel like the reasons the old wiki didn’t work because it wasn’t a wiki by this definition because people like me couldn’t change e.g. spelling errors that they spotted while just browsing.
Right, starting a new MediaWiki instance one is probably the main alternative to the plan I suggested. It’s worth considering.
I also agree that one of the main strengths of wikis is the low barrier to entry. The current hackfests pages are a one of the few examples where the wiki continues to perform well, and ease of access is a key part of that.
I would also say that, in the past, the wiki has worked fairly well for team pages, and I’m yet to identify a good alternative to the wiki for those. (Though, how many teams still want to maintain their own pages is unclear.)
However, for most of the other pages, my impression is that the key issues are not access, but rather that:
1. Teams have already migrated their activities to Gitlab
2. The community no longer prefers the wiki as a format
3. There’s a lack of ownership over project level docs
A comment on point 3: pages like communication channels and how to contribute have never attracted much attention and have never been in a great state during my entire time contributing to GNOME. I think the main issue there is the wiki itself: either there’s no clear owner for the page, which means that no one feels empowered to maintain it, and/or there’s too much flux from random edits, which impacts quality.
I’m not a GNOME contributor, so I don’t have a vested interest in what you pick, so I’m sharing some general thoughts from my experience.
rst vs Markdown
rst was my least favorite human-friendly markup language until I started working with AsciiDoc. It’s fine, but Markdown feels more intuitive (but maybe just because it’s more ubiquitous. On the other hand, there’s one rst. There are a billion Markdowns. Ultimately, most people won’t need to be particularly fluent in either one, so don’t spend too much time thinking about this.
Wiki vs Git-based docs
Wikis have a bad reputation for a good reason. But “the information is out of date” is a process problem, not a technical problem. Docs need to be actively curated (“gardening” is a term I like here), regardless of what platform they’re on. A wiki is more accessible, which helps get more people involved, but the workflow is worse. It’s not that “a git-based workflow is more familiar to developers”, it’s that the editorial workflow is bad on a wiki.
In a wiki, either people can edit the page or they can’t. This leads to one of two opposite outcomes: 1. they make big edits without talking to anyone which makes relying on the wiki for authoritative info risky. 2. people are afraid to make minor edits because they don’t want to break anything.
git-based docs allow anyone* to submit a proposal by way of a pull request. Then the trusted contributors can work with them to refine it if needed. It also allows trusted contributors to make a proposal for broader comment.
I agree. Personally I tend also to hesitate when doing changes in a wiki system, as those are immediately live and I fear to provide stuff, which is inaccurate or not wanted.
But yes, having people reviewing the changes is more effort, but could lead to more contributions (and of course to a higher quality).
The technical entry barrier is higher, but the needed skills are the same as for code contributions in general.
An other thought, maybe it makes sense to experiment with translation services (like deepl.com, which provides some pretty good results, but is not for free), to auto translate content to other languages to lower the barrier for consumer who prefer different languages.
Having to contribute over Gitlab will certainly decrease your amount of new contributors no matter how good the documentation is but I would understand that change. Getting rid of Markdown in favor of something no one knows seems like a plain awful idea to me tho!
We’re talking about reStructuredText, here, not a custom format that nobody knows. RST is commonly used in multiple contexts and communities—not the least, the whole Python ecosystem; it’s well-defined and well-specified, unlike the dog’s breakfast that is Markdown; and it can be used to reliably generate multiple formats, including HTML. The only upside to Markdown is that, if you choose the right Markdown format and parser and extensions, then you get something that may or may not be similar to what people that have used Markdown before can recognise.