Slightly more correct title might be “Wikimedia to migrate its software development product/project code review issue tracking management planning tools to Phabricator”. Or something like that.
The problem
The Wikimedia technical community has used plenty of different tools for tracking bugs / product management / project management / todo lists. Bugzilla, RT, Gerrit, Mingle, Trello, Scrumbugs, Google Docs, to mention most of them.
From my personal point of view, Wikimedia Foundation is pretty bottom-up: Each team can experiment and use those tools they prefer and suit them. That also means teams might have moved from Bugzilla to Trello to Mingle to Google Docs while other teams prefer(ed) other tools. Bugzilla is our public issue tracker but misses a lot of functionality when it comes to agile development workflows, design review work, or activity feeds.
We also have some connectivity between these tools. Bingle/Bugello to sync some parts between Bugzilla and Mingle/Trello, or its-bugzilla (previously “bugzilla-hooks”) to have Gerrit post comments in Bugzilla tickets about related patches (if the bug number was correctly refered in the commit message). But things are brittle – for example, just this Friday the Gerrit→Bugzilla notifications broke.
All in all, the multitude of tools and channels is not helpful for cross-team collaboration, keeping track of what’s happening, and transparency of discussions and decisions in general as things are discussed in several places.
The idea
In late 2013, the idea was to start a discussion about a possible agreement on a recommendation for a smaller set of tools that teams could agree upon. My colleague Guillaume and I had the pleasure to facilitate the discussion and to ensure it doesn’t remain an idea only. References were a previous evaluation attempt in 2009/2010 and the Gerrit evaluation in 2012.
The steps
The first step was asking interested teams and individuals to describe their needs and workflows on a wiki discussion page.
Its content was then consolidated by Guillaume and cleaned up a little bit more by me. (I felt reminded of GNOME’s decision process to migrate from Subversion to Git but that was a survey among GNOME foundation members and hence a very different approach.)
After having those (sometimes contractive) needs and workflows collected, we tried to decrease the items in the list of candidate tools to consider, plus investigate and encourage discussing them on the related discussion page. For candidate tools not having an online test instance offered on the project’s homepage we wondered whether to set up test instances on Wikimedia Labs to make testing easier, but we left it to anybody strongly favoring a tool to set up that tool. Wikimedia Deutschland already had Scrumbugs instance in production (Scrum on top of Bugzilla) we could point to, and for Phabricator somebody had set up a test instance in Wikimedia Labs already.
To gather the broader community opinion and broader support for investigating more potential (wo)manpower, we started to prepare a Request for comments (RFC). While we listed several options at the beginning (Keep the status quo; status quo+Fulcrum; status quo+Scrumbugs; move completely to Phabricator; move partially to Phabricator; move to GitLab) the feedback quickly turned this into one question to ask the community: Move to Phabricator?
We ran this RFC for three weeks until May 6th and announced it widely on mailing lists and via banners on top of Bugzilla and mediawiki.org.
Parallel to running the RFC, we were working on sorting out the blockers for a potential migration from Bugzilla and documenting things. My colleague Quim created a comparison page between Phabricator and Bugzilla.
The decision
The result of the RFC is that there seems to be general support for moving from our infrastructure tools to Phabricator. This won’t all happen at the same time though – we will start investigating replacing Bugzilla, RT, Trello, Mingle.
For the code review functionality (currently done via Gerrit), more work in Phabricator is needed to fit the needs defined by our community, for example when it comes to Continuous Integration. We do not plan to switch off Gerrit on the day we start using Phabricator in production and we got more items to sort out (see the list of code review related items).
For managing the project to move to Phabricator we use the Phabricator test instance itself (dogfooding for the win), by tracking missing features compared to our existing tools, and tasks that need to be solved for the migration. Also, we asked users of existing tools what they would specifically miss in Phabricator by creating (sub)tasks in our Phabricator test instance.
We have not created a Phabricator production instance yet to which we would potentially migrate to, because in the past Wikimedia ended up with a lot of tools by not enforcing migration.
Spreading the word and resource allocation
For the last months we also ran IRC office hours every two or three weeks in order to discuss and answer questions related to the project.
Last weekend the Wikimedia Hackathon took place in Zürich. There were several Phabricator related sessions (videos available for the sake of transparency; the first two videos are more like discussions though):
- A general quick introduction to Phabricator by my colleague Shahyar
- A discussion among ~10 people of what needs to be sorted out and done for Day 1 of Phabricator in production (being tracked in this Phabricator project board), plus related stakeholders and who’s going to work on what.
- A session by Shahyar specifically explaining the code review concept and tool in Phabricator (remember: we do not plan to switch from Gerrit to Phabricator for code review on the first day)
Current status and next steps
- General and central information page about Wikimedia’s Phabricator instance.
- Planning board for Wikimedia Phabricator Day 1 in Production (You see several columns here: ‘Need discussion’ needs more thoughts first until we know how to solve those tasks; ‘Ready to Go’ are tasks for which the way forward is clearly defined and somebody could start working on them; ‘Waiting for upstream’ are tickets that we have reported to upstream; ‘Doing’ is what is currently actively being worked on.)
- The team working on Wikimedia Phabricator and the stakeholders are defined. We have tasks such as investigating migrating Bugzilla and RT data to Phabricator, plus implementing missing functionality like restricting access to certain projects by default (like for Bugzilla’s “Security” product or RT’s procurement queue) via namespaces. The WMF Technical Operations team will set up low-level infrastructure and puppetize our Phabricator instance. We have stakeholders defined to keep in contact with the team of Product Managers and the Platform and Release Engineering/QA team, we have more interested stakeholders which will provide input to help making decisions and driving things forward, and I am happy to also see volunteers being interested to get involved by diving into the code.
- To track the tasks that Wikimedia is interested in in Phabricator’s upstream task tracker, we have a “Wikimedia” project in upstream Phabricator and the corresponding planning board.
The usual disclaimer: Plans might be subject to change and there is intentionally no timeline yet.
How you can help
Check out the Get involved section on the central project page and the planning board for Wikimedia Phabricator Day 1 in Production if there are tasks that interest you!
Thanks
I would like to thank everybody in the community who has provided input, help and support. Upstream Phabricator developers have been extremely responsive and interested in discussing our needs and fixing issues – it’s a great pleasure to work with them.
Furthermore, getting to this point would have been impossible without my wonderful colleagues in the Wikimedia Engineering Community Team who have helped so much with communication, prioritization, planning, support.
