Posts Tagged ‘documentation’

How NOT to write user-friendly documentation

Saturday, March 15th, 2008

Warning: This is essentially a long rant about the built-in help of git, but with specific problems pointed out and some constructive suggestions buried at the very end.

When I was starting on my PhD in mathematics, there was a time I remember where the professor in one of my classes referred to a problem on our homework assignment as “a calculus problem.” Later during the week, the six or so students in the small class were talking about the homework and one of the brighter students (i.e. not me) trepidly asked, “Did he really call that a calculus problem?” It turns out we were all stumped by the problem, and we were surprised–and scared–that the professor utterly trivialized it. We did all eventually solve the problem, and did ultimately agree that it was a generalization of ideas from calculus, but not until after getting a bit of a scare that we were in over our heads in graduate school.

In a similar way, Carl and others have argued that they just don’t see the big user-visible differences in the models offered by git and other VCSes. Many a git user seems to have had trouble understanding why other systems would be marketed as being easier than git. I am a lot slower than most of them, but I eventually figured it out and I do think that git could be just as easy if tweaked appropriately; however, it sure does seem to place every obstacle possible in the way of users discovering that.

This blog post is dedicated to what I believe is the single biggest obstacle in the way of users learning git: its built-in documentation (which doubles as its manpages, or vice-versa). Let me first point out that the git manpages are stellar from the point of view of comprehensiveness, or from the point of view of those writing their own UI for git. However, in this post I will judge these documents based on their friendliness to users who are new to git (which may not have been one of their design goals for those pages, so such judgement may come across as unfair).

Do not layer concepts — make it all or nothing

Imagine taking a computer user who is bright but unfamiliar with unix or common editors associated with unix, and giving them vi (or vim) to run. Also, imagine making them unable to use vi until they understand every feature and capability of it. Without a way to incrementally improve their productivity (better manuals, or a friend to explain things, etc.), most users will either give up or take an extremely long time to become productive. While this analogy is a bit of a stretch, for the most part this is precisely the way the git manpages are structured[1].

I’ll take push and pull as an example. If you take a look at git’s help for push or pull (or fetch), you’ll note that “refspecs” are featured prominently. Embedded in the explanation of refspecs, you’ll find concepts, terms, and other details such as rebasing, octopus, configuration file syntax and effects of such configuration options, fast-forwarding, nonlinear history, repository storage format, implementation details about git’s repository directory layout, local and remote repositories, remote tracking branches, and merge strategies–all inextricably intertwined (or so you’d think on your first 12 readings). refspecs pack a lot of information into a short amount of space, making them very convenient to the expert; but for new users this just places a mountain in front of them getting started. Since push and pull are the methods of collaborating with other users and refspecs are necessary to their functioning, new users often feel stuck.

In a similar way, when I was starting I thought that I would have to understand the index, reflogs, tree-ish’es, internal repository storage format, and a few other arcane things before I’d ever be able to do anything productive with git diff.

Assume that all details are equally useful

This section could likely be combined with the previous one, but I thought it was worth a special mention. It typically comes across as “this document is overloaded with detail”, though the existence of details is not necessarily the problem (you can have a comprehensive document and have it be understandable; take a look at the cvs, svn, or hg books at red-bean.com). The problem with documentation that assumes all details are equally useful (typically a subconscious assumption) is that it makes it extremely hard for the user to get started. git’s commands have boatloads of flags, and with few exceptions there is not much guidance at all about which are more important for just getting started. Users have to sift through it all and try to figure it out themselves.

The manpages also often specify details before concepts, erring as far as possible on the side of precision. For example, take the opening sentence of the description of the git diff command: “Show changes between two trees, a tree and the working tree, a tree and the index file, or the index file and the working tree.” After reading this, the user can only respond with comments like: “What’s the index file?” (or maybe even “is the ‘index file’ something slightly different than the ‘index’ I read about elsewhere, perhaps just a subset of it or how it behaves?”), “The working tree isn’t a tree?!?”, or, perhaps most likely, “Huh, what?” The git diff help tries, starting from the beginning, to introduce as many concepts as possible (without explanations) and just befuddles the user. For another example, take a look at the description of git checkout. For someone familiar with git, they won’t see anything wrong. In fact, I don’t see anything hard there anymore. However, do you know how many dozen times I tried to parse those two short paragraphs in attempting to learn git? I don’t — I long since lost count.

Require understanding of deeply nested commands

The documentation in git often defers explanation by referring to other (low-level) commands, in a way that makes users feel they have to understand all the low-level commands too.

Here’s an example. From the git push documentation:

The <src> side can be an arbitrary “SHA1 expression” that can be used as an argument to git-cat-file -t.

From the git-cat-file documentation, explaining the -t option:

Instead of the content, show the object type identified by <object>.

Finding the definition of <object> on the git-cat-file page:

The name of the object to show. For a more complete list of ways to spell object names, see “SPECIFYING REVISIONS” section in git-rev-parse

(More complete?!? There was no explanation here at all!) And for kicks, the very beginning of the SPECIFYING REVISIONS section of git-rev-parse:

A revision parameter typically, but not necessarily, names a commit object. They use what is called an extended SHA1 syntax. Here are various ways to spell object names. The ones listed near the end of this list are to name trees and blobs contained in a commit.

If the user gets to this point they are like to ask whether the object from the git-cat-file page refers to a commit object, a different kind of object, or a more general case that includes both. Also, they may ask, if it’s just a commit object, then part of the ensuing explanation is going to be for something else other than what we are interested in…but are we going to be able to differentiate which parts of the explanation that is?

Here are some other examples, all of them taken from the the first paragraph or two of the descriptions of each command. From git-checkout: “It updates the named paths in the working tree from the index file (i.e. it runs git-checkout-index -f -u)” (one has to love following up an unintelligible (to new users) explanation by referring to a lower-level command that assumes the user knows even more). From git-bisect: “This command uses git-rev-list –bisect option to drive the binary search…“. From git-log: “The command takes options applicable to the git-rev-list command to control what is shown and how, and options applicable to the git-diff-tree commands to control how the changes each commit introduce[d] are shown.

Prefer implementational details to user-level concepts

Just one example I know of immediately…open up the git-checkout manpage and you’ll see a section titled “Detached Head”; it’s completely meaningless to new users. Trying to describe the concepts (“working with no active branch”, “working on an unnamed branch”?) would be much better. I think they’ve tried to stomp out issues in this category, so there are not as many as there used to be.

Make your terminology inconsistent

I’ll jump straight to a quick example from git: You put things in the “index” with “add” which is known as “staging”, you remove them with “reset” (or “rm –cached” if it’s the initial commit), you use flags like “–cached” to refer to index stuff in some places, use “HEAD” to avoid it (specifically in diff), and sometimes the flags related to it are “soft” vs “mixed” (which also need to be juggled with “hard” in git-reset). While users eventually learn all the synonyms and related terminology, it really slows learning down.

Use inconsistent conventions

The git manpages do not use consistent conventions. For example, taking a look at the synopsis lines you’ll see “<filepattern>” on git-add, “file” on git-annotate, and “-pNUM” on git-apply; why are non-literal items denoted with angle brackets in one case, uppercase in another, and no special markings in a third case? On the git-push manpage alone, you’ll see “–receive-pack=<git-receive-pack>” and also “–repo=all”; angle brackets in one case, and plain text in another? On the git-diff manpage you’ll see “<commit>{0,2}” and also “<path>…”, while on git-bundle you’ll see “refname…”, and on git-checkout-index you’ll see “[<file>]*”; so denoting more than one item is done with a variety of regexes, an ellipsis combined with angle brackets, or just a plain ellipsis.

The more ways you have of referring to things, the more likely users are to confuse them. One that bit me particularly hard was the “<commit>{0,2}” in the git-diff synopsis. Now, I use regular expressions daily and my usage of them spans emacs, grep, sed, perl, python, and probably other places I’m forgetting…but it still did not occur to me that this syntax in this context happened to be a regex. It looked somewhat similar to reflog notation in git, and I launched into all sorts of reading up on reflogs (and the documentation it depended on) trying to understand this detail. I also read the online tutorial, parts of the git user manual, blog sites, and even mailing list archives. I gave up and moved on, but never felt comfortable that I understood things…until months later when it dawned on me that this was a simple regex.

(And no, I don’t care that my post whining about inconsitent conventions is itself riddled with inconsistent conventions.)

What can be done to fix this?

Well…some people have fled to hg and bzr to avoid these problems in git. As far as built-in documentation, both of them have done well and have very nice documentation for new users; bzr’s is particularly well polished. Adopting one of those systems is a reasonable choice which I can’t fault people for making.

Now, I dislike reaming a project without mentioning something good or at least providing constructive feedback. In addition to spelling out some specific problems throughout this post, I came up with some constructive ideas for improvement: Easy GIT, a brainstorming session about making git more user-friendly, in the form of a usable demonstration. It also doubles as a transition tool, helping users switch from other systems (particularly svn). I don’t know if anyone wants to adopt any of my ideas or documentation, but it’s out there for people to evaluate (even if not yet complete).

I will also note that Govind Salinas had a similar idea and created pyrite (and did so before I started eg, though I didn’t learn of it until after I was using eg daily). So it seems that I am not the only one thinking along these lines…


[1] The online tutorial and user manual of git are different, but sadly users don’t want to read big long tutorials when they are already familiar with another “similar” system; rather they just use ‘git help’ and get the git manpages…and end up more confused than they started.