[[!tag obnam survey]]

In January and February of 2016 I ran an Obnam user survey. I'm not a statistician, but here is my analysis of the results.

Executive summary: Obnam is slow, buggy, and the name is bad. But they'd like to buy stickers and t-shirts.

Method

I wrote up a long list of questions about things I felt were of interest to me. I used Google Forms to collect responses, and exported them as a CSV file, and analysed based on that.

I used Google Forms, even though it is not free software, as it was the easiest service I got to work that also seemed it'd be nice for people to use. I could have run the survey using Ikiwiki, but it wouldn't have been nearly as nice. I could have found and hosted some free software for this, but that would have been much more work.

Most questions had free form text responses, and this was both good and bad. It was good, because many of the responses included things I could never have expected. It was bad, because it took me a lot more time and effort to process those. I think next time I'll keep the number of free text responses down.

For some of the questions, I hand-processed the responses to a more or less systematic form, in order to count things with a bit of code. For others, I did not, and show the full list of responses (I'm lazy, we don't need a survey to determine that).

The responses

See http://code.liw.fi/obnam/survey-2016.html for the responses, after hand-processing.

For the questions for which it makes sense, a script has tabulated the various responses and calculated percentages. I haven't produced graphs, as I don't know how to do that easily. (Maybe next time I'll enlist the help of statisticians.)

Conclusions

  • There were 263 responses in total. I have no idea of knowing if the total number of Obnam users is about that, but the number correlates fairly well with the Debian popcon numbers, so I'm assuming Obnam has on the order of a few hundred users total.

    A larger number might be more impressive, but it'd also mean that I would be responsible for much more data loss if I make a horrible mistake. That said, it is probably time to start spending some effort on growing the developer base of Obnam.

  • People seem to hear about Obnam primarily from my blog posts, or by searching the web for backup software. Also, from the Arch Linux or Gentoo wikis, or Joey Hess.

  • People use Obnam mostly for personal machines, but also at work.

  • Those who have tried Obnam, but don't use it, rejected it primarily for speed or because it's unstable or buggy. I hope that the bad bugs have mostly been fixed, and I'm working on improving the speed.

  • People seem to use either the latest version, or the version included in the release of their operating system (e.g., Debian jessie). Other versions are relatively rare.

  • Most people started using Obnam in the past two years.

  • People use Obnam on a variety of Linux based operating systems, but also others. Obnam users are especially skewed towards Debian and Ubuntu, which is not surprising, as I'm involved in Debian and have been publicising it there, and provide package for Debian myself.

  • About half the people have at least hundreds of thousands of files, containing hundreds of gigabytes of data. All extremes (very few or very many files, very little or very much data) are represented, though. A couple of people have at least a hundred million files, or at least ten terabytes of data.

  • Most people don't have a backup strategy, or at least not a documented one, and if they do, it's not regularly tested.

    This isn't a good thing.

  • Most people had backed up within the past week as of the time of filling in the survey. This hopefully indicates that they back up frequently. Only one respondent said they'd never backed up.

    Rather more people hadn't tested their backups, however, with about a fifth of the people having never tested their backup. This is also not good.

  • Most people only back up one machine to each repository, or at most a few. A total of 17 respondents reported that they don't have a backup, and do not fear clowns.

  • About half the people back up to a local drive, and nearly two thirds to an SFTP server.

    People ask for more remote storage options, such as support for services like Amazon S3.

  • The things people like most about Obnam are on its list of core features: de-duplication, encryption, and ease of use / simplicity. FUSE is also well-liked, as are snapshot backups.

  • I didn't tabulate the reasons why people don't like Obnam, but performance and stability seem to be the most common reasons. My favourite response to this question is "the name obnam, does not sounds like a backup program".

  • Speed is also the pet bug people seem to have.

  • People seem to generally find Obnam documentation adequate. There's room for improvement, of course.

  • Nearly everyone finds it easy to get help if they have a problem with Obnam, but almost no-one uses the Obnam support mailing list or IRC channel.

  • Some people read the NEWS file, others do not. Few have sent patches, but some would like to. There's a bunch of suggestions for new features.

    None of this is surprising to me, except perhaps that so many Obnam users actually do read the NEWS file, as it's been my experience in other projects that that's rare.

  • About half the people have heard of the green albatross. It's the name of the new way in which Obnam will be storing data on disk, which is a big factor in how fast or slow Obnam is. When the green albatross soars, Obnam will fly faster.

  • People use other backup software as well, which is sensible: no point in having all one's eggs in one basket. The top choices are rsync, duplicity, attic, and rsnapshot, but the list seems to mention most free backup software.

  • There's some interest in helping Obnam development, either by direct contributions, donations, paying for support or development, or by buying merchandise. Nearly no-one wants a printed version of the manual, but stickers and t-shirts might sell well enough.

    A lot of people don't really want to, or are not able to, contribute, especially not by doing things, and that's OK. (They did contribute, however, by filling in the survey.)

  • When given an opportunity to say whatever they want to Obnam developers, most people say "thank you" in some form or another. This was very heartwarming.