Is closed source worth it for performance?

The following question appeared in my mailbox today:

If a certain program (that you need) was proprietary, and its open-source counterpart was (currently) 40% slower. Which would you use, the open-source one or the proprietary one?

The answer is: it depends. I’m going to answer this one in public because it’s a useful exercise in thinking about larger tradeoffs.

The first subsidiary question I’d have to ask myself is, how much does that 40% speed increase matter? If the answer is “not much” (say, because even the ‘slower’ program runs pretty fast, or because even though it’s noticeably slower I won’t use it often) then I’ll use the program I can read the source code for. Because what if I trip over a bug, or need to extend what the program does to get my task finished?

The more general point here is that by using the closed-source program I’m giving up some significant options, including (a) asking the open-source program’s maintainers for help, and (b) fixing or enhancing it myself. There is a tradeoff between the value of those options and the value of the additional performance which I have to evaluate on the facts in each case. (And yes, valuing these open-source options highly is based on the assumption that help is effectively unavailable from the maintainers of the closed-source program. Sadly, this is usually the case.)

From now on, then, we’re only talking about the case where that 40% is really important. Let’s consider a couple of easy subcases.

It might be the case that what these programs do is very simple and well-defined, so it’s easy to verify that they’re functionally equivalent except for speed. If that’s the case, my risk of getting locked in by the closed-source program is low – so I’ll go right ahead and use the closed-source one, knowing I can fall back to the open-source program at any time.

It might also be the case that buying faster hardware will make the open-source program fast enough for my purposes. Hardware is cheap and the benefits from improving it extend across a broad range of tasks, so I’d probably rather upgrade my hardware than accept the risks of using the closed-source program.

Another easy case is when the closed-source program jails my data, so I cannot examine or modify it with other tools. It is hard for me to imagine any scenario in which I would swallow that for a mere 40% performance boost. Forget it; no sale.

Could I make the open-source program go faster? I would at least spend an hour or two looking at the possibility.

Beyond these the decision process starts to get more difficult. I don’t think I can utter a completely general rule for when I will use closed-source software, but I hope I have illuminated some of the tradeoffs.