The Limits of Open Source
<p>A mailing list I frequent has been discussing the current financial meltdown, specifically a news story claiming that Wall Street foooled its own computers by feeding them risk assumptions the users knew were over-optimistic. </p>
<blockquote>
<p>This is also a very strong case for F/OSS software. Had such software been in use, I strongly feel that the inherent biases programmed in would have been found.</p>
<p>But then, that&#8217;s also true for voting machine software.</p>
</blockquote>
<p>As the original begetter of the kind of argument you&#8217;re making, I&#8217;d certainly like to think so&#8230;but no, not in either case.</p>
<p><span id="more-515"></span></p>
<p>You&#8217;re making an error in the same class as believing that the design of security systems is just a matter of getting the algorithms and protocols right.  Bruce Schneier could set you straight on that one real fast. Perhaps he will [Schneier is on the list].</p>
<p>Open source is great for verifying the integrity of the software itself, but doesn&#8217;t necessarily give you any purchase on auditing the software&#8217;s <em>assumptions</em>.  Suppose the software is modeling physics: it&#8217;s not too difficult under open source to verify that (say) it&#8217;s using the textbook value of G, in the Newtonian Law of Gravity, but verifying that the textbook value of G is physically correct is a different and far more difficult problem.</p>
<p>Similarly, if you&#8217;re looking the source code of complex risk-modeling  software, it&#8217;s relatively easy to know that the model logic is being implemented correctly.  But this gives you no purchase on whether the model is correctly descriptive of real markets.  Or real climate systems, or whatever.</p>
<p>How you find the right coefficients for the partial differential equations (and whether you&#8217;re using the right PDEs at all) is not a software problem and cannot be addressed by software engineering methods.  How you verify those coefficients are correct isn&#8217;t a software-engineering problem either.  Usually it involves running your model on old data and seeing if it retrodicts correctly.  Usually the big problem there is whether you can find that data at all, or trust it when you find it.</p>
<p>None of the special risks in voting-machine software are addressed by open source either.  Yes, it&#8217;s a good idea for the same software-engineering reasons open source is a good idea for all software, but!  Open sourcing the software cannot guarantee that the voting machine is actually running the correct software that you think it is, rather than a version that has been maliciously corrupted. Open source cannot guarantee that the data the software reports is not tampered with in transit or at the receiving end.</p>
<p>These problems can be addressed, but it takes sound design of the overall system at so many higher levels that open source is really only a minor part of the toolkit.</p>
