Children of a Lesser Good
<p>Regular readers of this blog are probably pretty clued in about my better-known software projects &#8211; gpsd, fetchmail, giflib, libpng, INTERCAL, ncurses, Battle for Wesnoth, Emacs VC and GUD modes, and the like. If those are the best, what about the rest?  Here&#8217;s a tour of some of the lesser-known stuff I&#8217;ve written or had my fingers in. Warning: obscurity, trivia, and obsolescence lie ahead!</p>
<p> <span id="more-2709"></span></p>
<h2>Retrolanguages</h2>
<p>It&#8217;s no news to my regular readers that I derive a perverse thrill from  reviving horrible ancient languages from the paleolithic strata of computing. INTERCAL is my best-known such project, but there are others.</p>
<p><a href="http://freshmeat.net/projects/cupl">cupl</a> actually gives you an interpreter for <em>two</em> revenants from the dawn age &#8211; Cornell University Computing Language (CUPL) and Cornell Compiler (CORC).</p>
<p><a href="http://freshmeat.net/projects/mad">mad</a> is the Michigan Algorithm Decoder, a wonderfully strange old language from the realy 1960s. What can you say about a lanuage where you have explicit operations to twiddle a  stack of values of a variable as a substitute for the function local scope you don&#8217;t have?  Yes, I know; it sounds like INTERCAL. No, not quite.</p>
<p><a href="http://freshmeat.net/projects/mixal">mixal</a> is an implementation of the hypothetical MIXAL assemply language from Donald Knuth&#8217;s &#8220;The Art Of Computer Programming&#8221;.  I didn&#8217;t write this, but I  ported it to Unix and got the permissions required to redistribute the  MIX examples and description from the book.</p>
<h2>Development Tools</h2>
<p>One of my very first C programs back in the early 1980s was <a href="http://catb.org/~esr/hexdump">hexdump</a>.  Yes, Unix had the <b>od</b> command, but I always prefered the CP/M style of hex dump &#8211; 16 hex bytes per line with ASCII over on the right and a gutter in the middle that helps you figure offsets by breaking them into groups of 8. Because I was working on an SNA terminal emulator when I wrote it, this hex dumper also has the unusual ability to dump EBCDIC. A  quarter-century later EBCDIC is long gone from my life but I still think the CP/M display style of hex display is superior.</p>
<p><a href="http://freshmeat.net/projects/semex">semex</a> is almost as old.  No, nothing to do with plastic explosive; it&#8217;s an interactive excerciser for the System V semaphore facilities found in Linux and other modern Unixes. You can use it to learn how they work with less friction than writing your own C code, or as a callable tool for use from scripting languages that don&#8217;t have a native binding to the semaphore calls.</p>
<p><a href="http://catb.org/~esr/ascii">ascii</a> is a handy little utility that recognizes many different ways of naming an ASCII character (hex, octal, binary, decimal, C escape, ISO character table pair, slang names, and others) and prints out all the equivalents.</p>
<p><a href="http://www.exactcode.de/site/open_source/minised/">minised</a>, aka small-sed, is the fast, small sed originally distributed in the GNU toolkit and still distributed with Minix. The GNU Project dropped it when they built their own sed around an enhanced regexp package, but minised still better for some uses (in particular, it&#8217;s faster and less memory-intensive).</p>
<p>Occasionally I hare off in a direction nobody seems interested in. Back when GNU gettext was just getting known, I thought it was too  heavyweight. I wrote my own tool for internationalization, <a href="http://catb.org/~esr/cstrings">cstrings</a> and used it on a couple of small C projects. Alas, I don&#8217;t think anyone else ever did.</p>
<p>When I was an active libpng developer, I wrote the support for six of the more obscure chunk types that had been described in the PNG standard but not implemented in the reference library.  It frustrated me that there were no decent tools for doing surgery on the more obscure chunk types, so I wrote <a href="http://freshmeat.net/projects/sng">sng</a>. This was a program that could unpack a PNG into an editable text  form <em>and reverse the transformastion</em>.  Need to wrote a program to tweak, say, the meters-to-pixel field in a pHYs chunk?  No problem &#8211; explode the PNG with sng, hack it in your favorite text editor, and  unexplode it with sng. Writing the regression-test suite was easy :-).</p>
<p>I&#8217;d love to know if anyone but me ever got any use out of my C-to-Python translator, <a href="http://freshmeat.net/projects/ctopy">ctopy</a>. It turns out that regexp-bashing can do about 80% of the job on well-formed C. I translated a couple of fairly large programs with it, including Super Star Trek (which see below).</p>
<p>It&#8217;s not likely there are many SCCS files left out there, but if you know what one is and you want to lift it to RCS, <a href="http://catb.org/~esr/sccs2rcs">sccs2rcs</a> is your tool. This isn&#8217;t the C-shell version that floated around unmaintained for many years, it&#8217;s my Python rewrite with much better error checking.</p>
<p>Because I cleverly got it accepted into the Subversion suite, svncutter is not its own project any more. This is a tool for slicing and dicing Subversion repositories.  The single trickiest thing it can do is merge runs of adjacent commits with the same commit comment &#8211; a not-uncommon form of scar tissue when a repo has been up-converted from SCCS/RCS/CVS. It can also selectively delete commits and excise files  from a repo history entirely.</p>
<p>Now we reach <em>deep</em> into obscurity for <a href="http://catb.org/esr/yacchack/">yacchack</a>, a project so ancient and neglected that I never even gave it a web page.  It was a set of hacks  to enable building multiple YACC parsers into the same runtime, turning the parser globals into members in a state structure that you could have more than one of.  I remember that this worked once but have forgotten what I wrote it for.</p>
<h2>Communications</h2>
<p><a href="http://freshmeat.net/projects/ssh-installkeys">ssh-installkeys</a> installs your SSH keys on remote sites for you.  Unlike its direct competitor ssh-keyinstall, you need enter your password only once; it uses a send/expect library to run an ssh session that does all the work.</p>
<p>The web is a wonderful thing, but I hate, hate, <em>hate</em> having to do mechanical clicky-dances on websites; service designers who don&#8217;t provide an API for scripting transactions with them deserve to be hung up by their thumbs (yes, SourceForge, I&#8217;m talking to you!).  My <a href="http://freshmeat.net/projects/freshmeat-submit">freshmeat-submit</a> tool exercises the API at <a href="http://freshmeat.net">freshmeat.net</a>; with it, you can post release announcements and edit project metadata.  This tool is actually intended to be called by other programs as part of your release process.</p>
<p>In fact, the main user of freshmeat-submit is my <a href="http://freshmeat.net/projects/shipper">shipper</a> program. shipper automates the tedious process of shipping a software release to standard places, including your own hosted website, FTP archive sites, and the like.  Optionally it can generate and upload a templated web page for the project and announce to Freshmeat.  This program is in alpha state and rapidly acquiring more capablity.</p>
<h2>Document and Web Tools</h2>
<p>There are few things more entertaining than to succeed at programming an algorithmic solution to a problem the domain experts thought was impossible to automate. It was supposed to be impossible to lift the ugly-presentation-level tag soup in troff markup (man, ms, mm, me etc.) to nicely structured XML-DocBook.  But, it turns out, with a sufficiently clever application of cliche analysis it can be done. <a href="http://freshmeat.net/projects/doclifter">ctdoclifter</a> has about a 95% successful conversion rate.  This may be the most algorithmically dense code I&#8217;ve ever written.</p>
<p>I found the conditionalization facilities in XML-DocBook too painful to use, so I wrote <a href="http://freshmeat.net/projects/xmlif">xmlif</a> to fix this.  It&#8217;s a simple preprocessor that interprets XML processing instructions in your text body in a way that allows you to conditionally include or export different portions of the text.</p>
<p><a href="http://catb.org/~esr/vh">vh</a> stands for Volks-Hypertext. Back when I was first working on the Jargon File, I distributed it for a while in a flat-text form in which all the references to entries were surrounded by curly brtaces, a la &#8220;{foo}&#8221;.  A guy named Ray Gardner thought &#8220;Hey, why don&#8217;t I&#8230;&#8221; and write what we&#8217;d now call a browser for the File &#8211; it was screen-oriented, could  page up and down, and could chase links.  The right and left arrow keys hopped from link to link on the page; press enter on a link and go to the entry.  The  idea for the browser was Ray&#8217;s, but some of the interface polish was mine.</p>
<p>What makes this interesting is that we weren&#8217;t imitating the text-mode  lynx browser and the World Wide Web &#8211; we were <em>anticipating</em> it. Shortly after vh&#8217;s first release in 1991 I got email from a  then-obscure programmer named Tim Berners-Lee hinting that he was doing some hypertext-related stuff and asking if I might be interested in collaborating on some standards.  I replied &#8220;yes&#8221;, but never heard back from him.</p>
<p>Of course Tim Berners-Lee didn&#8217;t stay obscure for very long after 1991.  I&#8217;ve wondered ever since if Ray Gardner and I actually influenced the design of HTML. If I ever meet Sir Tim, I&#8217;ll ask him.</p>
<p>The urgency to get rid of GIFs is gone now that the blocking patent has expired, but PNG is still a better image format and <a href="http://freshmeat.net/projects/gif2png">gif2png</a> is still your handiest tool for mass-converting the things.  I didn&#8217;t write it, but I&#8217;ve been the maintainer for a decade &#8211; and I did write web2png, which can convert an entire hierarchy of web pages in place, patching links as needed.</p>
<p><a href="http://freshmeat.net/projects/sitemap">sitemap</a> is a tool for automatically building a site map from your web pages, meant to be run as a cron job.  It mines indexing information  about the pages from meta tags.  I&#8217;ve had it running on my  website for thirteen years.</p>
<p><a href="http://freshmeat.net/projects/imgsizer">imgsizer</a> optimizes a web page, or an entire tree of them, by analyzing IMG tags and adding width and height attributes to them derived from analyzing the image. This hint to the layout engine of a browser can significantly speed up page rendering.</p>
<p>Because I&#8217;ve been such a Unix guy for the past million years, some people might find it difficult to believe that I ever worried about moving data back and forth from DOS/Windows systems. But I did, back in the 1980s, and wrote <a href="http://catb.org/~esr/mstrans">mstrans</a>  so I could batch-convert entire directories of manuscript files. It&#8217;s not hard to imagine where this could still be useful.</p>
<h2>Games</h2>
<p>Battle For Wesnoth isn&#8217;t my first game project, no, not by any means. I was a <a href="http://en.wikipedia.org/wiki/NetHack">Nethack</a> developer for many years; heck. it&#8217;s my doing that Nethack has color on the screen.  I used to write and port a lot of games back in the days when we used terminals instead of GUIs. Some of these retain considerable play value today; one or two even qualify as timeless classics.</p>
<p>Save the Federation from the invading Klingons!  Visit exotic planets and strip-mine them for dilithium!  Encounter mysterious space thingies!  Long, long ago, there was a family of tremendously influential games called &#8220;Star Trek&#8221;; the earliest of this seems to have been a tiny BASIC game from 1971.  One of the most influential versions was written in FORTRAN in 1973-1974 at the University of Texas; it was eventually translated into C to become BSD Trek. <a href="http://sst.berlios.de/">Super Star Trek</a> is the UT game, translated into Python and with most of the features from BSD Trek and several later versions folded in.  </p>
<p>Super Star Trek is, no question, a timeless classic. You can play it in either its original TTY-like mode, or you can enable the  screen-oriented interface.</p>
<p>Almost as venerable is <a href="http://catb.org/~esr/wumpus">wumpus</a>.  Hunt The Wumpus was first written in BASIC in 1974, and seems to have the first game with a map that was topologically interesting, e.g not a grid.  You hunt the wumpus in an icosahedral cave, hoping pot him by shooting one of your limited supply of arrows through nearby rooms.  But beware lest you stumble over the Wumpus himself (which will eat you) or fall into a pit or get carried off by bats!  This implementation lovingly preserves the crude interface of the original. Also includes superhack, which is a structurally similar game with a different premise.</p>
<p>Or there&#8217;s my <a href="http://freshmeat.net/projects/bs">bs</a> game from 1993 &#8211; yeah, it&#8217;s Battleships for the terminal screen. The point-and-shoot with arrow keys is primitive compared to clicking a mouse in a GUI&#8230;but, oh, wait, you can point and shoot with a mouse in this thing, too.  That&#8217;s not something you see often in a program running in a terminal emulator, and the magic I used is unlikely to work anywhere but under Linux. (Not originally mine, but I rewrote it to make the UI not suck.)</p>
<p>Then there&#8217;s <a href="http://catb.org/~esr/bluemoon">bluemoon</a> a nice card solitaire game that&#8217;s not injured by lacking a GUI. (Also not originally mine, but I rewrote it to make the UI not suck.)</p>
<p>Once upon a time in 1994, I saw a neat little game for the old pre-Unix Macintosh called &#8220;Galaxis&#8221;.  I thought it was fun, so I wrote a <a href="http://freshmeat.net/projects/galaxis">workalike</a>.</p>
<p><a href="http://catb.org/~esr/greed">greed</a> is a game that might not work as well if it did have a GUI. You use arrow keys to walk around a field of digits, trying to eat as many as possible. The catch is that the length of your next move is set by the digit you eat, and you can&#8217;t cross your own back trail. Somebody named Matt Day wrote this back around 1989 and then disappeared.  I&#8217;ve kept it maintained since 1994.</p>
<p>Imagine you are skiing down an infinite slope, facing such hazards as trees, ice, bare ground, and the man-eating Yeti!  Unfortunately, you have put your jet-powered skis on backwards, so you can&#8217;t see ahead where you are going; only behind where you have been.  However, you can turn to either side, jump or hop through the air, teleport through hyperspace, launch nuclear ICBMs, and cast spells to call the Fire Demon.  And since the hazards occur in patches, you can skillfully outmaneuver them.  This game,  <a href="http://catb.org/~esr/ski">ski</a> would <em>definitely</em> not work as well with a GUI. Not my design, but very funny and worth my effort to maintain it.</p>
<p>It&#8217;s not real likely anyone will ever want to play Tetris on a terminal screen again, but in case they do they can dust off  <a href="http://freshmeat.net/projects/tetrix">tetrix</a>.  Originally written for the Amiga; I ported it to Unix. Probably best mercifully forgotten now.</p>
<p>Finally, there&#8217;s <a href="http://freshmeat.net/projects/vms-empire">VMS-Empire</a>. This game was ancestral to the whole genre of 4X (expand/explore/exploit/exterminate) games, including Civilization and Master of Orion. Its history before 1990 is a bit murky, but I&#8217;ve been maintaining it since.  This game should <em>not</em> be  forgotten; it, too, is a timeless classic.</p>
<h2>Miscellanea</h2>
<p>I used to play Magic The Gathering.  To help me evaluate the probability of, say, pulling any of 3 Hurloon Minotaurs in my first 7 cards of a 60-card deck, I wrote <a href="http://freshmeat.net/projects/deal">deal</a>. It computes hypergeometric distributions for non-replacement probabilities. It&#8217;s based partly on an earlier program called &#8220;cardprobs&#8221; that seems to have sunk without trace.</p>
<p>Bayesian filtering has become a very popular spamfighting method. Shortly after Paul Graham invented the technique in 2002, I observed sadly that, much as I love Lisp, his use of it as an implementation language made it far less likely that this worthy idea would get the traction it deserved.  I wrote <a href="http://freshmeat.net/projects/bogofilter">bogofilter</a>, the first Bayesian spam-filter in C.  I handed it off to others about  a year later.  It has achieved extremely wide use.</p>
<p>In 2003, as a reaction to the SCO lawsuit, I wrote a <a href="http://freshmeat.net/projects/comparator">comparator</a> that uses a shred-and-hash technique find similar portions of program text in very large source trees (fuzzy matching after whitespace normalization, brace, and comment removal is supported). Ron Rivest (yeah, the &#8220;R&#8221; in RSA) was kind enough to design a custom hash function for me. A main virtue of this program is that it&#8217;s eye-poppingly fast &#8211; I have actually seen it compare up to <em>two million lines of code per minute</em> on a 1.8GHz Intel box.</p>
<p><a href="http://freshmeat.net/projects/letterize">letterize</a> takes a phone number and tries to turn it into the most plausible string of English characters that you could type on a phone keypad to get that number.  Most plausible, that is, given the relative frequency of trigraphs in English.</p>
<p><a href="http://freshmeat.net/projects/morse">Morse Classic</a> is a Morse code training program for aspiring radio hams. I was handed the maintainer&#8217;s baton years ago but haven&#8217;t had to do much more than merge sporadic contributions from others; the code is old but very stable.</p>
<p><a href="http://freshmeat.net/projects/nolan">nolan</a> was a program that gave the <a href="http://www.nolanchart.com/survey.php">Nolan  Quiz</a>, something we&#8217;d now hardly think of doing with anything but  a Web form.  The implementation technique was kind of interesting, though; I wrote a little markup language that was compiled to C by a script written in sed and awk.  </p>
<p><a href="http://freshmeat.net/projects/xve">X Video Explorer</a> was a tool I wrote in conjuction with the <a href="http://tldp.org/HOWTO/XFree86-Video-Timings-HOWTO/">XFree86 Video Timings HOWTO</a> I used to maintain. Because mdern display hardware autoconfigures using EDID, the tool and the HOWTO are both obsolete.</p>
