The value of long lived software in AI driven world

Over a quarter of a century ago, Joel Spolsky wrote about Netscape’s decision to rewrite its browser from scratch, which later led to Firefox. He called it Things You Should Never Do, Part I.

I remember reading that article at the time and fully agreeing with him. As a side note, it is well worth the time to go through the archive of his posts. Even today, a lot of them hold up very well.

Reading it back again, I found the following quote, which is really appropriate today:

Before Borland’s new spreadsheet for Windows shipped, Philippe Kahn, the colorful founder of Borland, was quoted a lot in the press bragging about how Quattro Pro would be much better than Microsoft Excel, because it was written from scratch. All new source code! As if source code rusted.

To be clear, he was writing in (April 2000) about something that happened in 1992, so 34 years ago. That statement is still very much in force today. Another quote from this article that still lives in my head is:

Back to that two page function. Yes, I know, it’s just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I’ll tell you why: those are bug fixes. One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn’t have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. That LoadLibrary call is ugly but it makes the code work on old versions of Windows 95.

There is an interesting question here. Do we have the same problem with AI? After all, AI can write a lot of code. Does it matter?

This post was started when I was working on optimizing RavenDB. That is something that I do as a habit, in an ongoing process. Right now I’m looking into optimizing the scenario of how RavenDB starts up after a cold boot, when running on an I/O-constrained system.

The scenario is a 100GB database running on an AWS gp3 disk with 125MB/sec and 3K IOPS. In most cases, you size up your machines so the working set will fit comfortably in memory. But what about when you start up after a cold boot (memory is empty)?

In that case, the access pattern you have for the data is really important. That is also not something that you’ll typically see in “I used AI to generate a storage engine”. That is the sort of “detail work” that is not interesting to teach (no fancy coding here); it isn’t “my benchmark got 15% faster” (it is very specific to a single scenario).

In long-lived software, however, you accumulate such features. Over time, those end up being a very significant advantage.

Another gem from Spolsky is this blog post, where he talks about software bloat, which is the other side. Then he points out that the review for a new word processor pointed out that the software is very fast, but it lacked (for a journalist) an essential word count feature.

Can you get AI to write all those features and behaviors? Of course, but in order to actually run into them (so you know what to do), you need actual in-field experience.

To really hammer that home, you need that support call when they are looking at the downtime clock while the disk is reading in small increments and their business is down. It’s the same as the old story about the chalk mark.

Going back to the original scenario of optimizing RavenDB for I/O-constrained environments, you need to understand how disks work. In general, for a gp3 disk, the cost of reading 8 KB is the same as reading 128 KB. If you are working from a cold boot and try to read an 8 KB page from the disk, it actually makes sense to read more to try to get things into memory faster.

That is where things get interesting, because the previous statement is true only about IOPS; you also have to balance bandwidth limits, as well as what happens when your memory is full and you’ll need to evict data from the cache.

RavenDB is using prefetching; it will analyze patterns in how you access the data and proactively try to read more when it can. That can dramatically improve cold boot performance.