jSuneido - progress and premonitions

For "fun", I decided to hook up the database request parsing and execution to the server code. It was pretty easy. I could then telnet into the server and create/modify/delete tables in the database. Can't actually do anything with the resulting tables, but still a good milestone.

On a big project, it's great when you can hook up some pieces and have them actually do something. Most of the time you're down in the bowels of the code plugging away at little pieces of functionality. It's nice to get reminded that the whole system is actually going to do something!

I've made good progress in the last week or two. I've been averaging about 200 lines of code per day. Unfortunately, it's getting hard to measure that because the lines of code are exaggerated by things like the generated parsing and lexing code. The metrics tool I'm using doesn't seem to allow excluding some files.

However, in all this progress estimation I've been glossing over one big stumbling block - concurrency. A port of the existing implementation will not be particularly useful. I still have to make it run multi-threaded - arguably a tougher nut to crack.

I just finished "reading" The Art of Multiprocessor Programming. (I'm not sure "Art" belongs in the title of a book full of lemmas and proofs!) It's pretty dense but I found it fascinating. I haven't encountered so many novel algorithms in a long time. Of course, it only reinforced the fact that concurrency is hard, and often counter-intuitive. The chapter on Transactional Memory was really interesting. This sounds like the direction for the future. (check out a podcast from OSCON and a paper from OOPSLA)

The issue of making Suneido multi-threaded has been in the back of my mind throughout this project. Every so often I spend some time thinking about it.

Although it would be possible to just add some naive coarse grained locking to the existing code, that's not going to give good results. I'm pretty much resigned to having to do some redesign to accommodate concurrency.

Yesterday I came up with an idea for using copy-on-write for index nodes. The more I thought about it, the better I liked it. Last night I was lying in bed thinking about it. (I know, what a geek, I can't help it.) It just seemed a little too good to be true. Sure enough, I had been overlooking some critical issues. Luckily (?) I stayed awake long enough to think of another alternative. It was a good example of how concurrency (or any complex system) is hard to reason about correctly with our (mine, anyway) limited brainpower.

PS. In case it wasn't obvious, the "premonition" part of the title refers to the concurrency issue. The definition seems appropriate: "A warning of an impending event, experienced as foreboding, anxiety and intuitive sense of dread." Although I'm not crazy about the rest of the definition: "Premonitions tend to occur before disaster"!