Blogger Issues

Yesterday afternoon I wrote a blog post about my gSuneido progress. In the evening I got an email saying "Your post has been deleted" because "Your content has violated our Malware and Viruses policy."

The post was just some text and a couple of screenshots. It's hard to see how it could contain malware or viruses. Of course, it was gone, so I couldn't prove that. And of course, there was no human to contact.

It was a funny because it actually made me a bit upset. I think that was partly from the feeling of helplessness against the faceless Google behemoth. A bit like dealing with the government. And it's free, so what can you say?

This morning I got another email saying "We have re-evaluated the post. Upon review, the post has been reinstated." Who exactly is "we"? Somehow I doubt that was a human. Now our software gets to use the royal "we"? I suspect it would have been more honest to say "sorry, we screwed up"

It was still not showing up, but then I found they had put it back as a draft and I had to publish it again.

A quick search found someone else reporting a similar issue and Google responding with "we're aware of the problem".

It was a good reminder to back up my content. Not that there's anything too important, but it's of nostalgic interest to me, if nothing else. (You can download from the blog Settings.)

Another gSuneido Milestone

This screenshot probably doesn't look too significant - just the Suneido IDE. The only noticeable difference is down in the bottom left corner. Normally it would show something like:

Instead it shows:

That means gSuneido is running "standalone", i.e. using its own database instead of connecting as a client to a jSuneido database. While the surface difference is tiny, internally this is a huge jump.

I've been working away on the gSuneido database over the last year at the same time that we've been debugging and then rolling out the gSuneido client.

If I had just ported the jSuneido database implementation it would have been much easier, but what would be the fun in that. I kept the query implementation but redesigned the storage engine and transaction handling. I'd call it second system effect, but it's more like third system since I also redesigned this for jSuneido.

I still have lots to do. Although the IDE starts up, it's quite shaky and easily crashed. Many of the tests fail. But even to get to this point a huge number of pieces have to work correctly together. It's a bit like building a mechanical clock and reaching the point where it first starts to tick.

Twenty Years of cSuneido

Last week was a milestone. We (my company) finished converting all our customers from cSuneido (the original C++ implementation) to gSuneido (the most recent implementation, in Go).

That means I no longer have to maintain cSuneido and I no longer have to deal with C++. (sigh of relief)

cSuneido has had a good run. We first deployed it to the first customer in 2000, so it's been in continuous production use for over 20 years. It's served us well.

When I first started developing Suneido, in the late 1990's, C++ was relatively new on PC's. I started with Walter Bright's Zortech C++ which became Symantec C++ (and later Digital Mars C++). Later I moved to Microsoft C++ and MinGW.

Suneido, like most dynamic languages is garbage collected. But C++ is not. I implemented a series of my own increasingly sophisticated conservative garbage collectors. But eventually I admitted my time was better spent on other things and I switched to using the Boehm-Demers-Weiser conservative garbage collector. Under normal use conservative garbage collection works well. But there are cases where memory is not recycled and eventually you run out. That was somewhat tolerable on the client side, but it wasn't so good on the server side. (That was one of the factors that prompted the development of jSuneido, the Java version that we use on the server side. Another factor was the lack of concurrency support in C++ at that time.) It seemed for a while that the pendulum was swinging towards garbage collection. But Rust has given new life to manual memory management.

Honestly, I won't be sorry to leave C++ behind. It has grown to be extremely complex, and while you can avoid much of that complexity, it's hard to not be affected by it. I've also had my fill of unsafe languages. Even after 20 years of fixing bugs, there are very likely still things like potential buffer overflows in cSuneido. (Ironically, one of the things that added a lot of complexity to C++ was template generics. Meanwhile I am anxiously waiting for the upcoming addition of generics in Go. However, Go's generics will be much simpler than C++'s Turing complete template programming.)

While it might seem crazy to re-implement the same program (Suneido) three times, it's been an interesting exercise. I've learned things each time, and made improvements each time. It's been extra work to maintain multiple versions, but it's also caught bugs that would have been missed if I'd only had a single implementation. Doing it in three quite different languages - C++, Java, and Go, has also been enlightening. And having the hard constraint of needing to flawlessly run a large existing code base (about a million lines of Suneido code) means I've avoided most of the dangers of "second system" effects.

So far I've only implemented the client side of gSuneido. We are still using jSuneido (the Java version) for the server side. I'm currently working on implementing the database/server for gSuneido (in Go). Once that's finished I intend to retire jSuneido as well and be back to a single implementation to maintain, like the good old days :-) And given where I'm at in my career gSuneido will almost certainly be the last implementation I do. I wonder if it will last as long as cSuneido did?

Tools: Joplin Notes App

I recently (Nov.) started using Joplin, "An open source note taking and to-do application with synchronization capabilities" and I'm quite happy with it.

I've been a long time Evernote user (over 10 years). Although it was a bit rough at first (see A Plea to Evernote) it has worked well for me. Like Joplin, it meets my requirement for something that runs on Windows, Mac, and phone/tablet, and works off-line. (Joplin also runs on Linux.)

I'm pretty sure Evernote is an example of Conway's Law at work. Their versions have the same overall features, but there are enough small differences to be quite annoying when you're switching back and forth. You'd think someone at a high level would push for consistency. It's stupid stuff like one version puts a button at the right and another at the left. Then they came out with a new Mac version that was missing a bunch of features, and made yet another set of differences.

I've looked for alternatives in the past, but haven't found anything that matched my needs. I can't remember where I came across Joplin. I hadn't found it when I looked in the past. I wasn't specifically looking for an open source solution, but it's nice that Joplin is open source. It seems to have an active and growing community and user base. 

One of the things I like about Joplin is that it's primarily Markdown. Most of my notes are plain text (see Taking Notes) but it's nice to have a little formatting at times. There is a new WYSIWYG editor, but previously it was the standard split screen edit & preview, or toggle back and forth. I mostly stay in the raw markdown mode.

One of the things I feared about switching was having to leave all my old notes behind in another program. But Joplin can import Evernote's export format. I haven't moved everything yet (there's a lot) but I transferred my Suneido notebook which is roughly 2000 notes. It took a little while to sync/upload and then sync/download on other devices but it worked well. There are a few formatting glitches, but that isn't surprising.

Interestingly, Joplin doesn't (yet) run its own sync servers. Instead you can use Nextcloud, Dropbox, OneDrive, WebDAV, or the file system. I already use Dropbox so that was the easiest for me. They are working on their own sync server software.

Since I got Joplin I have hardly touched Evernote. I use Joplin every day to keep notes on my work. If you're looking for a notes app it's worth checking out.

Unix: A History and a Memoir

I just finished reading Unix: A History and a Memoir by Brian Kernighan. I'm not much of a history buff, but I enjoyed it, mostly because it brought back memories.

By the time I was in high school I was obsessed with computers. My father somehow arranged for me to get a Unix account in the university computer science department. I'm not sure of the details - my father worked on campus, but wasn't part of the university, he was an entomologist (studied insects) with Canada Agriculture.

My father also arranged permission for me to sit in on some computer science classes. But my high school principal refused to let me take a few hours a week for it. I can't recall why, something about disrupting my school work. Which is totally ridiculous given that I was at the top of my classes. You wonder what goes through the minds of petty bureaucrats.

I remember being quite lost at first. I was entirely self taught which left plenty of gaps in my knowledge. I was intimidated by the university and too shy to ask anyone for help, But I muddled along, teaching myself Unix and C. This would have been in 1977 or 1978, so the early days of Unix. Of course, it was in the universities first.

I recall being baffled that C had no way to input numbers. For some reason I either didn't discover scanf or didn't realize it was what I was looking for. It wasn't really a problem, I just figured out how to write my own string to integer conversions. When the C Programming Language book (by Kernighan and Ritchie) came out in 1978, that helped a lot.

Software Tools (by Kernighan and Plauger) was probably the book I studied the most. I implemented a kind of Ratfor on top of TRS-80 Basic, and implemented most of the tools multiple times over the years. I still have my original copy of the book - dog eared, coffee stained, and falling apart. A few years later, I reverse engineered and implemented my own Lex and then Yacc. Yacc was a challenge because I didn't know the compiler-compiler theory it was based on. Nowadays there are open source versions of all this stuff, but not back then.

I read many more Kernighan books over the years, The Elements of Programming Style (with Plauger), The Unix Programming Environment (with Pike), The Practice of Programming (with Pike), and more recently, The Go Programming Language (with Donovan).

Nowadays, with the internet and Stack Overflow, it's hard to remember how much harder it was to access information in those days (especially if you didn't talk to other people). I had one meeting with someone from the computer science department.  (Again, arranged by my father, probably because I was so full of questions.) Looking back it was likely a grad student (he was young but had an office). I questioned him about binding time. He didn't know what I was talking about. I don't remember why I was fixated on that question. I must have seen some mention of it in a book. Me and unsolved questions/problems, like a dog and a bone.

Presumably the Unix man pages were my main resource. But if you didn't know where to start it was tough. I remember someone gave me a 20 or 30 page photocopied introduction to Unix and C. That was my main resource when I started out. Nowadays, I'd be hard pressed to make it through a day of programming without the internet.