The Future is Free?

An interesting article by Chris Anderson (also know for the "long tail") in Wired on how and why more and more things are "free".

Skype, Headsets, and Bluetooth

I'm going to be spending some time away from home without Shelley so I thought I should set up Skype so we could talk. It was no problem to download and install the software on my MacBook and on Shelley's Windows XP PC.

I bought a USB headset for the PC but I wanted something smaller for the MacBook and since it has Bluetooth I figured I could get a Bluetooth headset. I couldn't find a computer specific bluetooth headset, only cell phone ones. But would they work with the Mac? I did some research on the web and read about lots of problems, but mostly old issues that were supposedly fixed with Leopard. I didn't even bother trying to ask a clerk at Staples which bluetooth headsets worked with Skype on OS X on a MacBook. Although I guess if you were lucky you might get some kid who was an expert on the issue.

I chose a Bluetrek Tattoo headset, more or less at random. I charged it up, and managed to pair it with the MacBook, but I couldn't get any sound in or out of it. I couldn't even tell if it was "on" or not. It was supposed to have a green light when it was switched on but I got no lights. I played around for a while but it just seemed to be dead. I took it back and picked a Motorola H800, again for no particular reason. This time it worked fine, no problems. I still don't know if the Tattoo is incompatible, or if I just got a dud.

So now I have Skype working. The sound quality of the "echo" test call wasn't great but it says it's going to the UK so that might be part of the reason.

Of course, what I paid for the two headsets would have paid for more than enough regular long distance phone calls, but what would be the fun in that!

Suneido Build Frustrations

I made a minor improvement to the Suneido source code this morning, ran make, no problems, built-in tests ran successfully.

But when I tried to use the new executable I got an obscure database error. What's going on?

I had built with MinGW so I switched to Visual C++. Exact same error!?

This was at home and my last builds had been at work but I can't see why that would matter.

Remove all the object files and build from scratch. No good, same problem.

Check version control to see what I'd changed lately - very little, and nothing that seemed related to the error.

The error is from the database btree code. Maybe the database is corrupted. But all the exe's, old and new, say the database is fine.

Try creating a new database with just the standard library. Now I get a different error related to the Scintilla source code editing component.

Build a MinGW debug version and run it under GDB. That gives me a clue to what query is leading to the error. It had appeared to be outputting to the database, which seemed odd for start up, but it was actually building a temporary index for a query that it was reading. Although that query in the old working executable doesn't require a temporary index.

Turn on the query tracing at the start of the standard library Init to see where the query is coming from. It's loading the plugins.

Aha, that's why the database with only stdlib gets a different error - because it only has to look for plugins in a single library and therefore no temporary index. Yeah, if I disable the plugin loading then I get the other error.

Two unresolved questions
- why the temporary index in the new builds but not the older build?
- why the later UI error?

And how are these two questions related? (assuming a single cause) It seems like it would have to be something low level, like the garbage collector, to affect such unrelated areas.

Of course, it could be something like an uninitialized variable that happens to get a different value on my home system. But it seems too consistent for that. And something like would likely have been encountered before now.

What is different between my work and home machines? I call the office and have them install LogMeIn on my work computer so I can access it. I try building with MinGW and it works fine. The exe is a different size though. Something is different.

Transfer the home exe to work to see if it's the environment. Same error message, so at least it's not because of my Vista on Parallels on Mac setup at home.

md5sum the files at work and at home and compare. The only real difference is the change I made this morning.

But ... that couldn't be the problem could it? Revert that file and build.

Oh no, this is really embarrassing! It works. The problem was the most obvious first place I should have looked - the change I just made.

I'm really tempted not to post this - it just makes me look stupid.

Why did I go off on a wild goose chase? I guess because the error seemed to be so totally unrelated to my change, and I hadn't built for a while so it seemed likely that there could be a problem. And the change I did seemed trivial so I didn't suspect it. And because it seemed trivial I didn't write any tests. (The bug was also obvious, once I looked for it.)

Ouch. There goes a few hours down the drain. Maybe I learned a lesson, but sadly it's one I should have learned a long time ago.

ZENN and the art of slow progress

One of my recurring complaints is gadgets that are only available in the US. But at least it makes a certain amount of sense when they're made in the US. But here's an example where it's made in Canada and it's still only available in the US!

Creating Passionate Users Video

From the O'Reilly Tools of Change for Publishing conference. I like Kathy Sierra's Head First computer books and was a fan of her blog.

VirtualBox

In Tim Bray's ongoing blog he mentions Sun's acquisition of Innotek, the developers of VirtualBox - virtualization software for Windows, Linux, and OS X. I haven't tried it, but it looks interesting - and free for personal use.

Catching Up

I've been traveling in Ecuador for the last month and although I managed to check my email and update my personal blog periodically I haven't been keeping up on news in the computer world.

One big acquisition was Sun buying MySQL. Sun seems to have done ok with Open Office, hopefully things will work out ok with MySQL.

And Microsoft is trying to buy Yahoo. I find this a little scary. I wonder what would happen to things like Flickr. It seems unlikely that Microsoft could manage to stay "hands off".

I knew Apple would be announcing new stuff while I was gone and I kept meaning to check it out. It didn't turn out to be anything too exciting. A cool new smaller laptop, but since I bought a MacBook not that long ago, that makes me annoyed more than anything! They also announced movie rentals which is interesting. It makes an Apple TV more attractive.

Of course, then I found out movie rentals are US only. Argh! I guess I shouldn't complain. I have no desire to move to the US and Canada has better access to new technology than many/most countries. But this "US only" thing seems to be getting more common. I just heard that Bug Labs new product (that I've been waiting for) is also initially US only.

Sometimes I enjoy the relentless rush of new hi-tech products, other times it's annoying. I buy a MacBook, they announce the thinner, lighter MacBook Air. I buy a Pentax K10, they announce the K20. In most cases I don't need the new product, the old one doesn't suddenly stop working. But it's hard to stop yourself from thinking "if I'd just waited a few months". But you know that doesn't work, the treadmill doesn't stop. Of course, when you're waiting for a new product or feature then it seems to take forever to arrive.

My One Laptop Per Child OLPC XO arrived while I was gone.



It took me a while just to figure out how to open it. I was trying to use the latches on the bottom but they're for the battery. You have to open the antennas to unlatch it. It powered up ok but the user interface was pretty cryptic at first. I kept trying to click by tapping on the touch pad (like I'm used to on other laptops) but that doesn't work. My next problem was that the left "mouse" button is labeled with an "X" which I automatically associate with "close". So I kept trying to use the right button which is labeled with "O". I should have ignored the labels - the left and right buttons work like other systems. Once I got past these roadblocks it got easier.

The keyboard is too small to touch type on with adult hands but I was expecting that. The screen is nice and is quite readable in direct light with the backlight turned off - this saves power and also lets you use it in bright sunlight. I managed to connect to my home wireless network without too much trouble (had to pick hex and shared key) and was able to browse the internet and check my email. The built in camera seems to work fine.

The only real problem I've run into is that I get two CRC errors when I boot up. I'm guessing they're from the hard drive. So far they don't seem to be causing any trouble.

I don't have a particular use in mind for the XO, it's just interesting to see what they've come up with after hearing about the project for so long.

Also waiting for me were the new books by Christopher Alexander (of design patterns fame). Now I just have to find time to read four hefty volumes. Lots of pictures at least :-)

A Programming Hierarchy of Needs

After I wrote my last post - There's More to Software Design I was thinking about why so many programmers don't concern themselves with issues like style, grace, and elegance.

It occurred to me that part of the explanation might be something like Maslow's hierarchy of needs. Maslow theorized that people have a hierarchy of needs and that higher levels only come into play after lower levels are satisfied. The lower levels are things like food and shelter; the higher levels are things like morality and creativity. e.g. oversimplifying, starving people don't worry as much about morality

The analogy is probably obvious. The lower level of programming needs would be to get the code to [appear to] work. The inserted "appear to" is important. Without it, "getting the code to work" could be everything up to and including formal verification. Whereas I'm talking about most novice programmer's concept of "getting it working" which is to pass a few random tests.

Things like automated tests, test driven development, and refactoring would be higher up the hierarchy. (Or the equivalent from your methodology of choice.) And things like style, grace, and elegance would be near the top.

And the reason most programmers don't get to the top levels is that they're still struggling with the lower levels. That's not really their fault, either. Programming is unquestionably hard. Just to get something to work in the weak sense can be more than enough challenge.

Note: Just because I think about the higher levels doesn't mean I'm more "advanced". I'm still struggling along at the lower levels with my own code. I just like to dream :-)

There's More to Software Design

There's more to software design than just the "mechanical" aspects.

This article by Mark Hamburg about Lightroom's Goals should give you an idea of what I'm talking about. (And I think they've been fairly successful with this in Lightroom.)

I struggle with this with my company's vertical application, partly because it's hard to get people to see that issues like style, grace, and elegance are relevant to a business application. I think they are. I don't mean it has to be "pretty" or "artsy". But it should look good, flow well, be smooth not awkward. Part of this is definitely the mechanical aspects but part of it is more subtle things. I'm reminded of "quality" in Zen and the Art of Motorcycle Maintenance.

Still Learning

Even though I created Suneido and have used it pretty heavily for years, I still find myself learning better ways to apply it. (I think that's part of the reason I like programming.)

Suneido uses the open source Scintilla editor. ScintillaControl is the "wrapper" that interfaces Scintilla to Suneido's user interface framework. I needed to add a new method to it today:

    LineEndExtend()
        { .SendMessage(SCI.LINEENDEXTEND) }

I noticed I had a lot of these methods and I started wondering whether there wasn't a better alternative to adding so many simple repetitious methods. Ruby, and especially Rails, which I've been involved with on another project, make heavy use of "catching" calls to missing methods and implementing them.

I was able to replace all these simple methods with:

    Default(method)
        {
        f = method.Upper()
        if not SCI.Member?(f)
            throw "method not found: " $ method
        return .SendMessage(SCI[f])
        }

"Default" is Suneido's way to "catch" calls to missing methods.

Then I realized I could generalize it to handle methods with arguments:

    Default(@args)
        {
        f = args[0].Upper()
        if not SCI.Member?(f)
            throw "method not found: " $ args[0]
        args[0] = SCI[f]
        return .SendMessage(@args)
        }

"@args" is used to capture all the arguments and then pass them again. args[0] will be the method name.

This allowed me to remove a bunch of methods and I won't have to add any more in the future.

In addition, I noticed I had a lot of calls like .SendMessage(SCI.GETLINECOUNT) within the wrapper code. These could now be simplified to be like: .GetLineCount()

This would all fall into the category of "refactoring" since I'm improving the code without changing its behavior. (Strictly speaking, the behavior has changed slightly, but not in a way that should affect existing code unless someone is doing something unusual.)

I guess you'd call this refactoring something like: "Replace explicit methods with catching missing method calls."

More on Scratch

A few comments on Scratch:

I'd really like to be able to browse the code for the projects on the web site. (Unless there's some way I missed.) You can download the projects and presumably see the code that way but that's a bunch more steps and not very good for exploring. Since there isn't much documentation, it would be helpful to quickly look at other people's code. It doesn't seem like this would be hard to add.

Apart from the convenience, I think this is important for deeper reasons. Programming, and thinking "like a programmer" are as much or more about reading code as writing it. Seeing other people's results can give you ideas and inspire, but seeing how they did it is going to be a huge benefit too.

A suggestion for Scratch itself is to get rid of the traditional open/save file management. Alan Cooper in About Face 3 makes a good case for why open/save sucks. I never have to "open" or "save" in Lightroom. Gmail and Blogger save automatically. I don't have to pick/navigate to a directory in Google Docs. In a product for kids especially, you could avoid a bunch of issues by saving automatically to a standard location.

Finally, it's too bad Scratch is so rigid with respect to screen/window sizes. I can understand why they did it that way - it's a lot simpler than trying to use vector or higher resolution images and make things resizable. And for educational purposes maybe it doesn't matter. (Although I notice a number of people wanting to run it 800x600.) Nonetheless, it was a bit disappointing when I ran my program full screen for the first time and got a jagged grainy image (as a result of simple resizing of the low resolution stage). Maybe I'm just spoiled by things like Mac OS X's resizable icons.

Something Fun for Christmas

I recently discovered Scratch a programming system for kids, something like Logo.

I decided since it was Christmas I should do something fun and try it out. Here is my first "program":

Ubuntu Networking Resolved

This really shouldn't have taken so long. It wasn't even that difficult. But when you only spend a few minutes on something and only every few days or weeks, what do you expect! And the issues with Parallels and Leopard didn't help.

As Larry suggested, the "expert" solution was to edit /etc/network/interfaces and change:

#iface eth0 inet dhcp

to:

iface eth0 inet dhcp

i.e. uncomment it.

As he also suggested, there is a way to do this from the GUI. When I went to System > Administration > Network I saw this:


[Notice the title bar says "Network Settings" although the menu option was just "Network". I always give my programmers heck for that kind of inconsistency.]

"Roaming mode" ??? I selected Wired Connection, clicked on properties, and changed it to:


[Yet more inconsistencies - I selected "Wired Connection" but I got "eth0".]

i.e. un-checked roaming mode and picked DHCP.

This has a similar effect to the "expert" method, adding a line to /etc/network/interface:

iface eth0 inet dhcp

I can see roaming mode might be a good choice for laptops, but it seems odd that it installed this way. Maybe something in Parallels makes Ubuntu think it doesn't have a regular wired connection. It would be nice if the network icon options at the top of the screen included an option to "save" your choice of wired networking (or just did it automatically).

Now when I reboot I still have a network connection. The tooltip on the network icon now says "Manual network configuration" which doesn't seem quite right to me - DHCP is pretty automatic. But I guess it's more "manual" than "roaming mode" (whatever that is).

I feel a little stupid at not having sorted this out myself right from the start but you can't win 'em all, I guess. Thanks Larry!

Leopard Falters

I spoke too soon about no problems with Leopard. I forgot one major part of my setup - my Epson R1800 wide format photo printer.

I went to print a photo for a Christmas present and found ... no printer. Installing Leopard had silently removed my Epson printer driver. (The CUPS + Gutenprint driver was still there, but I only use it to handle printing from Windows under Parallels.)

I can see a driver not being compatible with a new version of an operating system, but to just silently remove it seems pretty lame. Ideally it would warn you at the start of the install so you had a chance to abort the upgrade if you wanted. At the least it could notify you that it had removed your printer!

Luckily, I had waited long enough to upgrade that Epson had released new drivers. (They were released on Dec. 18 - if I had upgraded a week earlier I'd have been screwed.)

All's well that ends well - my printer is working again and I got my Christmas present done :-)

OpenOffice.org2GoogleDocs

An Open Office extension to transfer documents to and from Google Docs.

I haven't tried it yet, but it sounds useful.

A Successful Leap for Leopard

I upgraded my MacBook to Leopard a while ago but I waited to upgrade my main MacMini.

With Leopard updates for my main apps (Parallels and Lightroom) I decided it was time to take the plunge. The upgrade went smoothly, although it seemed pretty slow - several hours. I'm not sure why it takes that long.

As a safety precaution I used SuperDuper to backup each machine before upgrading.

So far I haven't had any major problems. The first time I started Parallels I got the following error:


I found a blog post which said the MacFUSE included in Parallels is old and suggested installing the latest MacFuse. This seemed to do the trick, but:

• the error message seems backwards - the operating system was new, MacFUSE was old
• why didn't the Parallels update for Leopard include the required new version of MacFUSE?
• why did I have to get the solution from some user instead of from Parallels? even if the user community discovered the solution, wouldn't it make sense for Parallels to post it? (in fairness, maybe they have, but I didn't find it if they did)
  

Note: This problem doesn't stop Parallels from starting, it just stops it from mounting the Windows C drive in OS X

Now that Spotlight with Leopard lets you run the top application match by hitting enter, I was able to uninstall Google Desktop. (Nothing against Google Desktop, I just prefer to keep things simple if I can) (see my previous post)

Leopard also seems to have solved the issue of automatically mounting network drives. (see my previous post) so I was able to remove the login script I had created to do this, which was nice because it took a long time (why?) and slowed down logging in.

I do have a new complaint about OS X. The Finder doesn't have an option to show hidden files. I can understand hiding them by default, so does Windows. But at least Windows gives you a way to show them. This came up when I went to copy the .svn folder from a backup. It is possible to change Finder via a command line, but on top of not being user friendly, this also requires restarting Finder. This seems like an obvious weak point. Is there someone in Apple who refuses to recognize that you might occasionally want to see these files?

I'm still having problems with accessing my 4gb USB thumb drive from Parallels. At first I blamed this on the U3 software that came installed on it, but I removed this and reformatted and I'm still having problems. It works fine on my Windows machine at work. My current guess is that Parallels doesn't quite handle 4gb USB drives. The strange part is that it works fine, but after a short time it will hang during copying from it, and Windows Explorer can no longer access it. My 1gb USB thumb drive continues to work fine.

Ok, back to Ubuntu on Parallels. I copied the virtual machine that I had created on my MacBook over to my MacMini and started it up. No display problem, but the same problem with the Parallels Tool cd image showing garbled file names. Strangely I can't find anybody else with this particular problem. While flailing a bit more I rebooted the VM and lo and behold the Parallels Tools cd image had the right file names. I installed them and restarted X windows. It appears to work! One of the most noticeable features is being able to move the mouse seamlessly between OS X and the VM. I followed the same process on the MacBook and it also worked (although I could have sworn I tried rebooting before). So I appear to be back in business with Ubuntu (albeit starting from scratch with a new VM).

All in all, a successful day!

CouchDB

I found CouchDB referenced from one of the posts about Amazon's SimpleDB since they are both apparently written in Erlang.

It's interesting that people are exploring some alternatives to relational databases.

It's also interesting that people are implementing "real" products in alternative languages like Erlang.

I recently picked up Programming Erlang but I haven't read it yet.

One thing that caught my eye looking through the CouchDB web site was a brief note that they compact the database while it's running, by copying to a new database. Currently Suneido requires you to occasionally shut down the database in order to compact it. I had always thought about "on line" compaction in terms of doing it "in place", but that gets tricky due to updating indexes to point to new locations. But if you build a new copy you don't have that problem. You could copy the bulk of the database in a single read-only transaction (like the current on-line backup does) and then pause activity briefly to get any updates done during the copy, and then switch over to the new database. Hmmm... actually doesn't sound too bad. (famous last words!)

Amazon SimpleDB

Amazon has announced a new service - SimpleDB

We are pretty happy with our use of Amazon's S3 (Simple Storage Service)

I've been curious to try Amazon EC2 (Elastic Compute Cloud) but I haven't found a good application yet.

One of the big limitations with EC2 is that it's not well suited to running database servers. I've been waiting for them to improve support for this, but instead (or at least, first) we get SimpleDB.

I wonder if someone will make Rails work with SimpleDB as the database? How would the performance compare?

Finally!

Finally some good progress on the ACE version of the Suneido server. It's actually working well enough to run a client IDE from it, a major milestone. The last couple of problems were minor mistakes of mine. ACE and the Boehm GC seem to be working together.

Of course, this is just the start, now comes the "fun" part - actually making my code thread safe.

ACE + GC Progress

I spent a few more frustrating hours thrashing around trying to build and link with ACE statically.

Finally, I decided to start from scratch. Strangely "make clean" didn't clean up (as I discovered when a make after make clean didn't recompile!). (Note: Don't run make clean from the top level ACE_wrappers directory - it takes forever recursing into all the examples and tests.)

I rebuilt and ... it worked! Somehow I had still been getting left over shared library stuff. Another requirement is to #define ACE_AS_STATIC_LIBS before include the ACE headers.

Boy, that shouldn't have been so hard! But I can't really blame anyone but myself :-)

But ... now Suneido crashes right away on startup, which seemed more like a step backwards not forwards!

I created a small test program that used ACE and GC. It crashed the same way. Eventually, after another few hours of flailing I hit on the right combination. The key seems to be to initialize GC first, then ACE. But to achieve that, you have to prevent ACE from redefining "main" to do their startup. Here's my successful test program:

#define ACE_AS_STATIC_LIBS 1
#include "ace/Thread_Manager.h"

static ACE_THR_FUNC_RETURN thread_func(void* arg)
{
for (int i = 0; i <>
operator new(10000);
return 0;
}

extern "C" { void GC_init(); }
#undef main
int main(int argc, char**argv)
{
GC_init();
ACE::init();
ACE_Thread_Manager::instance()->spawn_n(2, thread_func);
ACE_Thread_Manager::instance()->wait();
}

At this point I'm quitting for the day. It should be easy to incorporate what I've learned into Suneido, but then I'll just run into the next problem. I'd rather end the day on a positive note!