Using S3 for Customer Backups

For some time we've been using S3 to back up our customers systems.

Originally we set them up to FTP the data back to our office. This was handy for us because if we wanted to look at their data we had it in-house. But as the number of clients and the size of their data grew we ran out of bandwidth. (Thankfully we did these transfers at night so it didn't slow down our daytime internet access.)

The other problem was the sheer size of the data and the issues of how to guard against disk failure. Although we don't promote this to our clients as a "backup" service many of them still end up falling back on us when they discover their own backups weren't done or were no good. And people often don't discover problems till days or weeks later so they need older copies, not just the most recent.

We decided to give S3 a try and so far it has worked out well. We're up to about 8 gb of data transfer per night and we currently have about 240 gb of storage. This is costing us about $50 per month - a bargain as far as I'm concerned.

We are currently keeping the last 8 days, last 5 weeks, last 13 months, and every year, or something like 25 - 30 copies per customer. On top of this redundancy, Amazon says they store multiple copies of each file.

A minor downside is that when we want to look at someone's data we have to download it, but that's not a big deal. And it's still a lot easier to download from S3 than to download directly from the customer.

There is a potential concern with storing data with a third party, but we encrypt the files and Amazon has decent security on top of that, so it seems ok. It's doesn't seem any worse than other hosting situations.

Overall, we're pretty happy with this setup.

G.ho.st - Global Hosted Operating System

G.ho.st – Home Page

Apparently this is implemented using Amazon S3 and EC2. Interesting idea, but pretty annoying how they maximize my browser window and hide the address and tool bars.

Spiceworks - Free Network Monitoring Software for Network Management

Spiceworks - Free Network Monitoring Software for Network Management

Zonbu Green Computer

Zonbu Green Computer

This looks pretty cool.

Telnet now optional in Vista

Today I went to use telnet to test some server code I was working on. Except there's no telnet!?

A quick web search tells me it's not "turned on" by default, you have to go to:

Control Panel > Programs and Features > Turn Windows features on or off

Microsoft's justification for this is to decrease the "footprint" of Windows and to increase security. Neither really makes much sense. The telnet client is a tiny program and having it in a directory on the path versus some other "turned off" directory doesn't seem to change the "footprint" much. The code to turn it on and off is probably bigger than telnet itself. And I'm not sure how a telnet client is much of a security risk. Any attack that gets far enough into your system to use your telnet client is probably not going to be stopped by it being "turned off", especially since it could easily "turn it on" from the command line. (I can see leaving the telnet service turned off, but that's not what I'm talking about.)

I'm really tempted to do some Microsoft bashing at this point, but I'll restrain myself.

Ask 37signals: Is it really the number of features that matter? - (37signals)

Ask 37signals: Is it really the number of features that matter?

The idea of "editing" is interesting. It's a role I play with our application software, both in deciding which features to add, and in reviewing what we've done to try to ensure it's usable.

Of course, I still struggle with trying to pursue simplicity. Sales people, customers, customer support - they all think more features is the answer.

Chumby Coming Soon, But Not Here

Soon you'll be able to buy a Chumby, but only in the United States, not in Canada :-(

I think one of the things that helped the internet spread so fast was that it wasn't restricted like this. Although when I went looking on the internet for a tv show that I missed I found Fox limits access to shows to just the US. And Amazon's MP3 sales are US only. So the internet is not necessarily free of restrictions either.

Ubuntu on Parallels on Mac

I just installed Ubuntu on my Mac under Parallels using these instructions:

How to install Ubuntu 7.04 in OS X using Parallels Desktop 3.0

It seemed to go smoothly. Of course, Ubuntu immediately wanted to download a pile of updates, but that worked ok.

The only problem is that you seem to have to manually turn on the networking within Ubuntu every time you boot it up. The instructions mentioned this, but I thought it was just during installation. There may be a fix, I haven't looked yet. I don't think it's a big deal because normally I just suspend the virtual machines rather than shutdown and restart them. Hmmm... that's assuming the network stays turned on after being suspended, I haven't actually tested that.

Parallels has a pre-installed Ubuntu virtual machine, but the downloads were very slow so I gave up. But I should be able to take the virtual machine I've now created and copy it from my Mac Mini to my Mac Book.

Aside from curiosity, one of my motivations for installing Ubuntu is to get back to working on porting Suneido to Linux. I should be able to do a lot of that natively under OS X but I assume I'll still want to work under Linux as well.

MinGW GCC Version 4

I see MinGW (Minimalist GNU for Windows) now has a preview of GCC 4. (It looks like it came out in August, but I didn't hear about it.) I've been waiting for this for a while.

I haven't looked at it yet, but I keep hoping that MinGW GCC will close the gap with commercial compilers so I can use it for Suneido.

The Big Rewrite

ChadFowler.com The Big Rewrite

I've been involved in a number of "big rewrites" over the years. The Suneido project could be called a rewrite of a previous in-house project called C4, which in turn was a "rewrite" of a framework built on top of a personal information manager called Lucid. Our accounting applications are also the third rewrite. So it is possible to pull it off. And I would bet some of Chad's projects have succeeded in the end. But his points about the dangers and problems of big rewrites are definitely valid.

Joel Spolsky calls the big rewrite "the single worst strategic mistake that any software company can make"

In the end it comes down to Software is Hard

Digital Music Not Quite Here Again

When Apple announced DRM-free music on the iTunes store I thought I'd finally be able to buy music without the roundabout system of buying a physical cd and ripping it to get the digital version which is all I use these days. But unfortunately, their selection of DRM free music is small and doesn't seem to be growing very fast. It seems like every time I go to look for something it isn't available.

So when Amazon announced DRM-free MP3's I was hopeful but knew the real test would be selection. I checked for a few things and although not everything was available it seemed better than iTunes (and cheaper, at least for some things).

BUT when I actually went to buy something I found out it's United States only. Argh!

I found someone on the web saying they had managed to purchase from Canada by giving a fake address but that didn't work for me. Maybe because I was logged in and therefore it knew where I was.

Note: My reasons for wanting DRM-free music have nothing to do with piracy. I just don't want the headaches of DRM, especially when I use multiple computers and music players. It seems crazy how much resistance the music companies are putting up against DRM-free sales, when they've been selling DRM music (i.e. cd's) all along.

Patience, grasshopper.

Problems Embedding Google Maps in Blogger

I decided to add embedded Google Maps showing routes to some of my recent Sustainable Adventure posts but I had some problems.

First, it's quite hard to get the zoom/scale to come up properly on the embedded map. It would either be zoomed out too far or zoomed in too far. Presumably the zoom in the embedded map is related to the zoom when you ask for the link/embed code, but there doesn't seem to be a simple correlation. With trial and error, zooming in and out and resizing the window, I could usually get it more or less right.

In some cases it seemed to help to copy the link address, open a new tab/window and go to that address. I'm not sure why that would help. Maybe it was just coincidence that it happened to work after doing that.

One of my maps refused to show the route. The map itself would display (although missing one tile). Eventually, after playing with it and making a minor change to the route it started working. (I gave up on trying to get the zoom right on this one, I was happy enough to just get it to work.)

It's sad how much the success of an "expert user" comes down to trial and error and randomly poking things. The equivalent of kicking the machine. Isn't software supposed to be consistent and predictable? The problem is that if the software gets complex enough (as most software is this days) then it becomes impossible to control/know all the inputs and state, so it ends up seeming random, unpredictable, and inconsistent. Yuck!

I searched for other people having this problem but I didn't find much. Is it something I'm doing different? Issues related to Blogger? But lots of people use Blogger. Maybe I just didn't hit on the right search terms.

More Mac & Parallels Printing Problems

This is a continuation of the saga I've posted about before.

Recap: I had moved my Epson R1800 printer to Firewire to free up the USB port. But then I found out Parallels doesn't virtualize Firewire.

So I moved the printer back to USB. Things seem ok. Some time later I go to print from Lightroom and I find that I'm back to the CUPS+Gutenprint printer driver which doesn't support all the printer features. I'm not sure how that happened since before the USB->Firewire switch it was using the Epson driver.

More time passes and I go to print from Windows under Parallels. The print job goes into the queue and gets stuck. I can't even delete the print job or the printer. I try disconnecting and reconnecting the USB port to the Parallels VM. I try rebooting Windows. I try rebooting the Mac. At some point during this thrashing my print job comes out and the printer deletes. I recreate the printer, thinking that it's now going to work, but my test gets stuck in the queue as before. I thrash some more, but can't seem to hit on whatever combination it was that released that first print job.

I search on the web and find various discussions of various problems more or less related to mine. One person says it takes 5 minutes for his print jobs to emerge. Maybe this was what happened with me - not anything I did while thrashing, just the amount of time I thrashed. It seems bizarre that it would take 5 minutes. What is it doing all that time? And why does it work after whatever it is doing?

I had gotten an error message about the Epson Status Monitor 3 (why 3?) and some of the discussions mentioned this. I tried disabling it and killing it etc. but it didn't seem to make much difference.

The strange thing is that I could swear I had the printer working from Windows via USB before I switched to Firewire. So why doesn't it work now? Changes to Parallels? Who knows.

Eventually I found Finally got Epson Photo R800 printer working in XP VM on the Parallels forums. (an R800 is the narrow carriage version of my R1800) It gave instructions on how to share the printer from OS X and then access it using Bonjour on Windows. I had run across references to this before but they all talked about using a generic postscript driver on Windows which is not what I wanted. I needed to use the Epson driver to access the printers features. But these instructions used the Epson driver.

The instructions were for the printer connected by USB but they mentioned they would likely work with Firewire. Since that's what I really wanted I thought I'd try it. Nope, couldn't get it to work. (no URI for Firewire device?) Back to USB. This time it worked.

The only place where I had problems with the instructions was with choosing the printer type. Bonjour only showed the generic postscript option. I had to choose Have Disk and then find and select my Epson driver.

Several hours later, I think I have the correct, functioning printer setup in both OS X and Windows. And even better, I think I should now be able to use the same method to access the printer from other networked Windows machines.

A few days ago someone suggested buying a MacBook for someone else. I said it probably wasn't a good idea because they'd want to run Windows programs. They said "I thought you could do that now?". Yeah, well, you can, but ... it can get ugly.

School

Paul Graham's latest post, News from the Front, was irrationally comforting.

I've often wondered if I should have taken a high school biology teacher's advice and applied to somewhere like MIT. My life obviously would have been very different, but also, obviously, it wouldn't have made me any smarter. Heck, I didn't even finish university, and I don't think that has hurt me any either. (Aside from my father's disappointment, and even he came around.)

On the other hand, I might have met different people by being somewhere more "active" than the middle of nowhere in the Saskatchewan prairies.

I don't have any regrets, but it's interesting to wonder about.

Holding a Program in One's Head

Another good essay by Paul Graham:

Holding a Program in One's Head

I'm not sure I totally agree with this part:

Don't have multiple people editing the same piece of code. You never understand other people's code as well as your own. No matter how thoroughly you've read it, you've only read it, not written it. So if a piece of code is written by multiple authors, none of them understand it as well as a single author would.
...
If you want to put several people to work on a project, divide it into components and give each to one person.

I agree you can never understand someone else's code as well as your own. Initially, this is probably good advice, although, even initially, pair programming can be worthwhile. But in the long run, multiple people are going to need to work on the same code. Of course, by this stage it's not in anyone's head anymore anyway.

Web Mail

I'm surprised that so many people put up with the annoying ads inserted into their email by Yahoo Mail and Microsoft Hotmail. When I started using Gmail I never really thought about this issue, but now I'm very thankful that Google doesn't do this kind of advertising.

I guess if you're just using your email for unimportant stuff it's not a big deal. But when you're using it to apply for jobs, or for your work, do you really want stuff like this on your messages?

Moody friends. Drama queens. Your life? Nope! - their life, your story. Play Sims Stories at Yahoo! Games.

or:

Windows Live Hotmail. Even hotter than before. Get a better look now.

I would guess one reason people don't object is that they don't see it when they're sending a message.

I realize this kind of "viral" marketing can be effective, and when Hotmail first did it, it was a clever thing to do. But nowadays, I just find it offensive. Of course, I find TV commercials offensive too and that won't stop them doing it.

In any case, I'm glad Google doesn't do it. They do some "viral" marketing by suggesting you invite people to join Gmail, but it's fairly unobtrusive. And the ads they show beside Gmail are easy to ignore as well. (And fairly easy to block if you really want to.) And these things are only shown to me, the person getting the free service, rather than to people I'm sending messages to, who shouldn't have to "pay" for my service by viewing ads!

Interestingly, Google, who made a big splash by offering 1 gb of Gmail storage when most people were offering measly amounts like 10mb, is now offering the least amount of storage - "only" 2.8 gb compared to unlimited for Yahoo and AOL, and 5 gb for Hotmail. more...

Jing

This looks interesting:

Jing Project: Visual conversation starts here. Mac or Windows.

Watch the video tour for a quick introduction.

It's from Techsmith who make Snagit and Camtasia - products we use.

Parallels Annoyance

I went to print from Windows running under Parallels on my Mac and it didn't work, although it had in the past. I tried the usual highly intelligent :-) problem solving technique of rebooting (in this case both Windows and the Mac) but it didn't help.

Eventually I remembered that I had switched the printer from USB to Firewire, since the printer (Epson R1800) had come with a Firewire cable and it freed up a USB port.

But ... Parallels doesn't virtualize Firewire, only USB. There are several workarounds but none of them are great since they don't let me use the specific features of the printer.

Now something else made sense. I had wondered why I had an HP postscript printer set up in Windows. I hadn't set it up myself so I assumed some application I had installed had done it, although that didn't make much sense. Now I realize Parallels must have created it as a way to print to Mac printers.

Another workaround is to print to PDF on Windows and then print the PDF from OS X - a bit tedious, but it works.

I may end up moving the printer back to USB, but that'll mean messing with the setup on OS X which I'd rather not have to do.

The reason I wanted to print from Windows was because I was using Canvas, a program that combines many of the capabilities of Photoshop and Illustrator (i.e. both vector and bitmap at the same time). I really like Canvas but it's been somewhat abandoned. The product was bought by ACD who haven't done much with it. There was/is a Mac version but not for Intel, so I'm still using the Windows version. There are rumors of a new Windows version but not Mac. Too bad.

Windows Live Writer

 I'm writing this using the Windows Live Writer beta. Tim Bray has mentioned it a few times in his ongoing blog.

I haven't really felt the need for a separate tool to write blog posts - doing it in the browser seems to work well enough, especially with the real-time spell check in Firefox. But I can't resist trying out new free software :-)

I added this screen shot to test inserting images. That's one area where Blogger can be a bit awkward.

[screenshot removed to allow publishing]

It also looks like you can add tables which I don't think Blogger allows (unless you edit the HTML directly).

Humorously, Live Writer shows "Firefox" as a spelling mistake. (but "Blogger" is ok)

So much for images. When I tried to publish it said "the weblog does not support image publishing". Blogger certainly allows images, so I would interpret that to mean that Live Writer's interface to Blogger doesn't handle images. It did offer to let me configure an FTP site but I wonder how "user friendly" that is for most people. I assume if you use Microsoft Live Spaces for your blog then you can publish images.

Alarming Development : Beautiful Code

Sadly, I would have to agree with this:

Alarming Development : Beautiful Code

But that won't stop me from reading the book, we can still dream of beautiful code :-)