Eclipse Keyboard Shortcuts

I don't tend to use a lot of keyboard shortcuts, but I use these Eclipse ones quite regularly:

• F3 - Open Declaration

• Shift + Command + G - References

• Alt + Command + M - Refactor - Extract Method

• Alt + Command + R - Refactor - Rename

• Command + F11 - Run

Even if you're only an occasional Eclipse user,  it's probably worth learning a few of the shortcuts.

Related posts:

Template Shortcuts in Eclipse

Eclipse Tip for Windows and Mac Users

Template Shortcuts in Eclipse

One of the features of the Eclipse IDE is that you can define "template" shortcuts. For example, you can expand "main" into the complete method definition. I don't use a lot of these, but they are handy once your fingers learn them.

Eclipse comes with a bunch of pre-defined templates and you can add your own. For example, I added a template that expands "sop" into "System.out.println".

Ctrl+Space brings up the choices. In theory if you select Auto Insert and there is only one matching choice it will automatically fill in the template. In practice there are usually multiple choices so you have to pick from the list. That's usually just a matter of hitting Enter (as long as your choice is the first one).

You can find the templates in Preferences - Java - Editor - Templates.

I also edited the "test" template to generate a JUnit 4 test (instead of the older style):

@Test
public void ${name}() {
    ${cursor}
}

A Faster, Cleaner Database Design for Suneido

Current design:

• entire database is kept in a single file
• the database file is accessed via memory mapping
• data records are immutable, updates add new versions
• indexes are mutable
• data records are used like a "log" for crash recovery
• indexes are not recovered, they are rebuilt

For concurrency, it would be really nice if the database was a persistent immutable data structure. But this is difficult. For example, you don't want to write a new copy of a btree index node to disk every time you update it. (At least, I'm assuming that the resulting database growth would be impractical.)

In theory Suneido happily accesses databases bigger than physical memory. In practice, performance drops off rapidly when this happens due to swapping (page faults). You would think there would be a "working set" of data that would be a small portion of the database. But users tend to run things like historical reports or searches that access large portions of the data.

The other issue is that memory mapping the database makes reads fast (once the file is mapped into memory). But writes still have to go to disk (albeit somewhat asynchronously). And the index update writes are essentially random which is the worst case for disk performance. (Maybe SSD drives would help with this?)

Which led me to think, why not just keep the database in memory. But...

• Despite the performance issues, I'm not sure it's reasonable to limit database size to physical memory.
• Loading the database into memory on startup and saving it to disk on shutdown would be slow.
• You still need to write something to disk for crash recovery.

Next iteration, why not memory map the database, write data records to disk (so they can still be used as a "log" for recovery), but keep index changes in memory and only write them out when you close the database.

This has some nice features:

• Similar start up speed (although slower shutdown)
• Less disk writes (only data, not indexes) - faster
• Disk writes are sequential  (index updates were random) - much faster
• Allows databases bigger than memory

Suneido's crash recovery rebuilds the indexes anyway, so it's not a problem that they're kept in memory. Theoretically, if indexes are now immutable you could take the last good state and roll forward from there for faster recovery.

You could also periodically update the on-disk indexes, presumably during idle time on a background thread.

This seems pretty good, but there's more. Now that index updates are not being written to disk immediately, it becomes feasible to make the indexes persistent immutable data structures.

jSuneido already sort of does this, but in a complicated round-about way. Each transaction maintains an "overlay" of updates to indexes. When the transaction commits, these are merged into the real indexes on disk and other concurrent transactions are given a copy of the old index nodes so they don't see the changes. It works, but it's ugly and error prone.

With this new scheme, each transaction has, in effect, it's own copy of the indexes. Since it's a persistent immutable data structure, updates by transactions have no effect on each other. When a transaction commits, you merge it's indexes with the master. I think this would make the code quite a bit simpler.

It's hard to say how much faster this would be. Updates would be considerably faster, but reading would be much the same. So the overall speedup would depend on the mix of updates and reads. However, in a multi-user scenario (the current target for jSuneido) if you can reduce the load, that benefits everyone, including reads.

I'm not going to drop everything and implement this, but it's an interesting idea. I'll have to mull it over some more - there may be flaws that I haven't thought of yet. It might be something to work on once jSuneido is "finished" and deployed.

As with any "bright idea" like this, I can't help wonder if I'm reinventing the wheel. I wouldn't be surprised if someone was already using this approach. Of course, that doesn't mean that there's any way to reuse their code, or even learn from them. The internet might give us access to the world's knowledge, but it's still hard to find this kind of stuff.

Related posts:
Language Connoisseur
A Java Immutable Persistent Map
A Java Immutable Persistent List
jSuneido Progress

Blogger + Picasa Tweak

When I reference photos on Picasa from my Blogger blog I use the HTML under "Link to this album" and then "Paste HTML to embed in website" (using the Edit HTML function of Blogger)

However, when people click on the result they get taken to the album page of thumbnails. Some of my less technically savvy friends don't realize or notice that they can get a slideshow so they end up clicking on the photos one at a time to look at them.

You can fix to go directly to the slideshow by changing:

?feat=embedwebsite

to:

#slideshow

 in the HTML. There are two occurrences - one for clicking on the photo and one for the caption.

You can see examples in my Sustainable Adventure blog.

Note: Picasa also has an "Embed Slideshow" option under "Link to this Album" but I don't normally use this because it only shows small images and I'd prefer to have people view my photos larger. I do use it when I'm not as concerned about image size. (You can still click on it to go to the Picasa album, but most people won't bother.)

Visibility

Now that jSuneido is more or less functional, I've been working on some of the peripheral functions like checking and rebuilding databases. (i.e. crash recovery)

The checking part went quite smoothly and quickly. The rebuilding/recovery part has been slow. I've been working on it for several days and it seems like I've made no progress. The only code I've written is tests and debugging! But whether it feels like it or not, I guess I was making progress because it "suddenly" started working today.

Recovery is hard for a number of reasons. For starters you're assuming the database has been corrupted, so you have to code a lot more defensively. Second, because you're working outside the normal operation of the database, you can't use the normal functionality. You're not doing transactions etc.

One lesson I (re)learned was how important "visibility" is. By that I mean being able to "see" the data you're working on. That's a large part of why debuggers can be valuable - they let you inspect the data. Often, inserting the right "print" statement in the "right" place is all it takes to figure out a bug. Of course, finding that right place is not so easy. In this case visibility meant writing a dump utility so I could see exactly what was inside the database. Obviously, I have a pretty good idea in general how it's structured, but not the details of exactly which types of blocks of data are in which sequence. And dumping to text files meant I could use tools like diff to compare before and after recovery.

PS. In hindsight, yes, it would have made sense to write the database checking much earlier so I could verify that databases weren't getting corrupted by operations. On the other hand, that was usually pretty obvious -  it crashed!

Readability

How often do you go to a web site to read an article and find that the page is plastered with distracting ads and other material, the text is too small, and the lines have way too many characters for comfortable reading?

Here's a solution - install the Readability bookmarklet. It will re-format the page you're on for easier reading - less distractions, bigger text, and shorter lines.

Before:

After:

Buy O'Reilly eBooks Direct

When I wrote my last post - Buy Pragmatic Programmer eBooks Direct - I should have mentioned that O'Reilly has a similar deal for their eBooks - ePub, Mobi, and PDF formats, DRM free, and free updates. Not all of their books are available as eBooks (some are print only) but most of the newer ones are.

They even have a 3 for 2 sale on right now.

Buy Pragmatic Programmer eBooks Direct

If you're buying Pragmatic Programmer books for your Kindle you have the choice of buying from Amazon or buying direct.

Buying from Amazon is the path of least resistance since you can buy right from your Kindle.

But if you do that, you'll end up with a proprietary format with DRM that only works with your Kindle. (Although you can read them on your PC or iPhone or iPod Touch with the Kindle app.)

Instead I'd recommend buying direct. That way you can download the book in multiple formats - PDF, ePub (e.g. for iPhone or iPod Touch using apps like Stanza, or for Sony readers), or Mobi (for Kindle). The books are DRM free, and as a bonus, you can download updated copies when they come out.

I've bought a few eBooks from Pragmatic Programmers in the past, either on their own or together with a paper copy. It was very nice to be able to go back and download the Mobi version so I could have them as a reference on my Kindle.

The only very minor hassle is that you have to buy and download the books on a separate computer and transfer them to the Kindle. In Canada that means by USB since emailing to your Kindle is not supported here.

Thanks to Pragmatic Programmers, at least a small part of my eBook library is not locked to the Kindle. I hope more publishers follow their example.

jSuneido Speed Issue Fixed

After some digging, I found the speed issue on Windows.

It didn't turn out to be Parallels - it was just as slow on my Windows machine at work.

It didn't turn out to be anti-virus - it was just as slow with anti-virus turned off.

I thought it might be Nagle problems again, although last time it was the opposite problem - slow on OS X and fast on Windows. But nope, not that either.

I narrowed it down to sending the response. Debugging showed it was fast receiving the request and executing it.

I found I had two methods for sending responses. One was doing selector.wakeup after changing the interestOps (like you're supposed to) and the other wasn't. Of course, the incorrect code was the method being used. I fixed the method being used and got rid of the other one. Problem solved.

Now, the speed is reasonable. 75 seconds to run the test suite with the cSuneido server, 100 seconds with jSuneido.  I'm pretty happy with that considering the cSuneido version is C++ code that has been tweaked and optimized for years, whereas jSuneido is my first Java program and has barely been debugged, let alone optimized.

I think jSuneido can probably be faster, but even if it can't, its ability to scale to multiple threads/cores should allow it to outperform cSuneido when there are multiple users. I say "should" because I've barely started testing this. One very preliminary test took about 40 seconds with one client, 50 seconds with two simultaneous clients (on two cores), whereas cSuneido would be pretty much linear i.e. 80 seconds with two clients. Of course, it would be nice to have a quad-core machine to test with, but I haven't quite convinced myself that's excuse enough to buy a new Core i7 iMac :-)

Removing Borders from Images on Blogger

This may be due to the template I'm using, or maybe even my browser, but in case other people are having the same problem, I thought I'd post my workaround. (Besides, it gives me a handy reference when I forget what to do!)

When I insert images into my blog posts I get a gray border around them. Like this:

Sometimes that's ok, but often, like with this image, I don't want that border.

NOTE: The border doesn't show up while composing, or in preview, only (annoyingly) after you publish.

To remove it, I edit the HTML and add style="border: none;" to the img tag. (I'm not sure why this is necessary because it already has border="0" - maybe something in the CSS.) The result should be the image without the gray border, like this:

Note: If you're reading this in something like Google Reader you may not see the border.

Back to Work on jSuneido

I fixed a couple of bugs and now all our application tests pass, both standalone on jSuneido, and client-server (with the Windows cSuneido client).

But quite slowly - about 4 times slower running client-server with jSuneido as the server instead of cSuneido. I'm not surprised it's slower, but 4 times seems like a lot. Strangely, when I run the tests under jSuneido standalone (i.e. not client-server) they run about the same speed as cSuneido client-server (which is still about half the speed of cSuneido standalone). So presumably the difference is the client-server networking.

I've been testing with the server running on OS X (since that's where I develop) and the client running under Parallels. Maybe the networking slows down the Java test so I decided to try running the Java server on Windows as well as the client.

First I had to remember how to package up jSuneido as a jar file. I've done this before, but not for a while. In Eclipse you use Export > Runnable Jar File.

Next, I found I didn't have Java on my current Windows 7 Parallels VM. I downloaded and installed the JRE (Java Runtime Environment).

I got some weird errors until I went back and rebuilt my jar file with "Extract required libraries into generated JAR" (instead of "Package required libraries into generated JAR"). I'm not sure exactly what the difference is, or why it makes a difference to running my code. I made a half-hearted attempt to find it in the Eclipse Help but didn't get anywhere.

Finally I got it running, but it was soooooo slow! Instead of the IDE starting more or less instantly, it took more like a minute. Ouch.

I figured it might help to run it with the -server option (that's the default on 64 bit Java, which is what I'm running on OS X). But I discovered the JRE doesn't include the server version. Grrrr. It comes with the JDK (Java Development Kit). That seems odd to me, but that appears to be the way it is.

So I installed the JDK, but the default Java that's on the path is still the JRE one. So now I have two separate installations of Java with the "wrong" one on the path. Sigh.

I ignored that problem and used the absolute path to run the JDK one to get the server version. But it's still just as slow. Argh!

The only thing I can think of is that jar files are zip files and working with zip files (or other compressed archives) has always seemed painfully slow under Parallels. I'm not sure why. Maybe decompression normally takes advantage of CPU features that aren't accessible under Parallels? Or the anti-virus program slowing it down? (I'm using Microsoft's anti-virus.)

The next step is to try it on my Windows machine at work to eliminate the Parallels issue. Or to make a simple Java client so I can try it client-server all on OS X without Windows or Parallels.

And if it's really that slow, then it's time to do some more profiling and see what I can do to speed it up.

Getting Past “Good Enough” eBooks

Getting Past “Good Enough” eBooks: Liza Daly | Digital Book World

Now I have a Kindle I'm noticing the same kind of things, but I have to shake my head - this stuff is so basic!

Another example is that section headings frequently show up at the very bottom of the page. We were fixing this problem in nroff 30 years ago!

Of course, software is no better, people keep making the same mistakes we've known were bad for decades.

My New Kindle eBook Reader

I just got an Amazon Kindle (the 6" global). Again I'm amazed by the speed of delivery these days. I ordered Friday afternoon and it was delivered Monday morning. (Despite an email that said delivery would be delayed by "extreme" weather.)

Several people have asked why I didn't wait for the Apple iPad. Of course I want an iPad. I love my iPhone and I'd love a similar device with a bigger screen. But ...

• The iPad is supposed to have 10 hours battery life - that's great for a laptop, not so good for an eBook reader. The Kindle, with it's E Ink display will last a couple of weeks. That's critical for travel, especially off the beaten path and away from power (e.g. trekking).
• The iPad isn't available yet, and it's unclear when it will be available in Canada. I could be waiting forever if I keep waiting for the next great thing (I've been waiting a while for the Kindle).
• As far as I can tell, Amazon has the best selection of books. That's what stopped me from buying the Sony eBook reader, which was available sooner in Canada, and in some ways is nicer than the Kindle. Apple hasn't been saying much about their books  - how much selection they'll have and what they'll have in Canada.

I'm a committed reader. In the past I've bought tons of books. Now I have a basement full of literally 1000's of books, most of which I'll never look at again, but which I still have a hard time parting with. My office at work has stacks of books everywhere. And most computer books are out of date and worthless after a year or two. I have nowhere to put more books!

On top of that, it doesn't seem environmentally friendly to be consuming all those trees and all the resources that go into converting them to physical books.

I've been trying to use the library more, but it's frustrating. Their software sucks. (They just upgraded to a new system and it sucks almost as bad as the old system.) They don't get new books very quickly. They don't get most computer books at all. And you never know when you'll get a book you reserve. It might be tomorrow, it might be next year. And it's no good for books you might want to refer back to. I'll continue to use the library, but mostly for fiction and general non-fiction where I don't care too much when I read it.

Sony has an interesting feature where you can borrow eBooks from your library, but only certain libraries support this (not ours) and there are various limitations. Sony also lets you lend your books to other people with Sony readers - I would really like this feature.

Kindle positives:

• nice display, very easy to read, good for reading outside in bright light
• small, light-weight
• great battery life
• ability to buy books right on the device (don't need to download to your computer and then sync like the Sony)
• ability to create bookmarks and add notes
• ability to read free content in pdf or mobi format
• much more convenient to carry around the Kindle rather than a bunch of books
• free 3G wireless (cell phone) access (but only to buy books from Amazon)

Kindle negatives:

• no back-light (Shelley loves being able to read on her iPod Touch at night)
• slow display - ok for turning pages, but not so good for user interface
• no touch screen (I'm so used to my iPhone I keep trying to tap links on the Kindle)
• no wifi
• no Wikipedia access in Canada
• no web browsing in Canada
• no pictures in magazines in Canada
• no color (black and white display)
• limited memory and no way to expand it (e.g. for an offline copy of Wikipedia)
• you have to buy books from amazon.com (US), but my wish-list and gift cards are on amazon.ca (Canada)
• too easy to buy books ;-)

How "green" is a Kindle? That's a tough question that depends on a lot of factors. Manufacturing the Kindle isn't green. And it's another device that will likely be out of date and discarded within a few years. But if I avoid buying several hundred books a year, that's a fair saving. Of course, the library is the greenest option, but not a very functional one.

I debated whether to buy the small or large (DX) Kindle. The large one would be a lot better for reading PDF's that you can't reformat, but I went with the smaller one for ease of carrying.

It's too bad there isn't a better display that combines the battery life and readability of E Ink with the color and speed of a laptop display. The One Laptop Per Child computer had an interesting display with a low power black and white mode (like E Ink) but also a color mode. But no one else seems to be pursuing this kind of display.

The other big negative to all this is that the books you buy are tied to a specific device/company. If I buy books for my Kindle I can't switch to the Sony reader and vice versa. This is mostly because of the DRM copy protection. Imagine if music was tied to a specific player instead of being able to play your mp3's anywhere. (However, Amazon has software that lets you access your Kindle books from your PC, Mac, iPhone, or iPod Touch.)

Thanks to the Weave Team!

I just installed the 1.0 release of Mozilla Weave and I was excited to see this in the Release Notes:

If you use a master password, Weave Sync will automatically connect after you enter in your master password. Weave Sync will stay disconnected until you enter your master password or you choose to manually connect.

It drove me crazy to have the request for the master password come up every time I started Firefox, and it totally baffled anyone else using my computer (you can just cancel, but they would think they couldn't use Firefox without the password)

I had actually disabled Weave on most of my computers because of this. But that was a pain because then my passwords weren't synced. Now I can re-enable it.

Now if only it would just sync my add-ons and work with my iPhone ...(users are never satisfied!)

The Magic Mouse

I finally got the new Apple Magic Mouse. It came out before I went on holidays in mid December, but no one had them in stock. As you'd expect, it's very slick looking.

So far I'm pretty happy with it. My biggest complaint about the previous Apple Mighty Mouse was that the right click was really erratic. Not a big deal on OS X since it doesn't use right click too much, but a major issue when running Windows (under Parallels). Right click on the Magic Mouse seems much better. The other less serious, problem with the Mighty Mouse was that the scroll ball would get dirty and quit working, but it became second nature to clean it (by running it around on your hand or pants).

My only complaint is with the left and right swiping for horizontal scrolling - it's a little too easy to trigger by mistake. Again, that doesn't matter most of the time because there's nothing to horizontally scroll. But in Adobe Lightroom I keep switching images by mistake. And it's touchy enough that it zips through half a dozen images, making it a nuisance to get back to the one you were working on.

I tried installing Better Touch, a free program that lets you tweak all kinds of Magic Mouse and touchpad stuff. But for some reason it doesn't seem to work with Lightroom. Maybe Lightroom is using a different interface to the mouse. There doesn't seem to be any way to adjust the behavior in Lightroom either. I'd be happy just to turn off horizontal scrolling by left and right swiping - it's not something I need a lot.

I'm still getting used to the scrolling with momentum. It's a bit like the iPhone/Touch. I've been catching up in Google Reader and it seems to work well there.

Another minor issue was the lack of a middle button. On the Mighty Mouse you could click on the scroll ball to get a middle click. I had this configured to trigger Expose since I had the function keys set to be function keys (for Suneido on Windows on Parallels). Better Touch solves this since it lets you add a middle click.

Why Git is so fast (and JGit isnt)

Some interesting comments on performance in Java

'Re: Why Git is so fast (was: Re: Eric Sink's blog - notes on git,' - MARC

More jSuneido Slogging

Another long tedious day working on getting more of our application tests to run on jSuneido.

These are large scale functional tests, so when one fails it's not easy to figure out why. It could take 5 minutes, it could take 5 hours.

I've found a couple of small bugs in jSuneido. One was because BigDecimal differentiates 1 from 1.0 from 1.00, which makes sense from a scientific precision viewpoint, but not when you're dealing with money. And the problem was actually even more obscure - it was because it differentiates 0 from .0 from .00

But the rest of the bugs (the majority) have been in our application code, either in the tests or in the code itself. Nothing serious, most of them were inadvertent dependencies on the order of unordered things.

But it's frustrating. It would be tedious enough doing all this testing to find bugs in jSuneido. But when I'm doing it to find other people's bugs it's annoying. And of course, as with any large body of code, a lot of it is confusing, hard to understand, and could be improved. (Don't get me wrong, I tend to think the same about my own code.)

Oh well, it's got to be done. Hopefully it doesn't take me too much longer.

Systems that Never Stop

An interesting (and entertaining) talk by Joe Armstrong (the principal inventor of Erlang) about writing fault tolerant systems. Well worth watching.

InfoQ: Systems that Never Stop (and Erlang)

A Debuggers Life

Another day of debugging, although with a twist - I found as many bugs in our application code as I did in jSuneido. Just minor stuff - there's nothing like multiple implementations of a language to flush out the edge cases.

It seems like a slow process, but the jSuneido bugs do seem to be getting smaller and more obscure, which gives me a certain amount of confidence that the main stuff is ok. Most stuff just works, which is a vast improvement over not long ago.

Software That Fixes Itself

Technology Review: Software That Fixes Itself

Cool but a little scary - will the software start to evolve?