Thursday, May 29, 2008

Benchmarks

One of the optimizations I saw mentioned for JVM languages is pre-creating small integer objects, e.g. from -128 to +128.

This seemed like an easy thing to try. So I made a simple benchmark and timed it both ways.

The problem was, there was no speed difference. There was as much variation between runs as there was between methods.

The problem with micro-benchmarks is that they generally don't do any "real" work, so if you're not careful the compiler/vm will optimize away the code you're trying to test. I figured that must be what was happening here.

Sure enough, I re-wrote the benchmark so it wouldn't be so "obvious" my code was useless, and then I got more reasonable results. (There was less variation, more consistent times as well for some reason.)

Without the optimization it took an average of 9.7 seconds. With the optimization it was 6.9 seconds. This is roughly a 30% speed improvement for a few lines of code. Not bad. Of course that's if you're doing 100% small integer operations which is never the case.

PS. I know - premature optimization. But at least I measured it! (the results anyway, if not the problem) But at this stage I'm still trying to get a feel for the JVM and this kind of experimentation helps. After all, the practicality of this whole project depends on achieving decent performance.

PPS. You won't see these (or any other) changes in Subversion until I figure out why my laptop is refusing to commit changes :-(

No comments: