Using ProGuard on jSuneido

ProGuard is a free Java class file shrinker, optimizer, obfuscator, and preverifier.

I'd seen a few mentions of ProGuard and then when I was updating to the latest version of the Guava library I saw that they were recommending it. So I decided to try using it on jSuneido.

It shrunk the size of the jSuneido jar from 2.5 mb to 1.4 mb.  That's pretty good for an hour or two of setup.

I haven't tried the optimization yet, partly because the Guava instructions disabled it. I'm not sure why.

ProGuard also does preverification which should improve startup time, although that's not a big issue for the server usage that jSuneido is aimed at.

It took some trial and error to get a configuration file that worked. I'm not sure it's optimal yet. I still get some warnings from Guava (mentioned in their instructions) and from jUnit, but they don't appear to cause any problems.

Eclipse += Mercurial

It wasn't quite as easy as I would have liked to get the MercurialEclipse plugin working, but it certainly could have been worse.

It was easy to install the Eclipse plugin using the update site: http://cbes.javaforge.com/update

But when I tried to import my project from Mercurial I got a bunch of errors about passwords.

I found a post from someone with the same problem, with a pointer to instructions on how to solve it.

I guess the simple username + password doesn't work with MercurialEclipse, at least on OS X. Sigh.

The answer is to use an SSH key, which I'd avoided till now, but it turned out to be straightforward with the help of the SourceForge instructions. It sounds like it might be a bit trickier on my Windows box - I haven't tried that yet.

I tweaked a few things for the new setup, committed those changes, and pushed the changes to the SourceForge repository. Seems like I'm good to go :-)

The Revenge of the Intuitive

Interesting article by Brian Eno on the user experience problem of too many options, a problem we struggle with all the time with our software. The challenge is that it is the users themselves that keep asking for more options, despite the fact that it ends up making the software harder to use.

Wired 7.01: The Revenge of the Intuitive

Moving jSuneido from Subversion to Mercurial

I've been planning on moving from Subversion to a distributed version control system for a while. Initially I assumed it would be Git since that's what I heard the most about. But I recently listened to a podcast with Eric Sink where he said Mercurial had better Windows support and was simpler to use, especially for Subversion users. An article on InfoQ confirmed this. And Google code has Subversion and Mercurial but not Git. And Joel Spolsky and Fog Creek had picked Mercurial. Joel even wrote a tutorial for Mercurial. So I decided to give Mercurial a try.

When explaining distributed version control a lot of people start by saying there's no central repository, it's peer to peer. This throws off a lot of people because they want a central repository. It's the explanation that's wrong. You can have a central repository, and most people do. The difference is that it's a matter of convention, it's not dictated by the software. And you can also have multiple repositories.

For me, the advantages would be having the complete history locally, even when I'm off-line, and being able to easily branch and merge locally.

Here are the steps I used to convert jSuneido from Subversion to Mercurial. (I haven't converted C Suneido yet, but that shouldn't be too hard, just more history so it'll be slower.)

Download and install Mercurial on OS X from mercurial.selenic.com

Convert the existing SourceForge Subversion repository to a local Mercurial repository:

hg convert http://suneido.svn.sourceforge.net/svnroot/suneido/jsuneido

Ideally you wouldn't convert directly across the network. Instead you'd clone your Subversion repository to your local machine and convert from there. That's primarily so if you have to tweak your conversion and re-run it, you're not dealing with the network delays multiple times.

I was lucky since my Subversion history did not have anything tricky like branches to deal with. And cloning the Subversion repository looked painful. I figured I could probably get away with doing the convert directly. It ended up taking a couple of hours to run.

One nice thing about convert is that you can run it again to pick up incremental changes. (I had to do this once because I had forgotten to send my latest change to Subversion.)

To check out the result I copied the repository to Windows (under Parallels) and installed TortoiseHg. The repository looked reasonable. (A little roundabout, but I wanted TortoiseHg anyway.) TortoiseHg seems to work well.

Enable Mercurial for the Suneido SourceForge project. Add a second repository for jsuneido. See sourceforge.net/apps/trac/sourceforge/wiki/Mercurial

Push the local Mercurial repository to SourceForge.

hg push ssh://amckinlay@suneido.hg.sourceforge.net/hgroot/suneido/jsuneido

I could have also done this with TortoiseHg (but don't look for a "Push" menu option, it's under Synchronize)

I can now browse the repository on SourceForge.

I found the series of three blog posts starting with From Subversion to Mercurial. Part 1, Setting the Stage quite helpful.

Next I have to figure out the Eclipse plugin for Mercurial. I hope it's less hassle than the Subversion one!

Stylebot

Changing the Look of the Web with Stylebot - Google Open Source Blog

A very nice Chrome extension that lets you easily create custom stylesheets for web sites. For example, I often want a different text size for a given site, or to hide certain annoying elements (e.g. ads!)

This is a good example of how Chrome seems to be pulling ahead in the browser competition.

Balsamiq Mockups

I've heard several people say good things about Balsamiq Mockups. I haven't used it myself but I'm planning to give it a try. I like how it produces something that looks like a sketch so people don't get the mistaken idea that it's a finished design.

What Can't the iPad Do?

The girl next to me on the bus had an iPad, so I asked her how she liked it. I expected the usual "it's great", but instead I got "I don't" I asked her why and she said there was too much stuff it couldn't do. I didn't get a chance to dig deeper.

Someone else I know bought an iPad and immediately got someone to jailbreak it for them. Why? Because there's too much stuff it can't do otherwise. When I asked for examples they couldn't really give me any. This is a non-technical person, it's not that they wanted to do anything special.

Obviously, there are things that an iPad is not ideal for. And there are things that an iPad can't do. But there are a huge number of apps and for most people, I just can't see what it "can't" do. All most people do is email and Internet anyway.

It can't run big programs like Photoshop, but most people don't need that. You can't run Microsoft Office, but you can get word processing and spreadsheets.

The touchscreen keyboard is not great for typing a lot, but most people don't type a lot. And if you really need it you can use a Bluetooth keyboard. Most people are happy typing on their cellphone! (I manage to type quite long blog posts only iPhone.)

I would guess one of the reasons for this idea is that people get overinflated expectations from all the hype. And when it doesn't (can't) live up to them, they have to blame it on something.

Which is where the critics come in. People don't remember the specific criticisms like the lack of multi-tasking or cameras or memory cards. All they remember is that "there's stuff it can't do".

And there's always the most common reason people say you can't do something - simply because they don't know how to do it. We get that all the time with our software. You ask people how they like the software and they say it's ok but they really wish it could do xyz. Half the time it already can, they just didn't know it.

iPhone Undo/Redo

I've was getting ready to write a blog post asking why the iPhone doesn't have undo/redo. Selecting text is tricky and it's easy to delete more than you wanted and then have no way to get it back. But when I did a quick search before writing the post, I found it does have it!

If you shake your iPhone you get an undo/redo popup. It appears you can even undo multiple steps, at least in the Notes app.

I'm surprised I didn't run across some mention of this before now. I'm not surprised I didn't discover it on my own. I don't tend to stop and shake my phone in the middle of typing! Maybe the idea is that you get so frustrated you shake your phone?

Honestly, I think this is poor design. First, it's not discoverable. And when people, even experienced ones, don't discover a feature they're probably going to assume it doesn't exist. Second, when I'm typing I generally hold the phone steady in my left hand and type with my right. Shaking is not a natural action, it's like having to take your hands off the keyboard.

Wouldn't it make sense to put an undo key on the keyboard? It could be on the secondary punctuation layout. At least then it would be discoverable. You could still keep the shake interface - presumably someone at Apple thinks it's great.

Connecting the Dots

Sometimes I'm a little slow :-( 

One of the talks at the JVM conference was about a JVM for mobile devices. It used a single pass compiler with no intermediate representation. Suneido also does this but there are some issues with it. Suneido takes the obvious approach of generating a push when it encounters an operand, and an operation when it encounters an operator (after processing its operands). This works great with simple expressions like "a + b" which compiles to "push a, push b, add". But it falls down for more complex things, even for "a = b" since you don't want to push the value of "a". I work around these issues but the code is ugly. And worse, a lot of the ugliness is in the parser, even though the issue is really with code generation.

I wasn't happy with it but, consciously or not, I assumed it was essential rather than accidental complexity. So I didn't look hard enough for a better solution. . 

So I was curious to hear more about their approach. The topic only got one bullet point, something like "code generation by abstract interpretation". 

I started thinking about what that meant. I'm pretty sure what they meant is that you only generate code when you encounter an operator. When you encounter an operand like a literal or variable, you push it on a compile time stack, "simulating" execution. Then you can defer generation of code until you know what is needed.

As I thought about this, I realized I knew about this technique, I'd read about it before. But for some reason I never connected it to my code generation issues.

So when I got home from the conferences I spent three days refactoring the code generation in jSuneido. It wasn't quite as straightforward as I imagined (it never is!). The parser code is definitely cleaner but I had to add more code to the code generator than I would have liked. Overall, I think it's better. I broke a lot of tests in the process, but I finally got them all passing again last night. 

Now I can quite easily add a number of optimizations that I've been wanting to do:

- constant folding

- call private methods directly (no method lookup required)

- call built-in global functions/classes directly

However, there are still things I'd like to do that are difficult with a single pass compile. For example, knowing which local variables are read or written by closures. One way to handle this would be to re-compile if the assumptions were wrong, using information from the first try to do it correctly the second time. But that seems ugly. Maybe I need to drop the single pass compile and just generate an AST. 

JVM Languages Summit Day 2

Another good day of talks starting with Doug Lea on parallel recursive decomposition using fork-join. Amazing how much subtle tweaking is require to get good performance.

This led into Joshua Bloch's talk on performance. There is so much complexity in all the layers from CPU to OS to language that performance varies a lot. He showed a simple example of a Java program that gave consistent times on multiple runs within a given JVM, but sometimes when you restarted the JVM it would consistently give quite different results! Cliff Click's theory was that it was caused by non-deterministic behavior of the JIT compiler since it runs concurrently. The behavior is still "correct", it can just settle into different states. The solution? Run tests over multiple (eg. 40) different JVM instances. That's on any given machine, of course you should also test on different CPU's and different numbers of cores. Easy for them to say. 

Neal Gafter talked about Microsoft's LINQ technology - pretty cool, although nothing to do with the JVM. 

Kresten Thorup talked about his Erlang implementation on the JVM using Kilim for coroutines. Erlang is an interesting language, and quite different from Java so it was interesting to see how he implemented it. He actually runs the byte code produced by the existing Erlang compiler. 

I talked to Remi Forax about whether I should use his JSR292 backport in jSuneido. This would let me use the new technology before Java 7 is released (who knows when). Of course, he said I should. But ... it means developing with the "beta" JDK 7 which still has bugs and is not supported by IDE's. And then it requires an extra run-time agent. I'm not sure I want to complicate my life that much!

JVM Languages Summit Day 1

My hotel is "behind" the Sun/Oracle campus so I had to circle around through the endless acres of parking lots to get to the right entrance. But it was pretty easy, and not as far as it looked on the map. I've never had much to do with giant organizations so it's still a little mind boggling when you use an automated kiosk (like the ones at the airport) to get your visitors badge.  


The sessions were a real mixture, from stuff where the nuances were pretty much over my head, to thinly veiled sales pitches with no technical details. It was pretty neat to be in the company of people you're used to thinking of as gurus, like Doug Lea, John Rose, Joshua Bloch, and Cliff Click. And nice to see they have their frustrations with the technology also.


It's naive of me, but unconsciously I expect really smart people to be "rational", and it's always a bit of a disappointment when that proves to be untrue. Smart people have egos, are insecure, or argumentative, or negative, or defensive, or obnoxious, just like everyone else. 


Maybe I'm just too soft, but I felt bad for one guy who basically got told he was doing it wrong. It seems like they could just have well asked "did you consider ..." or "what would you think about ...", rather than just "that's wrong, you should have done ..."


Mostly I just listened to the conversations. In the Java / JVM area I still feel like a relative novice and I'm not sure I have much to contribute yet. But for the most part I didn't feel too much out of my depth, so that's good.

OSCON Wrapup

The last two days of OSCON were good.

I got to hear Rob Pike talk about the Go language. Rob is a legend in software and Go is a cool language. In some ways Go is more like C or C++ in that it's compiled to machine code with no VM. But unlike C++ it compiles extremely fast. For what he called a party trick he showed it compiling every time he typed a character - and it kept up. It has features I miss in Java like pointers (but safe) and values on the stack (not always allocated). It also has "goroutines" - lightweight threads like coroutines. But its attractions aren't quite sufficient to tempt me away from Java. They don't even have a Windows version yet, let alone all the support libraries and frameworks that Java has.

I also got to hear another legend, Walter Bright, talk about his D language. I used Walter's C and C++ compilers for many years. D also has some very interesting features. Andrei Alexandrescu of Modern C++ Design fame is now working on D and has written The D Programming Language book.

One feature D has that is sorely missing from other languages is the ability to declare functions as "pure" (ie. no side effects) and have the compiler verify it. This (not lambdas) is the key to functional programming. And yet languages like Scala that claim to support functional programming don't have this.

I also went to a talk by Tim Bray whose blog I read. He was working at Sun but moved to Google after the Oracle buyout. His talk was on concurrency. He was very pro Erlang and Clojure but didn't mention Scala. When asked about it he said he thought the Scala language was too big. It does have a sophisticated type system, but the actual syntax is quite simple - smaller than Java. Scala's Actor implementation has been criticized but it's been improved in 2.8 and there are alternatives like Akka.

OSCON Another Day

After the less than thrilling key notes, I went to another Scala session. A lot of it was repetition, but I learn a bit more at each one. The recurring theme seems to be if you're going to use Scala, use Simple Build Tool (SBT).

Next I went to a talk on GPARS, a Groovy concurrency library. The talk was as much about concurrent programming patterns as about Groovy, which suited me.

Next, I was going to go to one of the database talks, but then I realized it was a vendor presentation and they're usually more sales spiel than technical. Looking for an alternative I noticed Robert (R0ml) Lefkowitz had a talk on Competition versus Collaboration. If you've never listened to one of his talks, it's well worth it. They are as much performances as presentations and always thought provoking. (search for Lefkowitz on the Conversations Network)

Next was a talk on Clojure (a Lisp that runs on the JVM). The title was "Practical Clojure Programming" which sounds like an intro, but it was actually about build tools, IDE's, and deployment. Like most of the audience, I would have rather heard more about Clojure itself. I guess we should have read the description closer.

Finally, I snuck into the Emerging Languages Camp to hear Charles Nutter talk about Myrah (formerly Duby) his close-to-the-JVM-but-like-Ruby language. I would have been tempted to go to the Emerging Languages Camp (it was even free) but by the time they announced it, I'd already signed up for the regular sessions.

All in all, a pretty good day. Lots of food for thought, which is, of course, the point.

One thing I forgot to mention yesterday is that a lot of the Scala people are also (or ex-) Ruby/Rails people.  Maybe that's simply because they're the kind of people that like to learn/adopt new things.

But a lot of people left Java and went to Ruby, so it's surprising to see them coming almost full circle back to Scala. Scala is better than Java, but it's still a statically typed language like Java, which is part of what people seemed to reject. Maybe Ruby wasn't the silver bullet they were hoping. Maybe it was performance issues. Maybe they realized that static typing does have advantages after all. Maybe they realized the advantages of running on the JVM (although jRuby allows that). Maybe Scala's improvements over Java are enough to win people back.

OSCON Part One - Scala

I just finished the first day and a half of OSCON - in the Scala Summit. And contrary to my anti-social nature, I even went to a Scala meetup last night.

As I've written before, I'm quite intrigued with the Scala programming language. It runs on the JVM and interfaces well with Java. It combines functional and object-oriented programming. It has actors for concurrency. And it has a rich type system that is probably Turing complete, like C++ templates. 

Scala seems to be attracting a lot of attention. Of course, the Scala Summit attendees are all keen, but there were also several other language developers (like Charles Nutter of jRuby) interested in borrowing ideas from Scala.

Programming in Java is like driving a truck. It's not fast or fuel efficient or fun to drive or comfortable. But it gets the job done without a lot of problems and it can haul just about anything. 

Why do people like to drive a fast car? It's not like they regularly need go from 0 to 60 in 5 seconds, or hit 200 km/hr. But they like to know that if they "need" to, they could.

Similarly, I think people like to know their language "has a lot of power under the hood", even if they never use it. And it gives lots of potential for books and conference talks that keep people excited.

And like you could use C++ simply as a better C, you can use Scala as a better Java. This lets less sophisticated users get involved as well. 

The funny part is that it's very reminiscent of C++.  There were all kinds of talks and articles and books about all the cool stuff you could do with templates and other fancy C++ features. Now, all you hear are negative comments about C++, too complicated, too hard to use, etc. I start to think I'm the only person that liked C++. 

But there are differences too. Scala didn't try to be backwards compatible with Java the way C++ was backwards compatible with C. And Java already had object-oriented programming, so Scala isn't as big a jump as C++ (in that respect). 

If nothing else, it's made me really want to spend some more time with Scala. It will sound crazy, but I'm very tempted to rewrite jSuneido in Scala. Thankfully, that's something that could be done gradually. I think it would let me make the code much smaller and cleaner. I think it would also make it easier to switch back and forth with Suneido coding, since Scala has things like named and default arguments, optional semicolons, and type inference that make the code much more similar to Suneido code.

Time Capsules

I use an Apple Time Capsule for my home wireless router and backup storage.

Time Capsules have a bad reputation for dying and I've had mine for quite a few years so I was a little nervous about it. If it died I wouldn't have a backup of my iMac. Which would be ok unless it happened to die at the same time. This seems unlikely, but it's surprising how often you do get simultaneous failures. For example, a power surge due to lightning. I can also keep the external drive at work so I have an offsite backup in case the house burns down.

I had a 500gb external drive that I'd used for backups, but it's not big enough to do a complete backup of my 1tb iMac. So I went and bought a Lacie 2tb external drive and used SuperDuper to make a backup. I used the free version, but I'll probably get the paid version so I can update the backup without redoing it.

Then I decided I should also backup my MacBook and my MacMini. I didn't have much critical stuff on them, but a backup would save hassle if I needed to restore. But SuperDuper takes over the whole drive so how could I backup additional machines? The answer seemed to be to partition the drive, but I didn't want to have to redo my iMac backup (600gb takes a while to backup, even with Firewire 800). I searched on the web and found various complicated ways to resize partitions. Finally, I found that with recent OS X you can resize right from the Disk Utility. All the complicated instructions were for older versions of OS X.

The "funny" part of this story is that a few days later I went to use wireless and it wasn't working. I went and checked on the Time Capsule and it was turned off. Strange, because I leave it running all the time. I turned it on and about 10 seconds later it turned itself back off.

My computer was still fine, so I didn't actually need the external backup, but I was glad to have it nonetheless.

I phoned the local Apple dealer (Neural Net). The receptionist wanted me to bring it in and they would look at it in the next few days. I didn't want to be without internet for days so she let me speak to the technician. When I described the symptoms he said the power supply had died. But Apple doesn't let them repair them and doesn't supply any parts. Apple has been promoting their environmentally friendly products, but no matter how they're built, a non-repairable "disposable" product isn't very environmentally friendly.

I would have preferred to give Neural Net the business but they didn't have any Time Capsules in stock so I picked up a 2tb Time Capsule from Future Shop. 10 minutes after opening the box I was back up and running. (Although the initial Time Machine backups took considerably longer.)

One nice thing is that Time Machine backups seem to be a lot less intrusive. Before, when Time Machine kicked in it would really bog down my machine. If I was trying to work I'd often stop it. But now I don't even notice when it runs. I'm not sure how a new external device would change the load on my computer, but it's nice anyway.

My old Time Capsule ran quite hot, even when it wasn't doing anything. I was hoping the new ones would be better, but it seems much the same. I haven't measured it, but I assume heat means power consumption. I'm not sure why it can't go into a low power mode when it's not active. The other reason they run hot is that they have no ventilation or heat sink. Apparently there is an internal fan but all it does is stir the air around inside a small sealed box. You'd think they could come up with some better heat management without compromising their design. I would guess the heat is one of the big reasons they have a reputation for dying. Electronic components tend to have a shorter life at higher temperatures.

Rather than throwing out the old Time Capsule, I passed it on to one of the guys at work that tinkers with hardware. I thought he could at least extract the 1tb drive. But he managed to repair the power supply and is using the whole unit. I guess the hardest part was getting the case open! I'm glad it was saved from the landfill for a while longer.

Now that I have a bigger drive I thought I might as well backup Shelley's Windows machine as well. I have it set up with Mozy for on-line backups but just with the free 2gb account so I'm only backing up selected things. I'd seen Mozy will now backup to external drives so I thought I'd set this up. Unfortunately, it only backs up to directly attached drives (i.e. internal or USB) not to network drives. I'm not sure what the logic is behind that choice. I could use different software, but I think what I'll do (I haven't got round to it yet) is to use the old 500gb external drive.

iPhone Multi-tasking

A lot of people made a big thing about how the iPhone didn't have multi-tasking. But personally, I never saw any of the reasons as very compelling. Given limited cpu power and memory and a tiny screen, it made perfect sense to single task.

Sure, there are probably a few reasonable uses of multi-tasking. But there are way more potential abuses.

I look at my Windows machine, which is continually thrashing away when I'm not doing anything. Every piece of software I install wants to run stuff in the background. Just to check for updates once a week they want to have a process running constantly. That's not a benefit, that's abuse.

Two days after I installed iOS 4 with its long awaited multi-tasking, I went to use my iPhone in the morning and the battery was totally dead - it wouldn't even turn on. It had plenty of power the evening before. I can't be 100% sure of the culprit, but I assume it must have been some application left running by the new multi-tasking. I suspect it was a mapping app that I'd been playing with, probably running the GPS constantly or something like that.

On the other hand, according to Apple, most apps do not actually run in the background, they just get suspended and resumed. If that's really true in all cases, then I'm not sure what drained my battery overnight.

So now I find myself regularly "killing" all the active apps because I'm paranoid about this kind of scenario. Great, a "feature" has imposed a large manual overhead on me.

Hopefully the situation will improve once apps are updated to work better with the multi-tasking. But there will always be poorly behaving apps.

What I find strange is that Apple went from not allowing multi-tasking, to making it the default, with no way to switch it off. It seems like an option that you had to explicitly turn on (or at least some way to disable it) would have still silenced the critics, but wouldn't have imposed the cost on those of us that didn't want it.

Chromium Embedded continued

Previously I got the Chromium Embedded (CEF) sample application to compile (and run). That was only a tiny step towards what I needed to do.

Suneido has a DLL interface, but it can only handle straightforward interfaces. Although CEF has a DLL with a C interface, it's fairly complicated. So I decided I needed to write a small DLL that Suneido could call that would talk to CEF.

But I wasn't sure what the minimum functionality I'd need. The sample application is actually quite large and complex because it is demonstrating a bunch of features like extensions and plugins. So I started by whittling down the sample to the bare minimum. That process helped me identify the core functions I'd need.

It took me a certain amount of struggling to recall / figure out how to build a Windows DLL. (It's not something I do very often.) Eventually I got to the point where Suneido could see the DLL functions. But calling them didn't work (it gave some weird stack errors). Now what?

Then I remembered the issues with building single-threaded versus multi-threaded. The main version of Suneido we use is built single-threaded (/ML), but CEF is built multi-threaded (/MT). I tried it with a multi-threaded build of Suneido and it worked! (Suneido is still single-threaded, it's just linked with the multi-thread runtime libraries.) Here is Chromium Embedded running in a Suneido window:

Coincidentally, the single/multi-thread version issue has also come up in another area. I'm looking at upgrading my work computer to Windows 7. But before I take the plunge I want to make sure the software I need is going to run under Windows 7. One of the "oldest" packages I use is Visual C++ 2003 (VC7). I would have moved to a more recent version long ago, but VC7 still generates the smallest, fastest executable. I think the main reason for this, is that VC7 was the last version that supported building with the single-threaded libraries. (/ML) I'd prefer to use GCC but it doesn't handle the interface with the Internet Explorer browser component. (It's also a bit slower, but not too bad.) So using CEF instead of IE might also allow switching to GCC.

Updating ASM

jSuneido uses ASM to generate Java byte code. I was using 3.1 and the current version is 3.3

I wasn't sure if this would be easy or hard. It turned out to be somewhere in between.

First I tried to update my Eclipse plugin. That was a little confusing because I had three update sites showing ASM - eclipse.org, asm.ow2.org, and andrei.gmxhome.de. I'm guessing the eclipse.org one is something internal to Eclipse. I thought asm.ow2.org would be the correct location, but it didn't have the latest version.

Next I downloaded asm-3.3.jar but that didn't work, I needed to download asm-3.3.zip and extract asm-all-3.3.jar  This wasn't entirely a surprise because previously I was using multiple asm jar files. I'm not quite sure why the asm-3.3.jar is available as a separate download, but not the asm-all-3.3.jar

Of course, tests failed. The first message was:


java.lang.IndexOutOfBoundsException: Trying to access an inexistant local variable 0
at org.objectweb.asm.tree.analysis.Frame.setLocal(Unknown Source)
at org.objectweb.asm.tree.analysis.Analyzer.analyze(Unknown Source)
at org.objectweb.asm.util.CheckMethodAdapter$1.visitEnd(Unknown Source)
at org.objectweb.asm.util.CheckMethodAdapter.visitEnd(Unknown Source)
at org.objectweb.asm.tree.MethodNode.accept(Unknown Source)
at org.objectweb.asm.commons.TryCatchBlockSorter.visitEnd(Unknown Source)
at org.objectweb.asm.MethodAdapter.visitEnd(Unknown Source)

[I thought "inexistant" was a typo, but it appears to be the French version of nonexistent.]

I searched for other people having this problem and found:

When visitMaxs returns an ArrayIndexOutOfBoundsException, more information would be nice

and

ClassWriter.COMPUTE_FRAMES causes java.lang.ArrayIndexOutOfBoundsException

I didn't resolve this. As a workaround I use CheckClass(visitor, false) to turn off the data flow checks. I am probably doing something wrong, but the byte code passes Java's verification and it runs ok, so presumably it's nothing too serious? 

The next problem was that you now have to call visitTryCatchBlock before defining any of it's labels. At first I thought that was easy to fix, just move the calls from the end of the try-catch block to the beginning. But one test failed - with nested try-catch blocks. The problem is that the JVM searches the exception table in order. So inner blocks must come before outer ones. By emitting at the end of the blocks I got the right order. But now that I was forced to emit at the beginning of the blocks, the order was wrong. 

This stumped me for a while. The problem is that jSuneido (and cSuneido) compiles in a single pass, generating code as it parses. This avoids building a syntax tree. But it doesn't give much flexibility in the order of generating code. I was having scary thoughts of having to make two passes, or switching over to building an intermediate tree. Of course, I could just go back to ASM 3.1, but in the long run I'd probably need to upgrade (e.g. for invoke dynamic support).

Again I searched for other people having this problem and didn't find anything. I was going to post on the ASM Tracker, but before I did I figured I'd better make sure there wasn't an existing post. As it turned out there was: Provide exception table sorting functionality (I'm not sure why my Google searches didn't find it.) Not only had someone else had the same problem, but they'd come up with a solution, and it was now included in ASM. All I had to do was add TryCatchBlockSorter. It's actually quite simple, once you know that the table is accumulated in a way that allows it to be sorted.

Still, I'm not sure why they added this requirement. It seemed to work fine without it in the previous version.

After that, I just had to fix a few tests to take into account the slight differences in the generated try-catch code and I was back in business.

Chromium Embedded

Our Suneido applications use the Internet Explorer browser component included in Windows.

This works pretty well, but we are running into a few problems:
- people have a wide variety of IE versions
- some people have Javascript turned off
- security settings are getting more and more restrictive

And in the long run I'd like to run on Mac and Linux which don't have IE built in for some reason :-)

So I've been thinking about using another browser component that we can have better control over. Initially I was looking at Gecko (the Mozilla Firefox engine). There's even a Mozilla ActiveX control with the same interface as IE, but it requires registry changes which isn't always feasible for our clients. I tried to find a workaround for the registry issue but couldn't get it working (and no one on Stack Overflow had an answer).

The recent trend seems to be to use WebKit - like in Safari, Chrome, Google Earth, etc. I found the Chromium Embedded project that seems like it would do the trick.

As a first step, I thought I should try to build the example application.Four hours later I think I have achieved this. Since it wasn't easy, I thought I'd recount the saga in case it might help someone else.

First, the standard build environment seems to be Visual Studio C++ 2005. I had various other version installed but not that one. Unfortunately, it's been replaced by VS 2008 which I tried building with, but it gave a lot of errors. (Although maybe the fixes below would have worked?)

A Stack Overflow post led me to someone's page with links to old versions. When I installed, I got compatibility warnings which suggested some updates. I downloaded the updates but when I tried to install it failed, saying the software wasn't installed. (Even though I could run it.) Instead, I was able to get the updates by running Windows Update.

When I tried to build it couldn't find atlbase.h  That's because Visual Studio Express (the free version) doesn't come with the Windows SDK. I already had a Windows Server 2003 SDK installed so I updated Tools - Options to point to it.

But I still got errors about shlwapi.lib missing. It turns out this is part of "Microsoft Web Workshop (IE) SDK", an optional part of the SDK. So I tracked down the right SDK installer and installed it.

Next, I had to patch atlbase.h and atlwin.h as explained in a Code Project post. This cleaned up a bunch more errors. But I was still getting errors about _Module which I fixed based on yet another post.

Then I got errors that I was missing opengl32.lib which stumped me for a while till I found someone else with a similar problem, and ended up at instructions for adding the SDK libraries to Visual Studio. Since you don't actually need opengl32.lib, I'm not sure why the project specifies it.

Initially the Debug version built. It ran, but it had a bad tendency to crash. The Release version seems to be fine.

Another issue is that it's big, about 20mb for the required dll's. I guess that's not that big these days, but it's still a fair size to deploy over the internet.

This all seems way harder than it should be. I realize it's Windows and Visual Studio and C++ and open source, but still! Maybe I've been spoiled by working with Java lately. Java does have benefits.

Azul's Better GC

InfoQ: Azul Systems To Open Source Significant Technology in Managed Runtime Initiative

Interesting. The more cores you have, the more garbage collection (GC) you need - a result of the shared memory model. So even if you avoid killing your multi-threaded performance with locking overhead, you still have scaling issues with GC.

This may also put pressure on Java to do more to avoid heap allocation (e.g. escape analysis and fixnums).