Large Scale Java Design

I ran into a problem when I reached the point where I wanted to start integrating my append-only database storage engine into the rest of Suneido.

The problem was that the old storage engine was too tightly coupled with the rest of Suneido. Not very good design on my part, but in my defense, the design came from the old C++ code, much of which is 10 or 15 years old.

So how should the code be structured? What's the best way to use Java packages and interfaces to reduce coupling? I looked for the Java equivalent of Large Scale C++ Software Design by John Lakos. (Seeing that it was published in 1996, I guess I don't really have an excuse for the poor structure of my old code!) But there doesn't seemed to be an equivalent for Java. (Let me know if you have suggestions.)

There are books like Effective Java but they're mostly about small scale. And there are books about J2EE architecture, but I'm not using J2EE.

Books like Growing Object-Oriented Software (recommended) talk about coupling from the view point of testability, but don't explicitly give guidelines for high level architecture.

In addition, what I needed wasn't just less coupling. I needed to be able to configure the code to run with either storage engine. For that, there couldn't be any static coupling between the storage engine and the rest of the code.

One really crude way to do this would be to simply rename one of the two storage engine packages to be the name referenced by the rest of the code. That would require changing every source file. Eclipse would do that, but it wouldn't play well with version control. And I would have to manually ensure that the two packages were defined the same public methods.

Instead I decided to clean up my architecture. In case it's helpful to anyone else, I'll sketch out what I came up with. Some of it is standard advice, some is a little more specific to what I needed. None of it is particularly original or unique.

First, other packages (i.e. the rest of the system) should only reference interfaces that are defined in a separate interface package.

That means no direct field access, you have to use setters and getters for public access.

It also means no public static methods or constructors. Instead there is an interface for a "package" object. A singleton instance of the package object is created at startup. All access to the package starts here. So to use the new storage engine you just have to use its package object. This class contains any public constructors and static methods and is the only public class in the package.

One mistake I made, coming new to Java from C++, was to make methods either private or public. I should have been using package scope a lot more than public. (Which might explain why it's the default in Java!) Now, the only methods that are public are the ones implementing public interfaces.

It's nice that Java (since version 5 / 1.5) allows covariant return types. i.e. the public interface can say that a method returns another public interface, but the actual method implementation can return the concrete type. This reduces the need for casting within the package.

I've finished refactoring so the old storage engine code follows these guidelines. I thought I might run into areas that would take a lot of redesign, but it's gone surprisingly smoothly.

Next I have to refactor the new append-only storage engine to implement the same interfaces. Then I should be able to easily switch between the two.

Of course, the interface I've extracted from the old storage engine doesn't match the new storage engine so I'll have to do some refactoring to end up with a common interface.

As usual, the code is public in Suneido's Mercurial repository on SourceForge.

There's Always a Better Way

I'm working on the parser for jSuneido and it's going well.

In a few places the parser needs to "look ahead" at future tokens.

In cSuneido I did this by looking ahead directly in the source string.

When I was porting this code it seemed a little ugly. For example, it handled skipping whitespace but not comments. (Meaning you couldn't put a comment in certain places in the code.)

My next thought was to save the state, scan ahead, then restore the state. Easy enough, especially since it didn't have to be nested.

But it still seemed a little ugly.

Then I realized, I could just clone the lexer, and read ahead in the clone without disturbing the original. (I didn't use the Java clone stuff since it's a little twisted. I just made a "copy" constructor that takes another lexer.)

I'm a lot happier with this solution. I bet there's an even better solution out there, but this one will do for now.

I think one of the big weaknesses in beginning programmers is that they tend to settle for the first thing that works. But there are always other solutions and the first one you come up with is unlikely to be the best.

Of course, this is also what refactoring is about - improving your code, not settling for the first draft.

The hard part in this porting is having to read all the old C++ code and see all kinds of ways it could be improved. But I have to resist refactoring the C++ code or I'll never make any progress on jSuneido. One thing at a time. It's enough of a challenge to make the Java code as clean as I can.

PS. I'm back to using Eclipse. I could be happy with NetBeans but I find Eclipse a little better. Of course, part of that is just that I've spent more time with Eclipse.

Still Learning

Even though I created Suneido and have used it pretty heavily for years, I still find myself learning better ways to apply it. (I think that's part of the reason I like programming.)

Suneido uses the open source Scintilla editor. ScintillaControl is the "wrapper" that interfaces Scintilla to Suneido's user interface framework. I needed to add a new method to it today:

    LineEndExtend()
        { .SendMessage(SCI.LINEENDEXTEND) }

I noticed I had a lot of these methods and I started wondering whether there wasn't a better alternative to adding so many simple repetitious methods. Ruby, and especially Rails, which I've been involved with on another project, make heavy use of "catching" calls to missing methods and implementing them.

I was able to replace all these simple methods with:

    Default(method)
        {
        f = method.Upper()
        if not SCI.Member?(f)
            throw "method not found: " $ method
        return .SendMessage(SCI[f])
        }

"Default" is Suneido's way to "catch" calls to missing methods.

Then I realized I could generalize it to handle methods with arguments:

    Default(@args)
        {
        f = args[0].Upper()
        if not SCI.Member?(f)
            throw "method not found: " $ args[0]
        args[0] = SCI[f]
        return .SendMessage(@args)
        }

"@args" is used to capture all the arguments and then pass them again. args[0] will be the method name.

This allowed me to remove a bunch of methods and I won't have to add any more in the future.

In addition, I noticed I had a lot of calls like .SendMessage(SCI.GETLINECOUNT) within the wrapper code. These could now be simplified to be like: .GetLineCount()

This would all fall into the category of "refactoring" since I'm improving the code without changing its behavior. (Strictly speaking, the behavior has changed slightly, but not in a way that should affect existing code unless someone is doing something unusual.)

I guess you'd call this refactoring something like: "Replace explicit methods with catching missing method calls."