Scala Native & More

I recently discovered Scala Native, a version of Scala that compiles to a native executable rather than running on the JVM like the original Scala. (via Modern Systems Programming with Scala Native)

Scala is an attractive language. Scala is to Java as C++ is to C. Meaning it's much more complex. (Scala promotes its simple syntax, but that doesn't mean it's a simple language.)

In some ways I like Kotlin better than Scala. It seems to give a lot of the advantages of Scala, without Scala's complexity. And there are non-JVM versions of Kotlin. But the problem is that Kotlin relies on the Java libraries, which is a portability problem for the non-JVM versions. Whereas, one of the strengths of Scala is its libraries (e.g. collections).

Libraries bring up an interesting issue. To write good libraries seems to require more features and complexity than application code. A lot of the complexity of C++ is to enable writing good libraries, and I suspect the same is true of Scala. The other alternative is the Go approach. Go has a built in "map" collection. You can't write map in the Go language. If you want a slightly different style of map, what you can write in Go is nothing like the built in one. This is at the heart of the Go generics struggle. On one hand, you want to be able to write your own map in Go. On the other hand, you want to keep Go's simplicity and not end up with another C++.

Another factor is that people seem to like complexity, regardless of whether it's counter-productive, myself included, sadly. Languages like C++ are fascinating. There's always something more you can learn. Whereas simpler languages like Go are vaguely disappointing in that respect. I love how the C++ folks gleefully pile complexity on top of glittering complexity. But when it comes time to actually get some work done ... then maybe you want something simpler. (Especially when the wondrous pyramid is built on unsafe ground.)

Complexity also tends to affect build times. C++ and Scala are both known for slow compiles, whereas Go is extremely fast.

Unfortunately, from what I can see of Scala Native it's not very mature. It's currently using the a conservative Boehm garbage collector. That's what I use in cSuneido and from experience I know the weaknesses of conservative garbage collection. It's definitely not as strong as the JVM garbage collector. But they are working on a new garbage collector. Go (and Mono) also started out weak in this area but improved greatly. I can't tell whether Scala Native has the same libraries (e.g. collections) as regular Scala. The libraries are (mostly?) written in Scala so I would hope they would be portable.

Scala Native also appears to be weak in the concurrency area, which seems like a fatal flaw in today's multi-core world.

One feature mentioned on the web site is that Scala Native provides access to malloc and realloc. While this might appear attractive to C programmers, it seems a little odd in a garbage collected language. It also allows you to explicitly allocate data on the stack (like alloca). I'd much rather have escape analysis (like Go) that does that automatically. (I did see a mention that they are working on escape analysis.)

I'm not sure if it's indicative of the project as a whole, but the PragProg book seems focused on writing unsafe C code in Scala syntax. Personally, I have no desire to go back to the days of buffer overflows and mysterious crashes from dangling pointers and memory access violations. It might give higher performance, but I suspect you'd get similar performance from writing it in safe Go.

It's an interesting project that I plan to keep an eye on, but I don't think it's ready for my use yet.

Dependencies

I've always been reluctant to take on third party dependencies in my projects. I do it, but not lightly, and preferably not often. I could justifiably be criticized for regularly reinventing the wheel. On the other hand, I've seen and experienced things like DLL version hell on Windows.

The web arena goes to the other extreme of embracing dependencies. You seemingly can't write a web page without multiple frameworks and libraries du jour.  I'm always horrified when I install a Node package and it installs hundreds of dependencies. It seems like asking for problems, although somehow it seems to work almost all of the time.

Dependencies also tend to add to bloat. Code intended for reuse has to satisfy many people's needs. Which means it has a lot of features that you don't need. Compilers and other tools can often strip out some of the unused code. But features that are interwoven into the code (e.g. extra fields in a data structure) are not easily stripped out. Often, the 80-20 rule applies and you can get what you need with a small fraction of the code of a general purpose library.

Complexity is the other side of bloat. Do you really understand that third party library? What are its performance characteristics? How does it handle errors? What algorithms is it using? What are the weaknesses? The documentation may cover some of that, but it's more likely to just tell you how to use it. None of that may be relevant at first, but eventually it probably will.

Longevity is another big factor. A dependency that isn't maintained over the long term sooner or later becomes a liability. Yet major long term projects don't seem to be worried about taking on dependencies that are maintained by one person in their spare time.

Errors are another issue. I can fix errors in my own code. Errors in third party code are not so simple. You can submit a bug report and wait, hoping that it might get fixed eventually. Or you can try to figure out the code, and if you can comprehend it, do your own patch or fork. But if it's a big complex code base, that's usually not feasible.

You can't escape dependencies - operating systems, languages, databases, etc. are all dependencies. But you can think about the costs and risks as well as the benefits

Here's a good discussion of dependencies from Russ Cox, one of the core Go language team. (the post is not Go specific) Fairly long, but worth reading.

https://research.swtch.com/deps