Java and Mac OS X

Trying to deploy my JavaFX application on Mac has been incredibly frustrating. But this is no accident. In fact, it is a direct result of Apple’s software philosophy. Specifically, Apple wants a couple of things: 1) to have 100% iron-fisted control of the whole Apple-branded technology stack and 2) to shift 100% of the pain of software problems from users to developers. You can’t argue that their philosophy has been successful in some sense—just check their stock price.

On the other hand, they have chosen to make life much more difficult for developers than necessary—especially Java developers. I’ll give a few concrete examples.

One of the most egregious examples of Apple’s software philosophy is how their products handle errors. The general idea seems to be to hide any remotely scary-sounding error message from the user. Back in the day, Windows was infamous for its “blue screen of death” (BSoD), an iconic example of the frightening error message. Now you tend to get more friendly, but still tech-arcane error messages on Windows. At least the user (or a developer) can do a quick web search for an error message and usually find the cause and/or solution. By contrast, Mac OS X tries to either hide the error and pretend like nothing happened (even if it means, your application appears to hang for no reason), or worse, give an outright lie as the message. For example, an application that is “improperly” digitally signed may generate a message saying that the application file is corrupted, and the user should try downloading it again. Heaven forbid we mention anything that might disturb the user like an untrusted cryptographic signature.

Signing an application with a well-known, trusted authority using a standard process is not enough for a Mac application. Instead, with the new Gatekeeper features, you need to sign your application using Apple’s process and Apple’s certificate, which they will happily sell you for $99 (no concern for the small-time developer who might not want to shell out $99 for a hobby project). If your application doesn’t jump through all of Apple’s hoops, the user might get anything from silent failure to an indication of a corrupted file.

Oracle is not blameless either. They should have made it a priority a long time ago to streamline deployment of rich client applications on any supported platform. The Java Web Start user experience is a travesty. Due to security issues, vendors like Apple and Mozilla have gone out of their way to make it difficult to run any Java applications. The result is a terrible user experience for launching Java-based applications, especially on Mac OS X. Worse yet, recent “bug fix” updates to Java 7 actually break some Web Start applications in the name of security.

But all is not lost. Enter JWrapper, a free and awesome solution for deploying Java apps as native applications on Windows, Mac, and Linux. Like other commercial solutions, JWrapper builds native installers that work regardless of any JRE being present. Unlike other solutions, JWrapper doesn’t cost thousands of dollars… in fact, the basic version (with a JWrapper logo) is free, and the paid license is only a few hundred dollars. Rock on!

I still had some trouble working out a few deployment kinks. In particular, I had to include a magical JVM option “-Djavafx.macosx.embedded=true” to get JavaFX to run properly and consistently in a Java 7 + Mac environment. But the build process is relatively simple, and now my application runs on a Mac with a good installation user experience. You plug in the code certificate files and it signs your application both in the standard process and the Mac signature process. I highly recommend JWrapper for developers of Java GUI applications.

Java Web Start and Code Signing

So Oracle has been steadily tightening the screws on Java security due to all the bad press recently. The latest version of Java suggests that they might block all unsigned applications in the future.  I find this “nanny” approach to running code to be kind of annoying; shouldn’t I decide what code to execute on my own computer?  Anyway, authors of Java Web Start applications need a cheap way to sign code.  Quick searches and StackOverflow suggest a few alternatives.   Here I’ll quickly mention two that I tried.

StartSSL is a small business from Israel that wants to end the protection racket known as “certificate authorities” (CAs).  They charge you a reasonable price for the actual work they do (i.e. validating your identity), then let you create as many types of certificates as you need.  Their SSL certificates are fine, and you can even get those for free.  They advertise code/object signing certificates if you pay for identity validation, and several blog and forum posts indicate that you can use their certificates to sign Java applications.  Their website says nothing to the contrary.  But, unfortunately, you can’t.  Long story short, I paid for identity validation and found out after the fact that StartSSL certificates are not trusted by Java and the company does not support Java code signing (from direct email correspondence). Too bad, I really wanted to like them.  Ugh.

Back to the drawing board.  Comodo offers code signing certificates that are trusted by Java.  They also have a bunch of resellers that sell the certificates at “reasonable” prices (reasonable relative to the “brand name” CAs).  I bought a certificate through KSoftware.  They want a bunch of random stuff after you pay for it, like they want you to update your phone number in some online directory (’cause no identity thief could ever do that?!), but overall it was a fast and good experience.  And most importantly, the KSoftware / Comodo certificate works and is relatively cheap, currently less than $100 per year. Some CAs charge five or six times that much.

The certificate gets magically installed in your browser. (I used Chrome.) You then export the certificate, making sure to “include the private key”, to a .pfx file.  Java’s keytool can convert the .pfx file to a Java keystore:

keytool -importkeystore -srckeystore theCertYouBoughtIncludingPrivateKey.pfx -srcstoretype pkcs12 -destkeystore yourShinyNewJavaKeystore.jks -deststoretype JKS

Then use the jarsigner tool to sign your jar file during your automated build process.  Note that the “alias” in the keystore will be a long string of letters and numbers with hyphens.

Patterns of Software: Tales from the Software Community

I stumbled across Patterns of Software: Tales from the Software Community by Richard Gabriel. (You can download the book as a pdf.) Gabriel approaches software design using principles from architecture, especially the work of Christopher Alexander.

Like other good writing on software architecture, Patterns of Software focuses on the tradeoffs involved. Rather than encouraging always more and stronger abstractions, Gabriel is more pragmatic and discusses what actually seems to work in software projects and the human reasons why.

Alexander described why a detailed “master plan” usually fails to create an aesthetically coherent community, and Gabriel applies this to software: first, we cannot know all the details of how the construction will be used and fit into its environment up front. Those details become apparent only when it is actually used. Software designers recognize this as the principle of “use before reuse”: you can’t create a good abstraction before you have several concrete use cases, because you do not know which details are important. These ideas are related to the “agile” development approach. In addition, the “users” subjected to a master plan (in the case of software, the implementers) often cannot feel ownership in a plan where they have zero influence. The master plan is a monolith, stuck in the past and owned by the original designer. Lack of ownership by programmers, over time, will translate to a disconnect between the whole and the parts of a design.

Gabriel warns of trying to create “perfect” or ultimately “clear” software from the start—a dubious task that leads to software that is not flexible in the ways that you need it. Why? Because humans cannot conceive all of the details and implications of a master plan all at once. It’s like trying to play an opening chess move by thinking about all of the possible endgames. Given our limited ability to reason, we have to focus on those things that we know and can reason about from the beginning, and keeping software understandable so we can make changes down the road. Gabriel calls this quality “habitability”, “the characteristic of source code that enables programmers, coders, bug-fixers, and people coming to the code later in its life to understand its construction and intentions and to change it comfortably and confidently”. I like this metaphor for software design. Make your software a place where future programmers (including yourself) can feel at home, so they can take ownership of the “home” and grow it as needed.

Kronos Quartet: Floodplain

Ran across this when I asked Pandora for “Iranian Traditional Music”. So apparently Pandora didn’t have much of that, but it tried at least, and Kronos Quartet’s Floodplain album does have some Middle-Eastern flavored tracks. One of the things I love about the album is how the pieces combine the exotic sounds of world folk music into richly-composed works. This isn’t the sort of wallpaper you sometimes get with folk or popular styles; it’s full-fledged art music that explores non-Western idioms. Some of my favorite tracks are “Nihavent Sirto” by Tanburi Cemil Bey and “Mugam Beyati Shiraz” by Rahman Asadollahi.

Building an application with Scala, sbt, JavaFX, ProGuard, Assembly, and Jarsigner

So I’m building a desktop app in Scala with JavaFX. I need a few things from my build process. At first, I thought the build process would be the easiest part of the application. Boy, was I wrong. I’ll save my griping about sbt for another post. Here I’ll talk about how I actually got it to work.

One of the keys (cough) to using sbt effectively is knowing which “settings” or “tasks” to override. You look in Keys.scala or the relevant source for any plugin you’re using to find the key, then figure out how to change it to what you need. Unfortunately, you really have to understand a bit about how sbt works (i.e. read the docs) to get anything done (on the plus side, there are docs). In my case, I needed:

  1. To build the application with all the dependencies
  2. Obfuscate it to discourage piracy
  3. Stuff it into a single jar with all the dependencies to make deployment less of a pain
  4. Sign the jar to reduce the scariness of the warnings

ProGuard is the de facto standard for obfuscation in Java-land, and Assembly is a commonly used tool for creating the single jar. We use jarsigner from the JDK to sign the jar.

There are at least two major ProGuard plugins for sbt: sbt/sbt-proguard by Typesafe and xsbt-proguard-plugin. I actually went back and forth between them several times, just trying to get either one to work with the flow described above. Finally I ended up with Typesafe’s plugin, because the other one doesn’t expose the output of ProGuard in a way that is easy to manipulate from sbt. Typesafe’s ProGuard plugin does have a “merge” function that appears to duplicate Assembly, but I couldn’t get it to work (ran into what appeared to be a temp directory naming problem), and I did get Assembly to work.

My final build flow in build.sbt looks like this:

  1. Normal compilation procedure
  2. Call Proguard, but using only the compilation output as the “input jar” and everything else (including the Scala runtime) is a “library jar”. I had to jump through some hoops here, because the sbt plugin authors seem to think the normal usecase is to shrink/obfuscate the Scala runtime, perhaps for an Android app. But any code you run through ProGuard as input means more nasty ProGuard configuration/warnings/etc., and I couldn’t find a good standard ProGuard config for the latest Scala (2.10.2).
  3. Run the output of Proguard plus all of the library dependencies into Assembly
  4. Run the output of Assembly through jarsigner
  5. Fist pump

In case anyone might find it useful, I’m including some of my build.sbt file. As a bonus, it also includes the trick to connect from your Ecliplse or other IDE debugger.

name := "myApp"

version := "1.0"

scalaVersion := "2.10.2"

libraryDependencies ++= Seq(
"commons-io" % "commons-io" % "2.4" // ...

// Uncomment to see warnings
//scalacOptions ++= Seq("-unchecked", "-deprecation")

// Force sbt to run the application in a separate JVM (needed for JavaFX)
fork := true

// Uncomment to run with debugger: connect to port 5005 from your IDE
//javaOptions in run += "-Xdebug"

//javaOptions in run += "-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005"

// Proguard


ProguardKeys.inputs in Proguard <<= exportedProducts in Compile map { _.files }

// Application-specific Proguard config
// dontusemixedcaseclassnames: workaround because Windows files are case-insensitive
ProguardKeys.options in Proguard += """
-keepclassmembers class * { ** MODULE$; }
""" // Add ProGuard config for your application

// Assembly


AssemblyKeys.jarName := "myApp.jar" // final output jar name

// Include the obfuscated jar from proguard, but exclude the original unobfuscated files
// Notice the dependency on ProguardKeys.proguard. This is to make sure it actually runs Proguard first;
// otherwise you can get an IOException. You would think ProguardKeys.outputs would be sufficient, but no.
fullClasspath in AssemblyKeys.assembly <<= (fullClasspath in AssemblyKeys.assembly, exportedProducts in Compile,
ProguardKeys.outputs in Proguard, ProguardKeys.proguard in Proguard) map {
(cp, unobfuscated, obfuscated, p) =>
((cp filter { !unobfuscated.contains(_) }).files ++ obfuscated).classpath

// If you have duplicate errors when Assembly does the merge, you need to tell it how to resolve them, for example:
//AssemblyKeys.mergeStrategy in AssemblyKeys.assembly <<= (AssemblyKeys.mergeStrategy in AssemblyKeys.assembly) { (old) =>
// {
// case PathList("org", "xmlpull", xs @ _*) => MergeStrategy.first
// case x => old(x)
// }

// Jarsigner

// Here we redefine the "package" task to suck in the Assembly jar and sign it.
Keys.`package` in Compile <<= (Keys.`package` in Compile, AssemblyKeys.assembly, sourceDirectory, crossTarget) map {
(p, a, s, c) =>
val command = "jarsigner -storepass myKeystorePassword -keystore "" + (s / "main/deploy/keystore.jks") + "" " +
""" + (c / a.getName) + "" myKeystoreAlias"
println(command) // just for fun
command !; // I love that ! executes the command in the shell
a // return the Assembly jar, which is now signed

// Result: run "package" command to generate and sign the "single jar" file called jarName

And as it says, running “package” from the sbt shell now does all the ProGuard-Assembly-Jarsigner magic. Whew!

JavaFX / FXML Injection Unit Testing

I have some UI components that use FXML files to define a view (layout), then the controller (input handling) is defined in the component code.  In Scala, it looks something like this:

class OptionsUi extends BorderPane {
private val fxml = new FXMLLoader(getClass().getResource("Options.fxml"))
@FXML protected var okButton: Button = _
// ...

The class “OptionsUi” serves as the controller, and the file “Options.fxml” is the view.  One problem is that the controls defined in the FXML file need to match up with the Scala class as variables annotated with @FXML.  For example, if the FXML file has a Button with the id “okButton”, then the controller class needs the variable okButton listed above.  The FXML API injects the okButton on fxml.load() using reflection.  I have to initialize the variable to something, so I use Scala’s ubiquitous underscore, which in this context means “I don’t care”, interpreted by Scala as a null reference.  (I use “protected” instead of “private”, because private variables generate different Java byte code that had confusing reflection-related interactions with JavaFX and FXML.)

The bad news: If I make a mistake, perhaps adding/removing/renaming a variable in one place and forgetting to do it in the other, it causes a NullPointerException when the variable is accessed.  Most of the time, NPEs are just a bad dream for Scala users, a memory of an earlier & uglier past, but there’s no way around it to use the FXML API in this way.

However, I could write an assertion that checks the okButton is injected with a non-null value.  One option is to write such an assertion for every @FXML variable in my controllers.  But we can go one step further:

test("UIs based on FXML have all @FXML members set from .fxml file") {

class FxmlTestApplication extends javafx.application.Application {
def createUis(implicit context: ApplicationContext): List[_] = List(new MainMenu, new OptionsUi)

override def start(primaryStage: Stage) {
try {
val context = new ApplicationContext()
val uis = createUis(context)
for (
ui <- uis;
f <- ui.getclass.getDeclaredFields;
if (f.getDeclaredAnnotations.exists { _.isInstanceOf[FXML] })
) yield {
val value = f.get(ui)
assert(value != null, s"${ui.getClass.getName} did not inject @FXML field ${f.getName}")
} finally {
// Workaround; can't avoid this
println("Please ignore exception printed by JavaFX: IllegalStateException: Attempt to call defer when toolkit not running")

Essentially, my unit test creates some components, looks for any @FXML fields, and checks that they were injected with non-null values.  I left the “ApplicationContext” in the example to show that you do have to create the UI components for this method, which means instantiating or mocking their dependencies.  Also, my test explicitly lists all the UI components in the application—with a littler effort you could get it to scan the ClassLoader for relevant/annotated components.  Finally, there is an exception printed by JavaFX on Platform.exit(), but it appears to be harmless and does not prevent the test from passing or failing appropriately.

Anyway, this provides the nice features of unit testing: more confidence in refactoring, catching typos much earlier, etc. with respect to the @FXML-injected variables.  It caught a mistake I made literally minutes after writing the test.

Back to an NPE-free Scala world!

Best music for programming

Different activities call for different music: Think about the kind of music we use for jogging, the kind we play at funerals, the kind we hear on the elevator, and the kind we appreciate in concert halls.  But what type of music is best for putting on in the background while programming?

Personally, I often like something that is more emotionally stable than symphonic repertoire, but not as repetitive as popular music.  If I’m solving a particularly difficult problem, I might want silence to give it my full attention.  But most software development involves some amount of mundane implementation work, where you already know the outcome, it’s just a matter of making countless small decisions to get there.  For that activity, I find it is nice to have something a little cerebral and a little bit interesting to make up the difference between a partially and 100% engaged mental state.  When my brain is 100% occupied, I can achieve flow and be optimally productive.

Baroque music is great for this.  The Baroque aesthetic usually requires a single “affect”, or sentimental state, for an entire movement.  This provides stability and avoids distracting emotional dynamics.  But within a stable framework, the music moves and changes, often with a steady running or motor effect, spinning out an idea over time.

At the moment, my favorite composer for programming is Dieterich Buxtehude, the Danish / German organist who is most famous for teaching Bach.  Pandora happily churns out mellow but thoughtful organ music on “Buxtehude radio”.  Arcangelo Corelli is another favorite.  A great example of the kind of piece I mean is “La Folia”, for violin and keyboard.   The piece is a passacaglia form, based on a repeating chord progression.  Over the course of the work, Corelli explores a surprising variety of textures and figurations within a fixed harmonic language.

Review: JavaFX and FXML

So I’ve been experimenting with UI development in JavaFX.  I initially was coding UIs by hand, to avoid introducing a dependency on any tool.  But I decided I rather like Oracle’s Scene Builder tool and FXML.  I’m not enamored of the idea of editing FXML by hand, although it’s nice that you can tweak it when necessary, and it makes a nice standardized serialization format for UI tools like Scene Builder.  Since Oracle is supporting it going forward, FXML is a stable format, and it’s nice that I can use it without being locked into a particular UI builder.  If someone makes a tool that beats the socks off of Scene Builder on day, and it supports FXML, I’ll be able to switch seamlessly.

After playing around with various approaches, I’m settling into a pattern for my current Scala application.  Part of the challenge is that there are so many ways to build a UI application in JavaFX that it’s not obvious which one to choose.  Here’s what is working for my desktop application:

I’m not using “pure” Model-View-Controller (MVC) with observers and binding to models all over the place.  Frankly, I think that’s overkill for many applications, though I see the value of it for large ones.  Magical bindings that update values back and forth are slick in simple cases (when you don’t need them anyway) but a haven for bugs in complex cases… I think they should be avoided unless the benefit is strongly needed (and it sometimes is).  So I’m using a “Flow Synchronization” approach, which fits well with the type of application I’m writing.

On the other hand, I am trying to make the two key separations that Martin Fowler describes as fundamental to MVC: between model and view/controller, and (secondarily) between the view and controller.  In Scala style, my model objects are either immutable or minimally-mutable.  The views are created in Scene Builder and stored in FXML, then loaded in a way that keeps this detail abstracted from the UI code.  The controller code, handling events and input, is then cleanly separated from layout code.  JavaFX has the nice touch of being able to style UIs with CSS, whether they are built through FXML or not.

Overall, I’ve been quite happy using JavaFX from Scala, even without the fun bindings in ScalaFX.  (I like their work, but I wasn’t ready to commit to a dependency for this application.)  I like being able to design UIs with a builder tool and load them cleanly and transparently from code, then use the concise and typesafe syntax of Scala to write the controlling code.