Speeding up compilation of large Flex 3 applications
Joa Ebert posted a very interesting blog post about Compiling ActionScript in The Enterprise and I thought it would be interesting to explain what I did a while ago as part of the Anvil Flex Open Source Project to speed up compiling Flex 3 applications. I should say we, because I had a lot of help from James Ward in building this. Anvil Flex was desinged somewhat like a portal server and the application was then broken down into smaller sub-applications or "modules" (in the orginizational context and not the flex context). Each module was designed so the it could be compiled and tested in a stand alone context. There was also several common libraries that where shared with the entire application
About 6 months into this project compile times for the entire application where exceding 20 minutes and it was nearly impossible to do continuous integration. One of the main problems with the flex compilers is each part of the compilation process (mxmlc, compc, opimizer, etc.) requires you to start a separate JVM, compile that small part of code and then shutdown the JVM. Not only does starting and stoping the JVM take time, you also lose all the compilation information when you shut down.
To solve this problem we build a custom Java compilation program called Taz that called directly into the Flex compiler classes, since the Flex compiler is written in Java. It consited of the following steps:
1 - We designed a heirarchial build system in Ant that would allow you to build any part of the tree and it's descendets automatically.
2 - The common libraries and modules each had there own Ant build files that described their dependecies.
3 - On building the application we would first walk the tree and build a dependcy heirarchy in memory of what modules and libraries need to be built.
4 - Then as we compiled each library we would cache any pre-processing and loaded libraries that could be reused throughout the compilation process.
The result was compile times dropped from 20 plus minutes to around 4 minutes! Then for continuous integration builds we set-up multiple builds that would first compile the individual modules and common libraries and then build the entire application. This would prevent a single module from breaking the build of all the other modules.
A very interesting and logic next step would be to take this process and allow the compilation process on individual developers machines to use the latest libraries built remotely. For example if a developer was working on module A that depended on module B they could specify to grab latest pre-built module B and any common libraries off a remote machine, like the last known good build from the continuous integration server. You could also distribute the build across machines this way.
A final note, a lot of people don't like Ant and say Maven is the build system of the future. Acutally part of the reason I stop working on Anvil is the entire build process was in Ant and I also thought the trend was to using Maven for builds. Then on my last project we used Maven extensively and I know believe that Ant actually might be better solution for compiling large and complicated Flex Applications. One of the reasons I didn't like the Maven build system was a large part of our build process actually used Ant behind the scenes for everything that could not be easily done in Maven. Yes, Flex Mojos and such can help compile a fairly standard Flex project, but when you project reaches a certain point of complexity that starts to fall apart. Things such as bundling custom resource files, including special assets in the final build, linking common library files, etc. it gets complicated really fast!
I had previously said that the Anvil Flex project was dead, but the code lives on in sourceforge. If there is any interested this build process could be extracted out into a separate, stand alone project.
