Saturday, 7 November 2009

Using groovy to generate java sources in maven

In this article I'll briefly cover how to use Groovy to generate java sources as part of a maven build.

There are times when you need to generate Java source which is either repetitive, based on external data or both. Now there are tools out there for some tasks like this, but for most simple tasks running a groovy script is more than enough.

Now for the purposes of this article, I'm going to presume we need an enum which represents the cards in a Standard deck of cards, one entry per card.

First we need to create our groovy script. Now the best place to put this is in the src/main/script directory of your project. Here's the script for this (writing groovy is not in scope for this article) which goes into the file src/main/script/CardDeckGen.groovy:
class CardDeckGen {

    public generate( project, String packageName, String className ) {

        // Where to write the classes
        File targetDirectory = new File( project.build.directory + '/generated-sources/groovy' )

        // The directory to write the source to
        File packageDir = new File( targetDirectory, packageName.replace( '.', '/' ) )

        // Now to create our enum
        def out = []
        out<<'package '+packageName+';\n'
        out<<'public enum '+className+' {\n'

        // We have four suits
        def suit = [ 'D', 'S', 'H', 'C']

        // Each suit has A, 2-9, T, J, Q & K
        def rank = [ 'A', '2', '3', '4', '5', '6', '7', '8', '9', 'T', 'J', 'Q', 'K' ]

        // For each suit, write an entry for each card
        suit.eachWithIndex {
            s, i -> def indent = i==0 ? '    ' : ',\n    '
            rank.each {
                r -> out<< indent + s + r
                indent = ','
            }
        }

        // Mark the end of the enum list
        out<<';\n'

        // Finish the enum class
        out<<'}\n'

        // Convert the array into a string
        StringBuilder sb = new StringBuilder()
        out.each { sb.append(it) }

        // Now write the source, ensuring the directory exists first
        packageDir.mkdirs()
        new File( packageDir, className + ".java" ).write( sb.toString() );
    }

}
Next we need to get maven to run this script, so in pom.xml we first need to make sure we are using the correct version of the jdk - 1.5 or 1.6 would be enough:
<plugin>
    <artifactId>maven-compiler-plugin</artifactId>
    <configuration>
        <source>1.6</source>
        <target>1.6</target>
    </configuration>
</plugin>
Next groovy, here we add the gmaven-plugin which runs a small code snippet calling our class:
<plugin>
    <groupId>org.codehaus.groovy.maven</groupId>
    <artifactId>gmaven-plugin</artifactId>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <goals>
                <goal>execute</goal>
            </goals>
            <configuration>
                <scriptpath>
                    <element>${basedir}/src/main/script</element>
                </scriptpath>
                <source>
                    CardDeckGen cdg = new CardDeckGen()
                    cdg.generate( project, 'uk.org.retep.example', 'CardDeck' )
                </source>
            </configuration>
        </execution>
    </executions>
</plugin>
Now that's enough to generate our enum, however maven doesn't know about it so it wont compile. To do that we need to add it so the compiler picks it up:
<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <executions>
        <execution>
            <id>add-groovy-sources</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>add-source</goal>
            </goals>
            <configuration>
                <sources>
                    <source>${project.build.directory}/generated-sources/groovy</source>
                </sources>
            </configuration>
        </execution>
    </executions>
Now you can run maven.mvn clean install
If all goes well you'll get a maven artifact containing an enum:
package uk.org.retep.example;
public enum CardDeck {
    DA,D2,D3,D4,D5,D6,D7,D8,D9,DT,DJ,DQ,DK,
    SA,S2,S3,S4,S5,S6,S7,S8,S9,ST,SJ,SQ,SK,
    HA,H2,H3,H4,H5,H6,H7,H8,H9,HT,HJ,HQ,HK,
    CA,C2,C3,C4,C5,C6,C7,C8,C9,CT,CJ,CQ,CK;
}
This was a simple example but quite a powerful one. I've used this technique for generating more complex versions of Card's to classes based on data contained within RFC's (STRINGPREP etc).

For help on writing Groovy stripts take a look at the Groovy Getting Started Guide.

Releasing to Kenai via Maven

Ever since the website feature appeared on Kenai I've been trying to get releases to work using it to host the maven repository. Although it works, it's always had the odd issue where whilst uploading artifacts to the repository, the connection would timeout causing the release build to fail.

For small projects this isn't really a problem, but for some of mine this is an issue to either the number of artifacts present (retepXMPP currently has over a hundred) we can't have the release build to fail part way through leaving both the mercurial and maven repositories in an inconsistent state.

The retepMicroKernel project is further complicated by the fact that it has native code - so a release has to be performed in parts with the native code built on the respective platforms.

Fortunately there is a solution to this - perform the release build into a staging repository then once it's complete merge that repository with the public one. This also means we can perform a release and if something horrible goes wrong we just redo the release without the public seeing the failed release.

Fabrizio Giudici also found this solution with an additional benefit. The maven release process makes several commits into the repository, but as mercurial is a distributed scm we can use that to our advantage - if the release build fails it doesn't cause issues remotely with having to remove tags etc (which I've done before).

So here's what I did when releasing version 9.11 of retepTools this week, first I followed Fabrizio's suggestions for configuring the root pom.xml file.

Then I created a temporary directory and inside that clone the public mercurial repo then make a clone of the clone:
# The maven staging repository
mkdir -p ~/tmp/stagingrepo

# The staging mercurial repository - this is a clone of the public repository
cd ~/tmp
hg clone https://hg.kenai.com/hg/reteptools~hg reteptools-stage

# The release mercurial repository - this is the one we will run maven against
# it is a clone of the staging repository
hg clone reteptools-stage reteptools
Now we run the release process:
cd ~/tmp/reteptools
mvn --batch-mode \
       -Dstaging.hg.repo.url=~/tmp/reteptools \
       -Dstaging.mvn.repo.url=file:///Users/peter/tmp/stagingrepo \
       -DreleaseVersion=9.11 \
       -DdevelopmentVersion=9.12-SNAPSHOT \
       -Dtag=retepTools-9.11 \
       release:prepare release:perform
Maven will now perform the release, making commits into the ~/tmp/reteptools repository and pushing them to ~/tmp/reteptools-stage.

Once we are happy that the release is clean we make it public. First mercurial - we need to push the changes back to kenai:
cd ~/tmp/reteptools-stage
hg push
Next the artifacts:
# A temporary directory for maven - it will not run without this directory
mkdir -p tmpdir
mvn org.codehaus.mojo:wagon-maven-plugin:1.0-beta-2:merge-maven-repos \
  -Dwagon.source=file:///Users/peter/tmp/stagingrepo/ \
  -Dwagon.target=dav:https://kenai.com/website/reteptools/maven/releases/ \
  -Dwagon.targetId=retep.releases \
  -Djava.io.tmpdir=target

[Edited to fix a couple of errors in the maven commands]

Tuesday, 11 August 2009

Forking projects

Just a quick one this (apart from trying out the ScribeFire FireFox plugin to see how it does for posting to the blog)... today whilst searching Google for something totally different I found a project on SourceForge called gnujpdf which is a fork of my old retepPDF project.

I've not tried it, but it's good to see someone resurrecting old code (although it does appear to have been dormant for about 2 years now).
This project is a new version of a project called 'retepPDF' by Peter Mount. The project hadn't been touched in a long time and I wanted to make some considerable changes to it, so I started a new project and renamed the package. I haven't been able to contact Peter yet, but all credit to him for the hard work of figuring out Adobe's wacky file format.
That's nice - actually Adobe's format is pretty well engineered, some of the better specification documentation I've ever seen so it was easy.

The one thing that got me was this:
This library started out as RetepPDF, by a guy named Peter T. Mount, whom I have yet to speak with. When researching existing projects to fill the need for the PDF functionality, I found RetepPdf on sourceforge and tried to get in touch with Peter, but never had any luck. The package hadn't been touched in a while, and I wanted to improve it, so I opened a new project on sourceforge.
Hmmm, unless it was an obsolete email address (not likely since 2000) it's probably been a spam filter as I always reply to this sort of thing.

I don't mind people forking my code, it actually shows to me it's being used. It's just nice to know of the forks as I'd probably either help out or see how differently people do things or find bugs etc.

I'm online in various forms, either email (peter at retep dot org dot uk), jabber (peter at retep dot org), msn (retep207 at hotmail dot com) the two blogs and various forums dotted around so it's not like I'm living on a different planet...