Stephen's Java Adventures

Note to self: Java Generics - The Get and Put principle

2011-10-24T11:19:00.005+01:00

With Java Generics, it can be useful to insert wildcards wherever possible, but how do you decide which wildcard is correct? When do you use ? super T, ? extends T and when should you not use a wildcard at all.

The Get and Put principle: Use an extends wildcard when you only get values from the structure. Use a super wildcard when you only put values into the structure. Don't use a wildcard when you both get and put values from/into the structure.

The best example of this principle is the following copy method signature:

public static <T> void copy(
Collection<? super T> destination,
Collection<? extends T> source);

The method gets values out of the source, so source uses extends, and it puts values into the destination, so that is declared with the super wildcard.

How to add Mac-like shadow to your screenshots

2011-10-19T14:33:00.000+01:00

Cmd+Shift+4 followed by Space is a lovely way to get a screen shot of a window on an Apple Mac...

In fact here is one such screenshot:

Notice the nice shadow effect around the screenshot? Ever wanted to add that shadow effect to your non-mac screenshots?

ImageMagick is your friend

convert NoShadowScreenshot.png $ +clone -background black -shadow 80x20+0+15 $ +swap -background transparent -layers merge +repage WithShadowScreenshot.png

There you are, nice equivalent shadow added.

Oh and if you hate the Mac's shadow:

To remove from existing screenshots?

ImageMagick is your friend again

convert WithShadowScreenshot.png -crop +40+25 -crop -40-55 WithoutShadowScreenshot.png

If you get sick of doing that all the time, you can just disable it.
Here’s the command. Just paste this line into your Terminal, and press return.
defaults write com.apple.screencapture disable-shadow -bool true

Then, you’ll need to restart the System’s UI Server, which is the component of the operating system responsible for doing things like taking screenshots and drawing drop shadows. To accomplish this, simply paste this line into the Terminal.
killall SystemUIServer

Java Serialization and locks

2011-08-15T12:17:00.003+01:00

Java has a couple of “nice” features, specifically:

Built-in locks
Baked in support for object serialization

Now I don't want to get into a war about how both of these are flawed (there was a reason that I put “nice” in quotes) but here is a quick note on how to make the two play nice (more as a reference to myself... but if others find this useful, well and good).

General principal

In essence, in each case you implement the readObject and writeObject methods per the Java Object Serialization Specification.

Using object's intrinsic lock

In this case you can just make the readObject and writeObject methods synchronized:

    private synchronized void readObject(ObjectInputStream stream)
            throws ClassNotFoundException, IOException {
        stream.defaultReadObject();
    }

    private synchronized void writeObject(ObjectOutputStream stream) throws IOException {
        stream.defaultWriteObject();
    }

Using Java 5 style locks

In this case you just wrap the functionality inside a try...finally block

    private void readObject(ObjectInputStream stream)
            throws ClassNotFoundException, IOException {
        lock.lock();
        try {
            stream.defaultReadObject();
        } finally {
            lock.unlock();
        }
    }

    private void writeObject(ObjectOutputStream stream) throws IOException {
        lock.lock();
        try {
            stream.defaultWriteObject();
        } finally {
            lock.unlock();
        }
    }

Using multiple locks

If your object uses multiple locks to guard the different fields, there are really two techniques you can use:

Get all the locks in one go, ensuring that you always get multiple locks in the same sequence (high-risk)
Stop relying on stream.default____Object() and get/set each field manually in code as that allows you to hold one lock at a time (may write a partially modified object)

Gotchas

Watch out for the fields you are writing being mutable and modified by other threads while you are writing them to the stream. Reads from a stream are less of an issue as the object should not be given to another thread until fully read back in.

2011-08-11T09:49:00.001+01:00

Book review: “EJB 3.1 Cookbook” by Richard M. Reese

Review

The kind people at Packt gave me a copy of “EJB 3.1 Cookbook” by Richard M. Reese to review. This book should be viewed as a collection of building block recipes, not as a collection of meal plans, and not as a how to cook book that you would read from cover to cover. When I look at the book from this point of view, I think that it is well structured and put together and the content is pitched just about right. This book does not tell you when to use a specific recipe, so it would be a mistake to think that this is the only book you would need for EJB 3.1. However, when you need a “Béchamel sauce”, this is a book that you can go to, quickly find and open up the right section and read the recipe to refresh your mind and get cooking.

The book itself is aimed at Java EE and EJB developers, promising to bring them quickly up to speed on how to use EJB 3.1 techniques through the use of step-by-step examples. I think that the back cover of this book could be designed a little better in terms of putting the book into context with regards to skill level and where the book should sit in a learning tree. In my view this book should be aimed for three audiences: the occasional EJB developer who has to work on other things a lot; the intermediate EJB developer who is moving to the EJB 3.1 platform from a previous release; and the beginning EJB 3.1 developer who has just read their first book or two on EJB 3.1 and now is trying to develop EJB 3.1 applications.

The cookbook format works well for a reference book when you know that you need to do XYZ, it will have a section that you can find quickly, and that section will be easy to find and contain some of the common tricks and pitfalls that apply to doing XYZ. When reviewing this book it is all too easy to fall into the “as an EJB architect” trap and point out that the book does not tell you when to do XYZ or how to decide between XYZ and ABC. Architects should not be reading cookbooks to find the answers to these kinds of questions.

There are some specific criticisms that I have with this book:

The book says that it uses Netbeans and Glassfish. To be honest I found very little in the book that is specific to either, and I think that the author would have been better off saying that all the examples were tested on Glassfish (which, as the reference implementation, should guarantee that they will work on all EJB 3.1 containers) and that IDE examples used Netbeans.
Chapter 11 covers packaging the EJBs, yet it does not cover or even hint at how to achieve this using the build tools that most developers use, e.g. ANT, Maven, etc. I think chapter 11 would be significantly enhanced if it included at least “Packaging EJBs using Apache ANT” and “Packaing EJBs using Apache Maven” recipes. Recipes for some of the other newer build tools like Buildr would be a bonus, but one could be forgiven for leaving them out.
There is no chapter on testing EJBs. In fact there are no testing recipes at all. No developer should be writing code without tests. At a minimum I would like to see a recipes like “Unit testing EJBs with mocking frameworks”, “Testing EJBs using OpenEJB’s embeddable container”, “Testing EJBs using JBoss’s embeddable container”, “Testing EJBs using Glassfish’s embeddable container”. I’m sure books could be written on the fine arts of each of these recipes but a book like this should at least be sketching out that this kind of thing is possible.

Despite these criticisms, the recipes that it does provide are clearly written and well set out in a consistent format. I specifically liked the “There’s more” section that each recipe has. If you try to read this book cover to cover you will be annoyed at the repetition, but then cookbooks are not meant to be read that way!

I would recommend this book for beginner and intermediate level developers who are either occasionally working with EJB 3.1 or who are moving from a previous version of EJB to 3.1. More advanced developers may not find many recipes that interest them, and architects will not find any guides about how to pick which recipes to combine to make the meal.

Overall: 6 out 10.

Book details

URL: http://www.packtpub.com/ejb-3-1-cookbook/book
Publisher: Packt Publishing
Language: English
Paperback: 436 pages [ 235mm x 191mm ]
Release Date: June 2011
ISBN: 1849682380
ISBN 13: 978-1-84968-238-1
Author(s): Richard M. Reese

Open Source: the Meritocracy vs the Circle of Trust

2011-08-04T11:38:00.003+01:00

There has been this idea running around the back of my head for a while, and it's only now that it is starting to crystalize into something that I can express.
When we look at Open Source projects, we see that there is a hierarchy of involvement. There are different levels at which you can be involved, and at each higher level, there will be less and less individuals. For now I am going to divide involvement up like this:

Interested: this group of individuals know that the project exists, and might even be following what it is doing, but have not been able to actually use the project at all yet (for whatever reason)
Consumers: these are the people who actually use the project. They may not even be following the project, e.g. a lot of people consume log4j or ANT without following the mailing lists, seeing what features are on the roadmap, or even filing issues.
Contributors: if you are filing issues, submitting patches, etc, but you do not have commit access to the project, then you are a Contributor.
Committers: you have commit access to the project
Management: you get to have a say in some of the following: whether the new release of a project can be released; the architecture/direction of the project going forward; who has commit access; etc.

Different Open Source projects but different barriers at different points. For example:

Contributor road-blocks:

You may have to create an account to file issues. [Technical]
You may have to sign a CLA and fax it off before you can commit any patches. [Legal]
You may have to get a notarized signed CLA sent via snail-mail before you can commit any patches. [Legal]
You may have to get your employer to sign-off on you contributing patches. [Legal]

Committer road-blocks:

You may have to be invited to become a committer.
You may have to establish merit before you can be invited to become a committer.
You may have to sign a CLA and fax it off before you can get commit access. [Legal]
You may have to get a notarized signed CLA sent via snail-mail before you can get commit access. [Legal]
You may have to get your employer to sign-off on you committing code to the project. [Legal]

Management road-blocks:

You may have to be invited to become management.
You may have to establish merit before you can be invited to become management.

(Note: I'm not going to even try and pretend that the above is a complete list of road-blocks)

Different people view success of Open Source projects differently. Some measures of success include: the number of consumers; the number of active contributors; the number of committers; the number of releases; the number of downloads; the number of issues raised and closed; the activity of mailing lists.

None of these are correct, and much like psychologists hypothesize a (in reality) unmeasurable “g factor” as a true measure of intelligence against which all real measures (such as IQ) are only partial measures. We could hypothesize an unmeasurable “s factor” which is the true measure of the success of an Open Source project.

But I don't want to go down such an academic road today. Instead lets look at the dependency tree in the measures of success that we know of...

Number of downloads depends on number of consumers: as a consumer has to download your project at least once... it is not a perfect dependency, because you could download each release of the project once and every time decide it is a load of rubbish and never actually consume it... and a consumer may have just downloaded version 1.0 once and stayed on that version for ever more, but in general, if you have a large number of consumers, you will have a large number of downloads and the opposite does not necessarily follow.
Activity of mailing lists depends on the number of active contributors: when you have a lot of active contributors, the mailing lists will be active... however the mailing lists can be active without any active contributors.
Number of issues raised depends on the number of active contributors: for similar reasons to the previous.
Number of issues closed depends on the number of committers: because you need commit access to close most issues (appart from the "not a bug" type of issues)

So a lot of the measures of success can be raised by increasing the number of consumers, active contributors, and committers. However, just because we have a large number of consumers/active contributors/committers does not imply that we have a successful project, it just says that projects that have a large number of consumers/active contributors/committers have a greater likelihood of being successful projects.

So how do we increase that number, and thereby increase the probability that our Open Source project is a “success”?

Well, if you are picking an project to use, i.e. deciding to become a consumer, one of the things you will look at is how active the community is (i.e. number of active contributors and number of committers). It's not the only thing you will look at, but assuming you have two projects which have the feature-set you need, and they are comparable in usability, etc. When you come to make the decision, the social beast in you will pick the larger active community (who wants to pick the dead project). The active community also act as evangelists out selling the project to potential consumers. So one way to grow the number of consumers is to grow the active community.

Number of consumers (partially) depends on the number of active contributors

If we have lots of active contributors and very few committers, very soon there will be a glut of patches that don't get applied, and the active contributors will wonder off to another project where there contributions can be consumed. So for each project, there is probably some upper limit to the number of active contributors that depends on the number of committers. That limit depends on things like the complexity of the project, the code base, the toolchain used by the project, the project management processes, etc. So I don't think there is any hard and fast rule like you can have X active contributors for every Y committers on any open source project.

But what I do think we can say is:

Number of active contributors is limited by the number of committers.

So at this point you might feel that my chain of logic looks a little like this:

But I should point out that what I am trying to say is really that:

Having a lot of committers
indicates that a project is more likely to be successful,
but it cannot guarantee that success.

Hopefully your back thinking I have a solid argument again.

So how do we get a lot of committers? Well the answer is easy, remove as many road-blocks to becoming a committer.

Some of the road-blocks are technical. So you have to create an account to file issues... well we can switch to a issue tracking tool that uses OpenID or oAuth and now you can use an account you already have to file those issues... or perhaps you can just provide your email address and fill in a captcha... etc. None of these completely remove the technical road-block, but they can make it easier for the consumer to graduate to active contributor.

Some of the road-blocks are legal. So you need a CLA signed with the blood of an Armur leopard before you can submit a patch. These are harder to work around, but not always impossible e.g. perhaps small patches (less than some size metric) don't need the CLA.

After all these road-blocks, we start to hit what I like to call the “merit wall”. And this is the problem that I see. In essence you need to establish that you have the merit to work on the project. You need to establish a chain of consistent patches (which get picked up by the existing committers) of good quality before you are invited to become a committer.

That to me is a crazy way to gate commit access.

Just because your submitted patches were all stellar quality, that could just be the result of the areas being patched having good test coverage, so you were forced to get the code right. It could be that the nature of bundling up your changes as a patch forced you to think more about the changes. You could become sloppy as a coder, etc.

There are many reasons why “past performance is not a guarantee of future returns”. Some people think the solution to this is to have merit expire, e.g. if you have not worked on the project recently, then you loose the right to work on the project. All that happens then is that people start doing busy work on the project just before their merit is due to expire, and such busy work is, by its very nature, not the work that drives a project forward.

In my view, the only merit to contribute to a project is the code you are contributing right now. Not the code you contributed yesterday, last month, or last year. Not the code you might contribute tomorrow, next month, or next year. But the code you are contributing right now. You don't have merit, but your code might.

Merit is the wrong thing to use as a gate for commit access.

Because what is giving commit access really about? It is about saying that we trust you to commit changes to the project. If you commit code which has no merit, we (or you) can use the SCM to back out those changes. The merit of your previous submissions is a very poor measure of our trust in you using the SCM. In fact because for a lot of projects, generating patches actually is about not using the SCM at all (Git pull requests changes that) it is exactly the wrong thing to use to establish trust.

Once we realize that commit access is about trust and not merit, then we can really start to build a community, and increase the likelihood of our project being a success:

You don't start with zero trust.
One big abuse of trust, and you have lost all your trust and you have to work a lot harder to earn what you had back again.
You have to be given scope in order to earn more trust.
Trust does not expire easily.

So what I am arguing is that if somebody shows up knocking at the door of your project looking to contribute, at some point you have to trust them enough to open the door and let them in. It's fine if you have a few hurdles that they have to jump, but remember that at the end of the day you are trying to decided if you trust them enough to let them make changes to the code, or to put it another way, if you trust them to make good judgements on the merit of any changes they want to commit.

Your merit as a committer is not the
merit of the code changes you have committed,
but all the code changes you did not commit because those changes had no merit.

A good committer is somebody who does not make changes that are bad. The ideal committer only makes commits that improve the project. Joe might be the ideal committer for your project even though he has never submitted a patch because Joe knows when a patch is improving the project and when it is not, but because Joe has never submitted a patch, he has earned no merit in your projects eyes, so he will never become a committer.

Commit access is about letting somebody inside the circle of trust. The easier you make that process, the more vibrant your community, and the more successful your project. Put a merit wall in place and you are alienating your potential community (never mind that it is the wrong barrier in the first place)

Look at the Jenkins project. If you want commit access, you just ask for it. How is that for initial trust. The result is a very successful project, oh and the sky has not fallen. Everyone that has been trusted so far has respected the trust they have been shown.

Look at most projects hosted on github. Pull requests have effectively removed a large chunk of the need for commit access, and because you can see the person's skills with the SCM, they can earn trust to be let loose on the canonical source directly.

A merit wall can leave you feeling like this

So my message to all open source projects that use merit based on previous patches as a measure of the worth of a potential committer, please look again at your policy. Are you sure you are looking for the right qualities to drive your project to bigger successes?

The worst Unit Tests I ever saw

2011-05-30T14:45:00.000+01:00

In relation to http://dhanji.github.com/#unit-tests-false-idol here is the tail of the worst test case I ever came across...

In a former employers, there was an employee who we will call Kevin McCallister in order to protect the guilty. In any case, for various reasons, I ended up having to maintain some of the code that Kevin wrote...

I ran all the test cases and measured the code coverage, it seemed high and there were lots of test cases... but bugs a plenty kept on hitting me... I finally found the answer in one test class... which looked a little something like this:

/**

* Tests the FooBar class
 * @author Kevin McCallister
 */
public void FooBarTest {  @Test
  public void smokes() {
    try {
      new FooBar().method1();
    } catch (Throwable t) {}
    try {
      new FooBar().method2();
    } catch (Throwable t) {}
    try {
      new FooBar().method3();
    } catch (Throwable t) {}
    ...
    try {
      new FooBar().method15();
    } catch (Throwable t) {}
    ...
    try {
      new FooBar().method30();
    } catch (Throwable t) {}
  }
  @Test
  public void method1DoesSomething() {
    // write later
  }
  @Test
  public void method1DoesSomethingElse() {
    // write later
  }
  @Test
  public void method1DoesAnotherThing() {
    // write later
  }
  @Test
  public void method1DoesSomethingWhenFoo() {
    // write later
  }
  @Test
  public void method1DoesSomethingWhenBar() {
    // write later
  }
  @Test
  public void method1DoesSomethingWhenBarAfterFoo() {
    // write later
  }
  @Test
  public void method1DoesSomethingWhenFooAfterBar() {
    // write later
  }
  
  ...
}

To make myself clear, there was one method at the top that called every function and would never fail (gives you the code coverage) and a couple of hundred empty test methods that did nothing but had names that sounded like nice test cases...

So there you have it, how not to write unit tests

Backwards compatibility

2011-05-27T14:55:00.000+01:00

This is a tail about Jenkins (née Hudson) and Kohsuke's policy of maintaining backwards compatibility...

Back in 2006 I started working for my previous employer, just a month or two after Peter Reilly started. Initially we were working on the same team. This was a team who's CI system was a nightly cron job that emailed off a list of failing tests to everyone... obviously Peter and I had many a WTF over that old system... so I convinced our boss that we should put some effort into setting up a proper CI system... Initially this was CruiseControl (as he thought Hudson at version 1.64 was too new and unheard of... go with the old reliable)... but after a couple of pains with the CruiseControl system (monolithic xml config file), we convinced him to switch to Hudson... (I don't think we ever looked back!)

What we really liked was that Kohsuke had put in place a plugin framework, so I started writing plugins, and shortly afterwards convinced Peter that it was "cool" (even if he did have to use Maven to build the plugins - Peter is on the Apache ANT PMC).

One of the earlier plugins that Peter wrote during his spare time was his "Simple Cobertura plugin" which allowed you to get the Cobertura coverage results on the build page[1]. Peter created his plugin against Hudson 1.129. By this stage Peter and I had moved onto separate projects, eventually Peter left the company, but the team he was working on still had the .hpi for his "Simple Cobertura plugin". As they moved to newer and newer Hudson builds, the plugin kept on working (no recompile needed). I had my own plugins which were built for an earlier version of Hudson (my closed source SilkTest plugin, my original closed source AccuRev plugin [not to be confused with the open source one I subsequently developed and handed on to others to maintain]) which also were still working (with no recompile) in newer versions of Hudson

Recently, I met up with Peter, and he told me the sad news.... his plugin is now broken... it no longer works in the latest version of Jenkins... this was strange news to me, as before I left my previous employer to join CloudBees I'd upgraded our CI servers to the latest and one of those was using Peter's "Simple Cobertura plugin" which did not seem broken...

After looking at Peter's code I finally found the problem... he was using the old javadoc annotation to mark the constructor as one for data binding, with one simple change:

6,7d5
< import org.kohsuke.stapler.DataBoundConstructor;
< 
22a21
>      * @stapler-constructor
24d22
<     @DataBoundConstructor

His plugin was back in action... Now I was puzzled, as I couldn't understand how I'd upgraded this old old plugin from Hudson 1.129 and got it running on Jenkins 1.397 without recompiling...

Well I found out the answer, the version Peter had left in behind, didn't use the data bound constructor at all... it was doing

    public Publisher newInstance(StaplerRequest req) throws FormException {
        return new C2Publisher(req.getParameter("c2_coverReportDir"));
    }

While the code Peter had was doing

    public C2Publisher newInstance(StaplerRequest req, JSONObject obj) 
            throws FormException {
        return req.bindParameters(C2Publisher.class, "c2_");
    }

If you change it back to the old way, and you have enough Maven-foo to build against that old a version of Jenkins (née Hudson) you can end up with a plugin that works on both Hudson 1.129 and Jenkins 1.413...

I am fairly sure that if I had a SilkTest license and a copy of the SilkTest plugin that I wrote against Hudson 1.96, it would also still run unmodified on Jenkins 1.413.

How many projects can maintain that level of backwards compatibility.

[1] Peter wanted to have the coverage results on the main page, and at the time Kohsuke did not want to allow plugins to add columns to the main page... I came up with the compromise of adding a column which would be just one icon wide and have a tooltip to which plugins could contribute information... and that was the birth of the weather (health) icons[2]

[2] at the time Peter thought it would be cool if he could generate his own weather icon on the fly so that if you had high code coverage that would be a full umbrella while poor code coverage would be a tattered umbrella with no fabric and only the frame remaining... this is actually part of the API, so the technology to implement this has been there since 1.115... just nobody has done it... yet!

Continuously deploy your java apps to the cloud

2011-05-18T10:29:00.000+01:00

In my previous post showed how easy it is to run your java application on CloudBees' RUN@cloud service. Today I'm going to use the CloudBees Deployer plugin for Jenkins that allows you to deploy your app to the cloud from your CI server. I am using the DEV@cloud Jenkins service for my CI infrastructure, but you can use this plugin from your own Jenkins (or Nectar) server.

So first step is to install the CloudBees Deployer plugin...

1. Goto Manage Jenkins

2. Goto Manage Plugins

3. Goto the Available tab

4. Scroll down until you find the cloudbees-deployer-plugin. Check the box.

5. Scroll down and click the Install button

6. Restart Jenkins after it's installed.

7. Goto Manage Jenkins, select Configure and scroll down to the bottom

8. Click the Add button beside the CloudBees accounts

9. Add in your CloudBees account details (which you can find on your user keys screen on grandcentral)

10. Click on Save and then goto the Configure page for your project. Enable CloudBees Deployment and fill in the details:

11. Save and then kick off a build

12. When the build is finished, your application has been deployed:

There you go. Continuous Deployment on RUN@cloud.

There is a whole host of things you can do with this. You can use build promotion to trigger the deployment, you can set up a staging deployment followed by the real thing if a test staging project builds successfully... I could go on... but there is always another day! Speaking of which, my next step will probably be enabling deployment straight from the project build (for those unfortunate enough to not have a CI server) probably using the ship-maven-plugin

Deploy your java apps to the cloud

2011-05-17T10:40:00.001+01:00

I work for CloudBees Inc., they are a great company with great products. I have mostly been working on the DEV@ side of the fence which is focused on continuous integration and basically the development side of your application, but we also have the RUN@ side of the fence where we provide a platform as a service (PaaS) for running your java web applications on the cloud. I could give you the sales pitch, but I'll leave it at: the technologies and people behind RUN@ were one of the key reasons why I decided to join CloudBees.

Well I've been busy on some stuff since joining, so I decided it was time to actually try out the RUN@ stuff for my self. So here is my experience:

My test application:

I'm on the Apache Maven PMC, so I'm going to build it with... shock... horror... Maven.
I am partial to the odd bit of JSF, so it will be a JSF 2.0 application based off of Apache MyFaces.
I love Jetty as a servlet container for local testing, so we'll use that hammer too.

Let's get started...

First the pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <!-- basic information -->

  <groupId>com.blogspot.javaadventure.cloudbees.run</groupId>
  <artifactId>jsf2-hello-world</artifactId>
  <version>0.1-SNAPSHOT</version>
  <packaging>war</packaging>

  <!-- Project information -->

  <name>JSF 2.0 Hello World</name>
  <description>
    A JSF 2.0 web application that says hello world.
  </description>

  <properties>
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    <project.build.outputEncoding>UTF-8</project.build.outputEncoding>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <!-- Dependency details -->

  <dependencies>
    <dependency>
      <groupId>org.apache.myfaces.core</groupId>
      <artifactId>myfaces-api</artifactId>
      <version>2.0.5</version>
    </dependency>
    <dependency>
      <groupId>org.apache.myfaces.core</groupId>
      <artifactId>myfaces-impl</artifactId>
      <version>2.0.5</version>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.8.2</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <!-- Build settings -->

  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <artifactId>maven-clean-plugin</artifactId>
          <version>2.4.1</version>
        </plugin>
        <plugin>
          <artifactId>maven-compiler-plugin</artifactId>
          <version>2.3.2</version>
          <configuration>
            <source>1.6</source>
            <target>1.6</target>
          </configuration>
        </plugin>
        <plugin>
          <artifactId>maven-deploy-plugin</artifactId>
          <version>2.6</version>
        </plugin>
        <plugin>
          <artifactId>maven-failsafe-plugin</artifactId>
          <version>2.8.1</version>
          <executions>
            <execution>
              <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
        <plugin>
          <artifactId>maven-install-plugin</artifactId>
          <version>2.3.1</version>
        </plugin>
        <plugin>
          <artifactId>maven-jar-plugin</artifactId>
          <version>2.3.1</version>
        </plugin>
        <plugin>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.8.1</version>
        </plugin>
        <plugin>
          <artifactId>maven-release-plugin</artifactId>
          <version>2.1</version>
        </plugin>
        <plugin>
          <artifactId>maven-resources-plugin</artifactId>
          <version>2.5</version>
        </plugin>
        <plugin>
          <groupId>org.mortbay.jetty</groupId>
          <artifactId>jetty-maven-plugin</artifactId>
          <version>8.0.0.M2</version>
        </plugin>
      </plugins>
    </pluginManagement>
    <plugins>
      <plugin>
        <artifactId>maven-release-plugin</artifactId>
        <configuration>
          <autoVersionSubmodules>true</autoVersionSubmodules>
          <goals>install</goals>
        </configuration>
      </plugin>
    </plugins>
  </build>

</project>

Then the src/main/webapp/WEB-INF/web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app version="2.5" xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd">
  <display-name>JSF 2.0 Hello World</display-name>
  <description>
    A JSF 2.0 web application that says hello world.
  </description>
  <context-param>
    <param-name>javax.faces.STATE_SAVING_METHOD</param-name>
    <param-value>server</param-value>
  </context-param>
  <context-param>
    <param-name>javax.faces.DEFAULT_SUFFIX</param-name>
    <param-value>.xhtml</param-value>
  </context-param>
  <context-param>
    <param-name>javax.faces.FACELETS_SKIP_COMMENTS</param-name>
    <param-value>true</param-value>
  </context-param>
  <context-param>
    <param-name>javax.faces.PROJECT_STAGE</param-name>
    <param-value>Production</param-value>
    <!--param-value>Development</param-value-->
  </context-param>

  <!-- [jetty] does not initialize myfaces correctly for some reason -->
  <listener>
    <listener-class>org.apache.myfaces.webapp.StartupServletContextListener</listener-class>
  </listener>
  <!-- [/jetty] -->

  <servlet>
    <servlet-name>Faces Servlet</servlet-name>
    <servlet-class>javax.faces.webapp.FacesServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>Faces Servlet</servlet-name>
    <url-pattern>*.xhtml</url-pattern>
  </servlet-mapping>

  <session-config>
    <session-timeout>60</session-timeout>
  </session-config>
  <welcome-file-list>
    <welcome-file>index.xhtml</welcome-file>
  </welcome-file-list>

</web-app>

Then the backing bean (src/main/java/com/blogspot/javaadventure/cloudbees/run/GreeterBean.java)

package com.blogspot.javaadventure.cloudbees.run;

import javax.faces.bean.ManagedBean;
import javax.faces.bean.ViewScoped;
import java.io.Serializable;

@ManagedBean(name="greeter")
@ViewScoped
public class GreeterBean implements Serializable {
    private String name;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getResponse() {
        if (name != null && !name.isEmpty()) {
            return "Hello " + name;
        } else {
            return null;
        }
    }

}

Should always have some tests (src/test/java/com/blogspot/javaadventure/cloudbees/run/GreeterBeanTest.java)

package com.blogspot.javaadventure.cloudbees.run;

import org.junit.Test;

import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;

public class GreeterBeanTest {
    @Test
    public void nullNameMeansNoGreeting() throws Exception {
        GreeterBean instance = new GreeterBean();
        instance.setName(null);
        assertThat(instance.getResponse(), nullValue());
    }

    @Test
    public void noNameMeansNoGreeting() throws Exception {
        GreeterBean instance = new GreeterBean();
        instance.setName("");
        assertThat(instance.getResponse(), nullValue());
    }

    @Test
    public void aNameMeansGreeting() throws Exception {
        GreeterBean instance = new GreeterBean();
        instance.setName("Fred");
        assertThat(instance.getResponse(), notNullValue());
    }
}

Next the page of our web application (src/main/webapp/index.xhtml), i'm going to use the JSF 2.0 ajax support (because it's there)

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:ui="http://java.sun.com/jsf/facelets"
      xmlns:h="http://java.sun.com/jsf/html"
      xmlns:f="http://java.sun.com/jsf/core">
<ui:insert name="metadata"/>
<h:head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
    <title>
        JSF 2.0 Hello World
    </title>
</h:head>
<h:body>
    <f:view>
        <h:form>
            <h:outputLabel for="greeter" value="Please tell me your name:"/>
            <h:inputText id="greeter" value="#{greeter.name}">
                <f:ajax event="keyup" render="text"/>
            </h:inputText>
        </h:form>
        <h:outputText id="text" value="${greeter.response}"/>
    </f:view>
</h:body>
</html>

Let's test it locally

$ mvn jetty:run

[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building JSF 2.0 Hello World 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]

...

WARNING: 

*******************************************************************
*** WARNING: Apache MyFaces-2 is running in DEVELOPMENT mode.   ***
***                                         ^^^^^^^^^^^         ***
*** Do NOT deploy to your live server(s) without changing this. ***
*** See Application#getProjectStage() for more information.     ***
*******************************************************************

2011-05-17 10:22:10.982:INFO::Started SelectChannelConnector@0.0.0.0:8080
[INFO] Started Jetty Server

Fire up a browser to http://localhost:8080/ and here's what we get:

OK, so now I turn off DEVELOPMENT mode in the web.xml, build my app and deploy it to RUN@cloud... and here's what we get:

That was cool. Didn't have to change anything (other than switch to production mode for safety as it's being deployed in the wild) and I did all this in under 20 minutes (including signing up for RUN@cloud)

My next steps will be to integrate this web application with DEV@cloud and our Jenkins plugin for deployment to RUN@cloud so that I can show off continuous deployment! But that will be a different day!

Quick and very Dirty Mavenizer

2011-04-26T15:21:00.000+01:00

The following quick and dirty bash script will take a pom and a jar and fake a maven build based on the source files for that that can be found in the current directory.

Really useful when running mvn dependency:analyze on a project you are validating POMs for.

#!/bin/bash

if [ "A$3" == "A" ]
then
  echo "Syntax: $0 pomfile jarfile dir"
  return
fi

rm -rvf "$3/src"
mkdir -p "$3/src/main/java"
cp -f "$1" "$3/pom.xml"
for name in $(jar -tf "$2" | sed -n -e "/\\$/d;s/\\.class/.java/p")
do 
  echo -n "Looking for $name ... "
  loc="$(find . | fgrep $name | head -n 1)"
  if [ "A$loc" == "A" ]
  then 
    echo "NOT FOUND"
  else 
    echo "$loc"
    mkdir -p "$3/src/main/java/$(dirname $name)"
    cp "$loc" "$3/src/main/java/$name"
  fi
done

My magic Maven and ANT version selection bash functions (for the Mac)

2011-03-25T09:45:00.001Z

OK, so a while back I posted my bash script for selecting the maven version to use for the current session http://s.apache.org/FQ2

Now that I have a Mac for my full time development machine, I thought I would share my version of these functions for Mac users:

usemvn ()
{
    if [ -z "$1" -o ! -x "/usr/share/java/maven-$1/bin/mvn" ]
    then
        local prefix="Syntax: usemvn "
        for i in /usr/share/java/maven-*
        do
            if [ -x "$i/bin/mvn" ]; then
                echo -n "$prefix$(basename $i | sed 's/^maven-//')"
                prefix=" | "
            fi
        done
        echo ""
    else
        if [ -z "$MAVEN_HOME" ]
        then
            export PATH=/usr/share/java/maven-$1/bin:$PATH
        else
            export PATH=$(echo $PATH|sed -e "s:$MAVEN_HOME/bin:/usr/share/java/maven-$1/bin:g")
        fi
        export MAVEN_HOME=/usr/share/java/maven-$1
    fi
}

useant ()
{
    if [ -z "$1" -o ! -x "/usr/share/java/ant-$1/bin/ant" ]
    then
        local prefix="Syntax: useant "
        for i in /usr/share/java/ant-*
        do
            if [ -x "$i/bin/ant" ]; then
                echo -n "$prefix$(basename $i | sed 's/^ant-//')"
                prefix=" | "
            fi
        done
        echo ""
    else
        if [ -z "$ANT_HOME" ]
        then
            export PATH=/usr/share/java/ant-$1/bin:$PATH
        else
            export PATH=$(echo $PATH|sed -e "s:$ANT_HOME/bin:/usr/share/java/ant-$1/bin:g")
        fi
        export ANT_HOME=/usr/share/java/ant-$1
    fi
}

Simply add the above into your ~/.bash_profile and you can use them from any bash shell.

For example:

[stephenc@stephenc ~]$ mvn -version
Apache Maven 3.0.2 (r1056850; 2011-01-09 00:58:10+0000)
Java version: 1.6.0_24, vendor: Apple Inc.
Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "mac os x", version: "10.6.6", arch: "x86_64", family: "mac"
[stephenc@stephenc ~]$ usemvn
Syntax: usemvn 2.2.0 | 2.2.1 | 3.0.2
[stephenc@stephenc ~]$ usemvn 2.2.1
[stephenc@stephenc ~]$ mvn -version
Apache Maven 2.2.1 (r801777; 2009-08-06 20:16:01+0100)
Java version: 1.6.0_24
Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "mac os x" version: "10.6.6" arch: "x86_64" Family: "mac"
[stephenc@stephenc ~]$

The joy of these as bash functions is that they only affect the current session. So I can have one Terminal tab running Maven 2.2.1 and another running Maven 3.0.2

Other tricks that people use by re-pointing the symlink are global and have global effect which makes hopping between tasks or testing different versions of Maven a whole lot harder.

Handy bash script to fix a pom's SCM section to the current Subversion location

2011-03-01T15:49:00.002Z

 #!/bin/bash

URL="$(svn info | sed -n -e '/^URL:/{s/URL: *//p}')"
ROOT="$(svn info | sed -n -e "/^Repository Root:/{s/Repository Root: *//p}")"
NEW_PATH="${URL#$ROOT}"
OLD_URL="$(sed -n '/< *scm *>/,/< *\/scm *>/p' pom.xml | sed -n '/< *connection *>/,/< *\/ *connection *>/{s/.*connection *> *scm:svn:\([^ <]*\)[ <].*/\1/p}')"
OLD_PATH="${OLD_URL#$ROOT}"
echo "OLD URL: $OLD_URL"
echo "NEW URL: $URL"
echo "ROOT: $ROOT"
echo "OLD PATH: $OLD_PATH"
echo "NEW PATH: $NEW_PATH"

sed -i '/< *scm *>/,/< *\/scm *>/{s/'${OLD_PATH//\//\\\/}'/'${NEW_PATH//\//\\\/}'/}' pom.xml

A handy string for when testing i18n

2011-02-10T10:10:00.000Z

Here's a handy string for i18n testing:

用户汉语

It can be handy to have a string in a non-english charater set that can be pushed end-to-end.

This should be a sad sad joke similar to the writing on the bottom of a leprechaun I saw in a gift shop in Ennis:

Déanta i dtír eile

I should point out that it was probably

Déanta sa tSín

Still

Mura féidir leat tuiscint a fháil ar an ansin is féidir leat léamh na Gaeilge.

Google seems to suggest that 中国扬声器可以阅读本 would be a better string to go with though!

The Clean Coder: QA or When do you flip a pancake?

2010-08-24T21:35:00.000+01:00

An interesting blog on QA and development: The Clean Coder: QA or When do you flip a pancake?

3D TV, level headed people will say it looks fine, looks bad lying down

2010-07-26T09:25:00.002+01:00

I had my first oportunity to see the new 3D TV's this weekend. They looked OK to me. There was a 3D effect. All around me were people who were going "wow! this is great! this is absolutely fantastic!".

But then I tilted my head, and low and behold, because the system uses horizontal separation to deliver 3D, it requires that your eyes be horizontally separated in order to deliver the 3D effect... when your eyes are vertical, however, it looks just as shit as when you have not got the glasses on.

Now I don't know about you, but one of my favoured ways of watching TV is lying on the couch with my head turned sideways to face rthe TV... which results in my eyes being vertical not horizontal...

So I'll add that to the top of the list of reasons why I'll not be buying a 1st generation 3D TV

Quick and dirty remove -SNAPSHOTS from your local maven repository

2010-07-19T09:36:00.001+01:00

Works on *nix

find ~/.m2/repository -type d -name \*-SNAPSHOT -exec rm -rvf {} \;

By searching for the directories we should catch the -YYYYMMDD.HHMMSS format of snapshots also

Keeping Hudson configuration and data in SVN

2010-07-08T11:54:00.002+01:00

We recently lost our hudson server due to a multiple disk failure in the RAID array storing our hudson configuration. [5 of the 15 disks died]

So I've been looking into a backup script that will allow us to keep a backup of the configuration. We use Maven for most of our builds, so the released artifacts are in our Maven Repository (which is hosted on two servers each with RAID arrays and using DRBD to mirror between the pair, with an rsync to a NAS in another cabinet and we are trying to get an rsych to an off-site storage going as well).

There seem to be two main options:

Use the Hudson Backup Plugin.
Backup your Hudson configuration to Source Control.

Option 1 is nice, but you still have to copy off the backup files and find some safe place to store them. Option 2 sounds better to me as if we loose our source control system, we'll there's nothing to build anyway!

So a quick search of the interwebs revealed Mike Rooney has been here before... cool... I have somewhere to start... with a few tweaks I have ended up with the following:

#!/bin/bash
if [[ ! -d $HUDSON_HOME ]]
then
echo "Hudson home directory ($HUDSON_HOME) is missing or undefined"
exit 1
fi

cd $HUDSON_HOME

# Add any new conf files, jobs, users, and content.
svn add -q --parents *.xml jobs/*/config.xml users/*/config.xml userContent/*

# Add the names of plugins so that we know what plugins we have
ls -l plugins > plugins.list
svn add -q -N --parents plugins.list

# Ignore things in the root we don't care about.
echo -e "war\nlog\n*.log\n*.tmp\n*.old\n*.bak\n*.jar\n*.json\nsecret.key\ntools\nshelvedProjects\n.owner\nupdates\nplugins" > myignores
svn ps -q svn:ignore -F myignores . && rm myignores

# Ignore things in jobs/* we don't care about.
echo -e "builds\nlast*\nnext*\n*.txt\n*.log\nworkspace*\ncobertura\njavadoc\nhtmlreports\nncover\ndoclinks" > myignores
svn ps -q svn:ignore -F myignores jobs/* && rm myignores

# Remove anything from SVN that no longer exists in Hudson.
svn st | sed -n -e "s/^\!//p" | xargs -r svn rm

# And finally, check in of course, showing status before and after for logging.
svn st && svn ci --non-interactive --username=hudson-build -m "automated commit of Hudson configuration" && svn st

The main changes are that I've updated the root level ignores and I capture a listing of the plugins directory so that you can know exactly what plugins you had installed

Easy parallelization with Bash in Linux

2010-05-14T10:52:00.001+01:00

Cool post I just found:

http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/7

Crazy counting...

2010-04-03T18:14:00.001+01:00

When you are keeping build/version number in strings, you really need to left pad with 0's in order for string sorting to work...

e.g. 1.0-alpha-2 > 1.0-alpha-1 > 1.0-alpha-10

So as long as we start off with a leading zero, we are fine, e.g. 1.0-alpha-01

Of course the next question is how much padding...

Eventually you will reach a point where the amount of padding is too much... enter crazy counting...

Start off with a small amount/no padding and when it looks likely that you will run out, skip versions...

With base 10, you'll probably want to start skipping somewhere around the 5-7 mark, so we'd have

1.0-alpha-1, 1.0-alpha-2, 1.0-alpha-3, 1.0-alpha-4
(hmmm we're burning a lot of alphas... looks like we might make 10 or 15, better start crazy counting)
1.0-alpha-50, 1.0-alpha-51, ... 1.0-alpha-99 and we're screwed unless we did another crazy counting...

If we thought we'd need lots more than 50 builds, we could go really crazy

1.0-alpha-4 -> 1.0-alpha-500 -> 1.0-alpha-501, etc

BTW, crazy counting is handy with Maven which uses string based comparison for qualifiers

Don't Do What Donny Don't Does #1 - Seam @Synchronized and synchronized methods

2010-03-24T11:58:00.002Z

I am starting a series of things you should not do:

Seam has this handy annotation: @Synchronized which ensures that only a single thread may access the methods/fields of the component at the same time.

Often times it is easy to forget that SESSION scoped Seam components are automatically @Synchronized

Java has this (formerly) handy modifier synchronized which when applied to a method, ensures that the object's implicit lock is held whenever the method is invoked.

Hilarity will ensue if you have an @Synchronized component with synchronized methods (which you should not have to apply because the component is @Synchronized), e.g.

@Name("donny")
@Scope(ScopeType.SESSION)
public class DonnyDont {
public synchronized String dont() {
// some stuff
}
}

OpenEjb, Jetty and Maven - Transaction Management

2010-03-10T13:01:00.001Z

I've been meaning to blog about getting transaction management working with OpenEjb and Jetty using jetty:run... it's still an on-going story... but the following might get you going...

First off, in your pom.xml you need to add the configuration for maven-jetty-plugin... we need to dance around the various activemq/activeio versions and ensure that we get the correct version of ant...

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
      <modelVersion>4.0.0</modelVersion>
    <groupId>org.apache.openejb.examples</groupId>
    <artifactId>jetty-openejb</artifactId>
    <packaging>war</packaging>
      <version>1.0-SNAPSHOT</version>
    <name>jetty-openejb Maven Webapp</name>
    <url>http://maven.apache.org</url>
    <dependencies>
          <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
          </dependency>
    </dependencies>
    <build>
        <finalName>${project.artifactId}</finalName>
        <plugins>
            <plugin>
                <groupId>org.mortbay.jetty</groupId>
                  <artifactId>maven-jetty-plugin</artifactId>
                <version>6.1.22</version>

                <dependencies>
                    <dependency>
                        <groupId>org.apache.activemq</groupId>
                          <artifactId>activemq-core</artifactId>
                        <version>4.1.1</version>
                        <exclusions>
                            <exclusion>
                                  <groupId>commons-logging</groupId>
                                <artifactId>commons-logging</artifactId>
                            </exclusion>
                             <exclusion>
                                 <groupId>commons-logging</groupId>
                                <artifactId>commons-logging-api</artifactId>
                            </exclusion>
                              <exclusion>
                                <groupId>org.apache.activemq</groupId>
                                <artifactId>activeio-core</artifactId>
                              </exclusion>
                        </exclusions>
                    </dependency>

                    <dependency>
                        <groupId>org.apache.activemq</groupId>
                          <artifactId>activemq-ra</artifactId>
                        <version>4.1.1</version>
                        <exclusions>
                            <exclusion>
                                  <groupId>commons-logging</groupId>
                                <artifactId>commons-logging</artifactId>
                            </exclusion>
                             <exclusion>
                                 <groupId>commons-logging</groupId>
                                <artifactId>commons-logging-api</artifactId>
                            </exclusion>
                              <exclusion>
                                <groupId>org.apache.activemq</groupId>
                                <artifactId>activeio-core</artifactId>
                              </exclusion>
                        </exclusions>
                    </dependency>

                    <dependency>
                        <groupId>org.apache.activemq</groupId>
                          <artifactId>activeio-core</artifactId>
                        <version>3.1.2</version>
                        <exclusions>
                            <exclusion>
                                  <groupId>commons-logging</groupId>
                                <artifactId>commons-logging</artifactId>
                            </exclusion>
                             <exclusion>
                                 <groupId>commons-logging</groupId>
                                <artifactId>commons-logging-api</artifactId>
                            </exclusion>
                          </exclusions>
                    </dependency>

                    <dependency>
                        <groupId>org.apache.openejb</groupId>
                        <artifactId>openejb-core</artifactId>
                          <version>3.1.2</version>
                        <exclusions>
                            <exclusion>
                                <groupId>org.apache.activemq</groupId>
                                  <artifactId>activemq-core</artifactId>
                            </exclusion>
                            <exclusion>
                                <groupId>org.apache.activemq</groupId>
                                  <artifactId>activemq-ra</artifactId>
                            </exclusion>
                            <exclusion>
                                <groupId>org.apache.activemq</groupId>
                                  <artifactId>activeio-core</artifactId>
                            </exclusion>
                            <exclusion>
                                <groupId>junit</groupId>
                                  <artifactId>junit</artifactId>
                            </exclusion>
                        </exclusions>
                    </dependency>
                    <!-- in order to use the latest version of openejb, we need to exclude
                           the dependencies provided in jsp-2.1-jetty -->
                    <dependency>
                        <groupId>org.mortbay.jetty</groupId>
                        <artifactId>jsp-2.1-jetty</artifactId>
                          <version>6.1.22</version>
                        <exclusions>
                            <exclusion>
                                <groupId>ant</groupId>
                                  <artifactId>ant</artifactId>
                            </exclusion>
                        </exclusions>
                    </dependency>

                 </dependencies>
                 <configuration>
                    <jettyConfig>${basedir}/src/main/jetty/jetty.xml</jettyConfig>
                </configuration>
            </plugin>
        </plugins>
      </build>

</project>

Next we need to configure a src/main/jetty/jetty.xml to bind the UserTransaction instance into jetty...

  <?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">
  <Configure id="srv" class="org.mortbay.jetty.Server">

    <New class="javax.naming.InitialContext">
        <Arg>
            <New class="java.util.Properties">
                  <Call name="setProperty">
                    <Arg>java.naming.factory.initial</Arg>
                    <Arg>org.apache.openejb.client.LocalInitialContextFactory</Arg>
                  </Call>
            </New>
        </Arg>
        <Call name="lookup" id="tm">
            <Arg>openejb:TransactionManager</Arg>
        </Call>
      </New>

    <New class="org.mortbay.jetty.plus.naming.Transaction">
        <Arg>
            <New class="org.apache.openejb.core.CoreUserTransaction">
                <Arg>
                      <Ref id="tm"/>
                </Arg>
            </New>
        </Arg>
    </New>

</Configure>

And presto-chango, now jetty has a transaction manager provided by openejb. (Note: if we don't mind storing that in a jetty-env in /WEB-INF, you can put the same config in WEB-INF/jetty-env.xml)

OK, so here are the issues:

Reloading does not work (because org.apache.openejb.core.ivm.naming.IvmContext does not support the destroySubcontext(Context) method
We are using jetty's JNDI provider in the web-app and openejb's JNDI provider for the EJBs... this is because

When jetty binds names to JNDI (using org.mortbay.jetty.plus.naming.Resource or org.mortbay.jetty.plus.naming.Transaction) it binds the object to JNDIName and it also binds a NamingEnrtry for the object to __/JNDIName

Unfortunately, openejb's JNDI implementation seems to be somewhat strange in this regard... if we add the SystemProperties to jetty to have it use openejb's JNDI implementation, e.g. add the following to /project/build/plugins/plugin[maven-jetty-plugin]/configuration/systemProperties

                        <systemProperty>
                            <name>java.naming.factory.initial</name>
                            <value>org.apache.openejb.client.LocalInitialContextFactory</value>
                        </systemProperty>

Then when we bind __/UserTransaction it gets bound to openejb:__/UserTransaction but when we lookup __/UserTransaction openejb looks up openejb:local/__/UserTransaction

And that is just for starters... there seems to be a whole host of other JNDI strangeness between jetty's side and openejb's side

The side effect of all this is that if you want resource refs to work correctly, you need to fish them out of openejb's JNDI context and push them into jetty's JNDI context

In any case this is at least a start!

Review Board and Subversion Hooks

2010-02-17T13:20:00.004Z

Review Board is quite nice... it has a handy program for posting reviews (postreview)... and you can integrate this into your subversion hook scripts quite nicely...

But what if you want to automate submitting reviews on only parts of your code base...

What I want is to be able to set a property on a folder and then any time a file is changed in that folder or it's children, then a review will automatically be scheduled...

So I wrote the following C/C++ helper (because we would potentially be forking a lot of svnlook processes) which checks to see if a property is set on any of the changed paths or the changed path parents.

/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements.  See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership.  The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License.  You may obtain a copy of the License at
*
*   http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied.  See the License for the
* specific language governing permissions and limitations
* under the License.
*/

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>

/**
 * Returns a stream for the output of dirs-changed from svnlook.
 */
FILE *svnlook_dirs_changed(char *svnlook, char *repo, char *rev) {
    char *cmd;

    cmd = (char *) malloc(strlen(svnlook) + strlen(" dirs-changed ") + strlen(repo) + strlen(" -r ") + strlen(rev) + 1);
    strcpy(cmd, svnlook);
    strcat(cmd, " dirs-changed ");
    strcat(cmd, repo);
    strcat(cmd, " -r ");
    strcat(cmd, rev);
    return (FILE *) popen(cmd, "r");
}

/**
 * Uses svnlook to check and see if the specified property is set on the specified path in the specified repository.
 */
int svnlook_is_prop_set(char *svnlook, char *repopath, char *propname, char *path) {
    int status;
    pid_t pid;
    pid = fork();
    if (pid == 0) {
        fclose(stdin);
        FILE* fp = fopen("/dev/null", "w+");
        if (fileno(fp) != STDIN_FILENO) {
            _exit(EXIT_FAILURE);
        }
        if (dup2(STDIN_FILENO, STDOUT_FILENO) == -1) {
            _exit(EXIT_FAILURE);
        }
        if (dup2(STDIN_FILENO, STDERR_FILENO) == -1) {
            _exit(EXIT_FAILURE);
        }
        execl(svnlook, svnlook, "pg", repopath, propname, path, NULL);
        printf("exit failure\n");
        _exit(EXIT_FAILURE);
    } else if (pid < 0) {
        status = -1;
    } else {
        if (waitpid(pid, &status, 0) != pid) {
            status = -1;
        }
    }
    return status;
}

/**
 * Checks to see if any of the directories from the stream have the specified property set.
 */
int check_svn_dirs(FILE* fdirschanged, char *svnlook,char *repopath, char *revnum, char *propname, int showallpaths) {
    char line[1024];
    int rv = 0;;
    while (fgets(line, sizeof line, fdirschanged)) {
        for (unsigned int i = 0; i < sizeof line; i++) {
            if (line[i] == '\n') {
                line[i] = 0;
                for (int j = i; j > 0; j--) {
                    if (line[j] == '/') {
                        line[j] = 0;
                        if (0 == svnlook_is_prop_set(svnlook, repopath, propname, line)) {
                            printf("%s/\n", line);
                            if (!showallpaths) {
                                return 1;
                            } else {
                                rv = 1;
                            }
                        }
                    }
                }
                line[0] = 0;
                if (0 == svnlook_is_prop_set(svnlook, repopath, propname, line)) {
                    printf("/\n");
                    if (!showallpaths) {
                        return 1;
                    } else {
                        rv = 1;
                    }
                }
                break;
            } // else it's a really long path and I'm refusing to check it!
            
        }
    }
    return rv;
}

int main(int argc, char **argv) {
    FILE *fpipe;

    char *svnlook = "/usr/bin/svnlook";
    char *repopath = "..";
    char *revnum = NULL;
    char *propname = NULL;
    int showallpaths = 0;

    int index = 1;
    int help = 0;
    while (index < argc) {
        if (strcmp("--svn-look", argv[index]) == 0) {
            index++;
            if (index < argc) {
                svnlook = argv[index];
            } else {
                help = 1;
                break;
            }
        } else if (strcmp("-r", argv[index]) == 0) {
            index++;
            if (index < argc) {
                revnum = argv[index];
            } else {
                help = 1;
                break;
            }
        } else if (strcmp("--repo", argv[index]) == 0) {
            index++;
            if (index < argc) {
                repopath = argv[index];
            } else {
                help = 1;
                break;
            }
        } else if (strcmp("--property", argv[index]) == 0) {
            index++;
            if (index < argc) {
                propname = argv[index];
            } else {
                help = 1;
                break;
            }
        } else if (strcmp("--show-all-paths", argv[index]) == 0) {
            showallpaths = 1;
        } else if (strcmp("--help", argv[index]) == 0 || strcmp("-h", argv[index]) == 0 || strcmp("-?", argv[index]) == 0) {
            help = 1;
            break;
        } else {
            help = 1;
            break;
        }

        index++;
    }

    if (help || svnlook == NULL || repopath == NULL || revnum == NULL || propname == NULL) {
        printf("Syntax: %s options\n", argv[0]);
        printf("Options:\n");
        printf("    --svn-look PATH     Specify an alternative svnlook binary location (default /usr/bin/svnlook)\n");
        printf("    --repo PATH         Specify the repository to work against (required)\n");
        printf("    -r REVNUM           Specify the revision to process (required)\n");
        printf("    -property NAME      Specify the property to check (required)\n");
        printf("    --show-all-paths    Shows all the changed paths with the property rather than just the first (optional)\n");
        return 2;
    }

    int rv = 0;
    fpipe = svnlook_dirs_changed(svnlook, repopath, revnum);
    if (fpipe) {
        rv = check_svn_dirs(fpipe, svnlook, repopath, revnum, propname, showallpaths);
        pclose(fpipe);
    }
    return rv;
}

Now I should warn you that it's a wee while since I wrote C/C++, so apologies if the above is not perfect... it works for me.

Next I use a post-commit.sh script to check for my property:

#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#

path=$(/usr/bin/svnlook props --repo "$1" -r $2 --property reviewboard:autoreview)
if [ $? == 1 ]; then
  if [ $(svnlook changed "$1" -r $2 | sed -e "/^_.*/d;" | wc -l) -eq 0 ] ; then 
    # Commit does not contain changed files, only changed properties
    exit 0
  fi
  people=$(for p in $(/usr/bin/svnlook props --repo "$1" -r $2 --property reviewboard:autoreview --show-all-paths); \
           do /usr/bin/svnlook pg "$1" reviewboard:autoreview "$p"; echo ""; done \
           | sed -e 's/[, \t]/\n/g;' | sort -u | tr '\n' ',' | sed -e "s/,$//;")
  post-review --server=... --username=... --password=... --submit-as=$(/usr/bin/svnlook author "$1" "$2") \
        --repository-url=... --revision-range=$(($2-1)):$2 "--description=$(/usr/bin/svnlook log "$1" -r "$2")" \
        "--summary=Commit r$2" "--target-people=$people" --publish
fi
exit 0

And now, as if by magic, all I need to do is set the property reviewboard:autoreview to the list of reviewboard usernames to be reviewers and then any time there is a commit on the folder with the property set, a review is automagically scheduled.

Lovely...

Now all I need to do is tone down how often reviewboard sends emails!

HowTo: CentOS 5, Apache 2.2, Subversion 1.5 with ActiveDirectory Authentication

2008-11-28T09:51:00.007Z

Here's my step by step:

Install CentOS 5.2

Configure Network and Proxies as needed
I usually create a login script: /etc/profile.d/login.sh as follows:

function set_proxies() {
 local s
 PROXY_ADDR="http://proxy.example.com:8000/"
 for s in HTTP HTTPS FTP GOPHER NEWSPOST NEWSREPLY\
          NEWS NNTP SNEWSPOST SNEWSREPLY SNEWS\
          WAIS FINGER CSO; do
     export ${s}_PROXY=${PROXY_ADDR}
 done
    for s in http https ftp; do
     export ${s}_proxy=${PROXY_ADDR}
 done
}

set_proxies

I installed from the DVD, so yum update to ensure everything is up-to-date.
Get my mod_authn_sasl rpm:
wget http://www.one-dash.com/blog/mod_authn_sasl-1.0.2-3.i386.rpm
Install it (this should pull down the):
yum --nogpgcheck localinstall mod_authn_sasl-1.0.2-3.i386.rpm
Get my sasl-magic-config script:
wget http://www.one-dash.com/blog/system-saslauthd-active-directory-config.sh
dos2unix system-saslauthd-active-directory-config.sh
Run the script:
sh system-saslauthd-active-directory-config.sh example.com
Change the security level:
system-config-securitylevel-tui
Set SELinux to Permissive, Customize and enable WWW and Secure WWW on the firewall
Create the sasl2 configuration for apache:
echo "pwcheck_method:saslauthd" > /usr/lib/sasl2/apache-httpd.conf
Now we need to install Subversion 1.5.2:
wget http://summersoft.fay.ar.us/pub/subversion/1.5.2/rhel-5/i386/subversion-1.5.2-1.i386.rpm
wget http://summersoft.fay.ar.us/pub/subversion/1.5.2/rhel-5/i386/neon-0.27.2-1.i386.rpm
wget http://summersoft.fay.ar.us/pub/subversion/1.5.2/rhel-5/i386/mod_dav_svn-1.5.2-1.i386.rpm
yum install perl-URI
rpm -i neon-0.27.2-1.i386.rpm
rpm -i subversion-1.5.2-1.i386.rpm
rpm -i mod_dav_svn-1.5.2-1.i386.rpm

Next modify the apache conf file for our subversion repositories: /etc/httpd/conf.d/subversion.conf

# Needed to do Subversion Apache server.
LoadModule dav_svn_module     modules/mod_dav_svn.so

# Only needed if you decide to do "per-directory" access control.
#LoadModule authz_svn_module   modules/mod_authz_svn.so

<Location /svn>
 DAV svn
 SVNParentPath /var/www/svn

 <IfModule mod_authn_sasl.c>

    # Limit write permission to list of valid users.
    <LimitExcept GET PROPFIND OPTIONS REPORT>
       # Require SSL connection for password protection.
       # SSLRequireSSL
       AuthType Basic
       AuthName "EXAMPLE"
       AuthBasicProvider sasl
       AuthSaslPwcheckMethod saslauthd auxprop
       AuthSaslAppname apache-httpd
       AuthSaslRealm example
       Require valid-user
    </LimitExcept>

 </IfModule>

</Location>

Next, create the subversion projects root and restart Apache:
mkdir /var/www/svn
chown -R apache.apache /var/www/svn
service httpd restart
At this point you can now create a test repository:
svnadmin create /var/www/svn/test
chown -R apache.apache /var/www/svn/test
Let's test if subversion is working:
svn info http://localhost/svn/test
svn mkdir --username myWindowsUsername --message "a test commit" http://localhost/svn/test/trunk
At this point we should have subversion up and running.

Apache 2.2 Authentication with Active Directory via Cyrus SASL

2008-11-27T14:02:00.008Z

Here is my version of the holy grail, i.e.

Authentication for Apache HTTPD against Active Directory.

This is not the only way to skin this cat, for example you could also use Sander Marechal's technique (which uses mod_authnz_ldap). However, my problem with Sander's technique is that you need to have an account in Active Directory which you will use to bind to the LDAP server. That means that the accound password has to be stored in a plain-text file on the Apache server, and if the password expires everything breaks until you go fix the password.

I want as near to zero maintenance as possible, running on CentOS 5.2 with minimal custom work - so that I don't have to maintain it when it does break.

I have previously configured Kerberos to authenicate against Active Directory, so my first attempt was with mod_auth_kerb. That worked... but very slowly... trying to an empty access Subversion repository took over 2 minutes. The problem being that successful authentication was not being cached.

Then I tried using mod_perl and the Authen::Simple::ActiveDirectory reasoning that I could always hack caching once in perl... but getting that lot installed on a CentOS 5.2 from RPMs was an exercise in tears.

So, anyway, I found out that Cyrus SASL supports two modes of LDAP authentication:

The bind method uses the LDAP bind facility to verify the password. The bind method is not available when ldap_use_sasl is turned on. In that case saslauthd will use fastbind.

'bind' is the default auth method. When ldap_use_sasl is enabled, 'fastbind' is the default.

The custom method uses userPassword attribute to verify the password. Suppored hashes: crypt, md5, smd5, sha and ssha. Cleartext is supported as well.

The fastbind method (when 'ldap_use_sasl: no') does away with the search and an extra anonymous bind in auth_bind, but makes two assumptions:
1. Expanding the ldap_filter expression gives the user's
fully-qualified DN
2. There is no cost to staying bound as a named user

So bind is pretty much the same technique as that used by mod_authnz_ldap, and (as ActiveDirectory does not support - at my company at least - anonymous bind) is ruled out of the running.

"fastbind" sounds exactly like what I want... ok and saslauthd can be configured to cache authentication, so none of the performance hit of mod_auth_kerb. All we need is a way to tie this into Apache.

So, enter mod_authn_sasl which does just that.

First, we need to create an RPM, so I did a quick

sudo yum install httpd-devel rpmbuild mock
sudo
echo "%_topdir %(echo $HOME)/rpmbuild" > ~/.rpmmacros
mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}

I added my account to the mock group, logged out and in again for the group membership to be recognised and I was ready to go. This was my first experience playing with mock, but it was lovely, especially as CentOS has its own version.

I pre-primed my mock environments for i386 and x86_64 as I need RPMs for both of these:

mock -r centos-5-i386 init
mock -r centos-5-x86_64 init

Then I downloaded mod_authn_sasl-1.0.2.tar.bz2 and started writing my spec file.

You can download the RPC spec: mod_authn_sasl.spec and the apache config file: mod_authn_sasl.conf.

I put mod_authn_sasl-1.0.2.tar.bz2 and mod_authn_sasl.conf in the ~/rpmbuild/SOURCES and then running

rpmbuild -ba mod_authn_sasl.spec

produced my source RPM, then I used mock to create the two binary RPMs that I needed:

mock -r centos-5-i386 ~/rpmbuild/SRPMS/mod_authn_sasl-1.0.2-4.src.rpm
mock -r centos-5-x86_64 ~/rpmbuild/SRPMS/mod_authn_sasl-1.0.2-4.src.rpm

The rpms built from the mock environments end up in /var/lib/mock/centos-5-i386/result and /var/lib/mock/centos-5-x86_64/result. If you want to be lazy and trust my binaries here they are:

Now all we need to do is configure everything.

I've written this shell script to simplify configuring saslauthd for most good deployments of Active Directory, i.e. where there are SRV records for the domain in your DNS server. You need to run it as root. It looks up the SRV records for the domain name you provide, and creates /etc/saslauthd.conf from them, assuming that the Active Directory domain name is the first word of your DNS name converted to uppercase. So for example, if your Active Directory is set up correctly, and you can login to windows as either EXAMPLE\joebloggs or jbloggs@example.foo.com then you would run

sh system-saslauthd-active-directory-config.sh example.foo.com

And that should setup saslauthd for you. You can test this using the testsaslauthd program.

Then all you need to do is secure the appropriate locations in apache, e.g. by adding the following to your VirtualHost configuration:

<Location /private>
AuthSaslPwcheckMethod saslauthd
AuthSaslAppname httpd
AuthSaslRealm example
AuthType basic
AuthBasicProvider sasl
AuthBasicAuthoritative On
AuthName "sasl@example.com"
require valid-user
</Location>

And define the SASL application provider for the AuthSaslAppname you specified, e.g. for the above configuration you need to create /usr/lib/sasl2/httpd.conf with the contents:

pwcheck_method:saslauthd

That's it. You should be done.

Updated Friday 28th November 2008: I noticed that the spec file I had provided did not have the correct Requires and BuildRequires. I have fixed this and now I have posted updated rpms (these are 1.0.2-4) with the correct requires to help with a minimal install

Acer Aspire One, Huawei E169G and Three IR

2008-10-21T08:39:00.002+01:00

One of my co-workers has asked me to post this up. It's rough and ready, so make of it what you want.

First off, Acer recently pushed an update for better performance with the Huawei USB modems... I'm assuming that you have this update... check if the file /etc/udev/rules.d/10-Huawei-Datacard.rules
exists.

If that file exists then when you plug in a E169g it will be correctly autodetected without requiring poking about with usbmodeswitch... it will bind the three serial ports of the E169G to /dev/HuaweiMobile-0, /dev/HuaweiMobile-1 and /dev/HuaweiMobile-2.

OK, so assuming you see these device nodes after plugging in the E169G, the next problem is getting wvdial to connect. Here's the wvdial.conf file I use


[Dialer Defaults]
Init2 = ATZ
Init3 = ATH
Init4 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Stupid Mode = 1
Modem Type = USB Modem
ISDN = 0
Phone = *99\#
Modem = /dev/HuaweiMobile-0
Username = username
Password = password
Dial Command = ATDT
Baud = 460800
Init5 = AT+CGDCONT=1,"IP","3ireland.ie"

By the way, the username is actually "username", and the password is actually "password".

That should be enough to get any self-respecting linux freak 90% of the way there. There was some stuff I had to tweak to get DHCP to populate resolve.conf from the pppd connection... and I added the following udev rule as 70-huawei-e169g-dial.rules

SUBSYSTEM=="usb" SYSFS{idProduct}=="1001",SYSFS{idVendor}=="12d1",RUN+="/usr/sbin/e169g_dial"

And, /usr/sbin/e169g_dial is just


#!/bin/sh
sleep 5
/usr/bin/wvdial 2>&1 > /var/log/wvdial &

You might be able to tune down from 5 seconds if you can be bothered... but you need at least some delay

Writing a Hudson plugin (Part 7 - Putting it all together)

2008-06-21T10:25:00.010+01:00

Life gets in the way... but we're back with our final installment! So where to start, let's start with a publisher for freestyle builds, then we'll add a publisher for Maven 2 builds... These will both require some reports to display results, and then finally we'll need the plugin entry point. But before we get into all that, perhaps I should briefly explain structured form submission support

DataBoundConstructors

Hudson uses Stapler as it's web framework. One of the things that Stapler provides is support for constructing objects from a JSON data model. Basically, if you have a class with a public constructor annotated with @DataBoundConstructor, Stapler will bind fields from a JSON object by matching the field name to the constructor parameter name. If a parameter also has a @DataBoundConstructor, then Stapler will recurse to construct this child object from the child JSON object.

Note: The only hole in this (at the moment) is if you want to inject a variable class, i.e. it does not support the case where there are three ChildImpl classes all implementing Child, and all with @DataBoundConstructor and Parent's constructor has a parameter which takes Child... However, plans are afoot to fix this!

JavaNCSSPublisher

Publishers in Hudson must have a Descriptor, this will be registered with Hudson and allows Hudson to create Publisher instances which have the details for the project they are publishing. Descriptors are normally implemented as an inner class called DescriptorImpl and there is normally a static field of the publisher DESCRIPTOR that holds the Descriptor singleton. 99.995% of the time, you will want your publisher to have a @DataBoundConstructor, so without further delay, here is the publisher:

package hudson.plugins.javancss;

import hudson.maven.MavenModule;
import hudson.maven.MavenModuleSet;
import hudson.model.AbstractProject;
import hudson.model.Action; 
import hudson.model.Descriptor;
import hudson.plugins.helpers.AbstractPublisherImpl;
import hudson.plugins.helpers.Ghostwriter;
import hudson.tasks.BuildStepDescriptor;
import hudson.tasks.Publisher;
import net.sf.json.JSONObject;
import org.kohsuke.stapler.DataBoundConstructor;
import org.kohsuke.stapler.StaplerRequest;

public class JavaNCSSPublisher extends AbstractPublisherImpl {

    private String reportFilenamePattern;

    @DataBoundConstructor
    public JavaNCSSPublisher(String reportFilenamePattern) {
        reportFilenamePattern.getClass();
        this.reportFilenamePattern = reportFilenamePattern;
    }

    public String getReportFilenamePattern() {
        return reportFilenamePattern;
    }

    public boolean needsToRunAfterFinalized() {
        return false;
    }

    public static final DescriptorImpl DESCRIPTOR = new DescriptorImpl();

    public Descriptor<Publisher> getDescriptor() {
        return DESCRIPTOR;
    }

    public Action getProjectAction(AbstractProject<?, ?> project) {
        return new JavaNCSSProjectIndividualReport(project);
    }

    protected Ghostwriter newGhostwriter() {
        return new JavaNCSSGhostwriter(reportFilenamePattern);
    }

    public static final class DescriptorImpl extends BuildStepDescriptor<Publisher> {

        private DescriptorImpl() {
            super(JavaNCSSPublisher.class);
        }

        public String getDisplayName() {
            return "Publish " + PluginImpl.DISPLAY_NAME;
        }

        public boolean isApplicable(Class<? extends AbstractProject> aClass) {
            return !MavenModuleSet.class.isAssignableFrom(aClass)
                    && !MavenModule.class.isAssignableFrom(aClass);
        }

    }

}

By inheriting from AbstractPublisherImpl we get a lot of the work done for us, all we really need to do is provide a Ghostwriter and the project level report (JavaNCSSProjectIndividualReport which we will see later

We need hudson/plugins/javancss/JavaNCSSPublisher/config.jelly to allow the user to specify the report file name pattern... not much to this, so here it is:

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form" xmlns:bh="/lib/health">
    <f:entry title="JavaNCSS xml report pattern"
             description="
             This is a file name pattern that can be used to locate the JavaNCSS xml report files
             (for example with Maven2 use &lt;b&gt;**/target/javancss-raw-report.xml&lt;/b&gt;).
           The path is relative to &lt;a href='ws/'&gt;the module root&lt;/a&gt; unless
           you are using Subversion as SCM and have configured multiple modules, in which case it is
           relative to the workspace root.&lt;br/&gt;
           JavaNCSS must be configured to generate XML reports for this plugin to function.
           ">
        <f:textbox name="javancss.reportFilenamePattern" value="${instance.reportFilenamePattern}"/>
    </f:entry>
</j:jelly>

JavaNCSSMavenPublisher

This is fairly similar to the freestyle publisher, except that we do not need the user to configure everything for us, as we can grab some of the stuff from the pom.xml.

We could, if necessary, tweak the pom.xml to ensure that the report we are looking for is generated... an example of this is the cobertura maven plugin which does not generate an XML report by default. A Hudson plugin can modify the cobertura plugin's configuration before it executes in order to ensure that the xml report is generated. Note: some people regard this kind of thing as evil, as the pom.xml is no longer behaving the same as when run from the command line.

Ok, so here is the Maven publisher...

package hudson.plugins.javancss;

import hudson.maven.*;
import hudson.model.Action;
import hudson.plugins.helpers.AbstractMavenReporterImpl;
import hudson.plugins.helpers.Ghostwriter;
import net.sf.json.JSONObject;
import org.apache.maven.project.MavenProject;
import org.codehaus.plexus.component.configurator.ComponentConfigurationException;
import org.kohsuke.stapler.DataBoundConstructor;
import org.kohsuke.stapler.StaplerRequest;

import java.io.File;

public class JavaNCSSMavenPublisher extends AbstractMavenReporterImpl {

    @DataBoundConstructor
    public JavaNCSSMavenPublisher() {
    }

    private static final String PLUGIN_GROUP_ID = "org.codehaus.mojo";
    private static final String PLUGIN_ARTIFACT_ID = "javancss-maven-plugin";
    private static final String PLUGIN_EXECUTE_GOAL = "report";

    protected boolean isExecutingMojo(MojoInfo mojo) {
        return mojo.pluginName.matches(PLUGIN_GROUP_ID, PLUGIN_ARTIFACT_ID)
                && PLUGIN_EXECUTE_GOAL.equals(mojo.getGoal());
    }

    protected Ghostwriter newGhostwriter(MavenProject pom, MojoInfo mojo) {
        // get the name of the xml report 

        String tempFileName;
        try {
            tempFileName = mojo.getConfigurationValue("tempFileName", String.class);
        } catch (ComponentConfigurationException e) {
            tempFileName = null;
        }
        if (tempFileName == null) {
            // the name was not overridden in the pom, so use the default
            tempFileName = "javancss-raw-report.xml";
        }

        // get the xml output directory

        File baseDir = pom.getBasedir().getAbsoluteFile();
        File xmlOutputDirectory;
        try {
            xmlOutputDirectory = mojo.getConfigurationValue("xmlOutputDirector", File.class);
        } catch (ComponentConfigurationException e) {
            xmlOutputDirectory = null;
        }
        if (xmlOutputDirectory == null) {
            // the directory was not overridden in the pom, so use the default
            xmlOutputDirectory = new File(pom.getBuild().getDirectory());
        }

        String searchPath;
        String targetPath = makeDirEndWithFileSeparator(fixFilePathSeparator(xmlOutputDirectory.getAbsolutePath()));
        String baseDirPath = makeDirEndWithFileSeparator(fixFilePathSeparator(baseDir.getAbsolutePath()));

        if (targetPath.startsWith(baseDirPath)) {
            searchPath = targetPath.substring(baseDirPath.length()) + tempFileName;
        } else {
            // we have no clue where this is, so default to anywhere
            searchPath = "**/" + tempFileName;
        }

        return new JavaNCSSGhostwriter(searchPath, targets);
    }

    private String makeDirEndWithFileSeparator(String baseDirPath) {
        if (!baseDirPath.endsWith(File.separator)) {
            baseDirPath += File.separator;
        }
        return baseDirPath;
    }

    private String fixFilePathSeparator(String path) {
        return path.replace(File.separatorChar == '/' ? '\\' : '/', File.separatorChar);
    }

    public Action getProjectAction(MavenModule module) {
        for (MavenBuild build : module.getBuilds()) {
            if (build.getAction(JavaNCSSBuildIndividualReport.class) != null) {
                return new JavaNCSSProjectIndividualReport(module);
            }
        }
        return null;
    }

    public static final DescriptorImpl DESCRIPTOR = new DescriptorImpl();

    /**
     * {@inheritDoc}
     */
    public MavenReporterDescriptor getDescriptor() {
        return DESCRIPTOR;  //To change body of implemented methods use File | Settings | File Templates.
    }

    public static final class DescriptorImpl extends MavenReporterDescriptor {

        /**
         * Do not instantiate DescriptorImpl.
         */
        private DescriptorImpl() {
            super(JavaNCSSMavenPublisher.class);
        }

        /**
         * {@inheritDoc}
         */
        public String getDisplayName() {
            return "Publish " + PluginImpl.DISPLAY_NAME;
        }

        public MavenReporter newInstance(StaplerRequest req, JSONObject formData) throws FormException {
            return req.bindJSON(JavaNCSSMavenPublisher.class, formData);
        }

    }

}

The only complexity is in the newGhostwriter method, where we have to find out what the configuration for the maven plugin is in order to find the xml report.

We will need a hudson/plugins/javancss/JavaNCSSMavenPublisher/config.jelly file for this publisher... not much to this one as we get what we need from the pom.xml

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form" xmlns:bh="/lib/health">
</j:jelly>

The reports

We have a total of four reports to generate:

Individual build report - used for freestyle and maven 2 modules
Individual project report - used for freestyle and maven 2 modules
Aggregated build report - used for maven 2 projects
Aggregated project report - used for maven 2 projects

To keep to our DRY principals, we'll use some abstract classes to pull together the common code. First, AbstractBuildReport which will form the basis of our build reports:

package hudson.plugins.javancss;

import hudson.model.AbstractBuild;
import hudson.plugins.helpers.AbstractBuildAction;
import hudson.plugins.javancss.parser.Statistic;
import org.kohsuke.stapler.StaplerRequest;
import org.kohsuke.stapler.StaplerResponse;

import java.io.IOException;
import java.util.Collection;

public abstract class AbstractBuildReport<T extends AbstractBuild<?, ?>> extends AbstractBuildAction<T> {
    private final Collection<Statistic> results;
    private final Statistic totals;

    public AbstractBuildReport(Collection<Statistic> results) {
        this.results = results;
        this.totals = Statistic.total(results);
    }

    public Collection<Statistic> getResults() {
        return results;
    }

    public Statistic getTotals() {
        return totals;
    }

    public String getSummary() {
        AbstractBuild<?, ?> prevBuild = getBuild().getPreviousBuild();
        while (prevBuild != null && prevBuild.getAction(getClass()) == null) {
            prevBuild = prevBuild.getPreviousBuild();
        }
        if (prevBuild == null) {
            return totals.toSummary();
        } else {
            AbstractBuildReport action = prevBuild.getAction(getClass());
            return totals.toSummary(action.getTotals());
        }
    }

    public String getIconFileName() {
        return PluginImpl.ICON_FILE_NAME;
    }

    public String getDisplayName() {
        return PluginImpl.DISPLAY_NAME;
    }

    public String getUrlName() {
        return PluginImpl.URL;
    }

    public boolean isGraphActive() {
        return false;
    }

}

Similarly, we have AbstractProjectReport which will be used for project reports:

package hudson.plugins.javancss;

import java.io.IOException;
import java.util.Collection;
import java.util.Collections;

import hudson.model.AbstractBuild;
import hudson.model.AbstractProject;
import hudson.model.ProminentProjectAction;
import hudson.plugins.helpers.AbstractProjectAction;
import hudson.plugins.javancss.parser.Statistic;
import org.kohsuke.stapler.StaplerRequest;
import org.kohsuke.stapler.StaplerResponse;

public abstract class AbstractProjectReport<T extends AbstractProject<?, ?>> 
        extends AbstractProjectAction<T>
        implements ProminentProjectAction {

    public AbstractProjectReport(T project) {
        super(project);
    }

    public String getIconFileName() {
        for (AbstractBuild<?, ?> build = getProject().getLastBuild(); 
                build != null; build = build.getPreviousBuild()) {

            final AbstractBuildReport action = build.getAction(getBuildActionClass());
            if (action != null) {
                return PluginImpl.ICON_FILE_NAME;
            }
        }
        return null;
    }

    public String getDisplayName() {
        for (AbstractBuild<?, ?> build = getProject().getLastBuild(); 
                build != null; build = build.getPreviousBuild()) {
            final AbstractBuildReport action = build.getAction(getBuildActionClass());
            if (action != null) {
                return PluginImpl.DISPLAY_NAME;
            }
        }
        return null;
    }

    public String getUrlName() {
        for (AbstractBuild<?, ?> build = getProject().getLastBuild(); build != null; build = build.getPreviousBuild()) {
            final AbstractBuildReport action = build.getAction(getBuildActionClass());
            if (action != null) {
                return PluginImpl.URL;
            }
        }
        return null;
    }

    public String getSearchUrl() {
        return PluginImpl.URL;
    }

    public boolean isGraphActive() {
        return false;
    }

    public Collection<Statistic> getResults() {
        for (AbstractBuild<?, ?> build = getProject().getLastBuild(); 
                build != null; build = build.getPreviousBuild()) {
            final AbstractBuildReport action = build.getAction(getBuildActionClass());
            if (action != null) {
                return action.getResults();
            }
        }
        return Collections.emptySet();
    }

    public Statistic getTotals() {
        for (AbstractBuild<?, ?> build = getProject().getLastBuild(); 
                build != null; build = build.getPreviousBuild()) {
            final AbstractBuildReport action = build.getAction(getBuildActionClass());
            if (action != null) {
                return action.getTotals();
            }
        }
        return null;
    }

    protected abstract Class<? extends AbstractBuildReport> getBuildActionClass();

}

Now that we have these abstract classes, we can roll out our concrete reports. First the individual build report:

package hudson.plugins.javancss;

import java.util.Collection;
import java.util.List;
import java.util.Map;

import hudson.maven.AggregatableAction;
import hudson.maven.MavenAggregatedReport;
import hudson.maven.MavenBuild;
import hudson.maven.MavenModule;
import hudson.maven.MavenModuleSetBuild;
import hudson.model.AbstractBuild;
import hudson.plugins.javancss.parser.Statistic;

public class JavaNCSSBuildIndividualReport extends AbstractBuildReport<AbstractBuild<?, ?>>
        implements AggregatableAction {

    public JavaNCSSBuildIndividualReport(Collection<Statistic> results) {
        super(results);
    }

    @Override
    public synchronized void setBuild(AbstractBuild<?, ?> build) {
        super.setBuild(build);
        if (this.getBuild() != null) {
            for (Statistic r : getResults()) {
                r.setOwner(this.getBuild());
            }
        }
    }

    public MavenAggregatedReport createAggregatedAction(MavenModuleSetBuild build,
            Map<MavenModule, 
            List<MavenBuild>> moduleBuilds) {
        return new JavaNCSSBuildAggregatedReport(build, moduleBuilds);
    }

}

That was fairly painless... Note that we interfaces for both the freestyle and maven2 project types, this is OK as the freestyle projects will ignore the Maven2 stuff and vice-versa while the common code is shared by both. Next we need the aggregated build report:

package hudson.plugins.javancss;

import hudson.maven.*;
import hudson.model.Action;
import hudson.plugins.javancss.parser.Statistic;

import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.Map;

public class JavaNCSSBuildAggregatedReport 
        extends AbstractBuildReport<MavenModuleSetBuild> 
        implements MavenAggregatedReport {

    public JavaNCSSBuildAggregatedReport(MavenModuleSetBuild build, 
            Map<MavenModule, List<MavenBuild>> moduleBuilds) {
        super(new ArrayList<Statistic>());
        setBuild(build);
    }

    public synchronized void update(Map<MavenModule, List<MavenBuild>> moduleBuilds, 
            MavenBuild newBuild) {
        JavaNCSSBuildIndividualReport report = 
                newBuild.getAction(JavaNCSSBuildIndividualReport.class);

        if (report != null) {
            Collection<Statistic> u = Statistic.merge(report.getResults(), getResults());
            getResults().clear();
            getResults().addAll(u);
            getTotals().add(report.getTotals());
        }
    }

    public Class<? extends AggregatableAction> getIndividualActionType() {
        return JavaNCSSBuildIndividualReport.class;
    }

    public Action getProjectAction(MavenModuleSet moduleSet) {
        for (MavenModuleSetBuild build : moduleSet.getBuilds()) {
            if (build.getAction(JavaNCSSBuildAggregatedReport.class) != null) {
                return new JavaNCSSProjectAggregatedReport(moduleSet);
            }
        }
        return null;
    }

}

This report is only used for the Maven2 project types. The two key methods are update which is called as each module completes, and getProjectAction which should return the project level aggregated report if there is a report to show. At this point we're ready for the individual project report:

package hudson.plugins.javancss;

import hudson.model.AbstractProject;
import hudson.model.Actionable;
import hudson.model.ProminentProjectAction;
import hudson.model.AbstractBuild;
import hudson.util.ChartUtil;
import hudson.util.DataSetBuilder;
import hudson.plugins.javancss.parser.Statistic;

import java.util.Collection;

public class JavaNCSSProjectIndividualReport 
        extends AbstractProjectReport<AbstractProject<?, ?>> 
        implements ProminentProjectAction {

    public JavaNCSSProjectIndividualReport(AbstractProject<?, ?> project) {
        super(project);
    }

    protected Class<? extends AbstractBuildReport> getBuildActionClass() {
        return JavaNCSSBuildIndividualReport.class;
    }
}

Don't repeat ourselves comes in handy here as essentially all the work has been done for us!. The project aggregated report:

package hudson.plugins.javancss;

import hudson.model.Actionable;
import hudson.model.ProminentProjectAction;
import hudson.model.AbstractBuild;
import hudson.model.Action;
import hudson.maven.MavenModuleSet;
import hudson.maven.MavenModuleSetBuild;
import hudson.plugins.javancss.parser.Statistic;

public class JavaNCSSProjectAggregatedReport 
        extends AbstractProjectReport<MavenModuleSet> 
        implements ProminentProjectAction {

    public JavaNCSSProjectAggregatedReport(MavenModuleSet project) {
        super(project);
    }

    protected Class<? extends AbstractBuildReport> getBuildActionClass() {
        return JavaNCSSBuildAggregatedReport.class;
    }
}

Again DRY to the rescue... At this point all that remains is to present the reports from these backing objects... so on with the jelly views. The helper classes and our inheritance makes this easy... all we need is two jelly files: hudson/plugins/javancss/AbstractBuildReport/reportDetail.jelly and hudson/plugins/javancss/AbstractProjectReport/reportDetail.jelly. Here they are:

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
    <h1>Results</h1>

    <table border="1px" class="pane sortable">
        <thead>
            <tr>
                <th>Package</th>
                <th title="Class count">Classes</th>
                <th title="Function count">Functions</th>
                <th title="Javadoc count">Javadocs</th>
                <th title="Non-commenting Source Statements">NCSS</th>
                <th title="Javadoc line count">JLC</th>
                <th title="Single-line comment line count">SLCLC</th>
                <th title="Multi-line comment line count">MLCLC</th>
            </tr>
        </thead>
        <tfoot>
            <tr>
                <th align="left">Totals</th>
                <th align="right">${it.totals.classes}</th>
                <th align="right">${it.totals.functions}</th>
                <th align="right">${it.totals.javadocs}</th>
                <th align="right">${it.totals.ncss}</th>
                <th align="right">${it.totals.javadocLines}</th>
                <th align="right">${it.totals.singleCommentLines}</th>
                <th align="right">${it.totals.multiCommentLines}</th>
            </tr>
        </tfoot>
        <tbody>
            <j:forEach var="r" items="${it.results}">
                <tr>
                    <th align="left">${r.name}</th>
                    <td align="right">${r.classes}</td>
                    <td align="right">${r.functions}</td>
                    <td align="right">${r.javadocs}</td>
                    <td align="right">${r.ncss}</td>
                    <td align="right">${r.javadocLines}</td>
                    <td align="right">${r.singleCommentLines}</td>
                    <td align="right">${r.multiCommentLines}</td>
                </tr>
            </j:forEach>
        </tbody>
    </table>
</j:jelly>

Yep, the two files are identical! Other plugins may not be quite so lucky... but in general the project level report should be the same as the report for the latest build

Making a plugin

Now we are ready to make our plugin.... for this we need a class that extends hudson.Plugin and registers our publisher's descriptors with the appropriate lists... here it is:

package hudson.plugins.javancss;

import hudson.Plugin;
import hudson.maven.MavenReporters;
import hudson.tasks.BuildStep;

public class PluginImpl extends Plugin {
    public void start() throws Exception {
        BuildStep.PUBLISHERS.add(JavaNCSSPublisher.DESCRIPTOR);
        MavenReporters.LIST.add(JavaNCSSMavenPublisher.DESCRIPTOR);
    }

    public static String DISPLAY_NAME = "Java NCSS Report";
    public static String GRAPH_NAME = "Java NCSS Trend";
    public static String URL = "javancss";
    public static String ICON_FILE_NAME = "graph.gif";
}

And that's pretty much it... we should have a working plugin

Finishing touches

OK, so the plugin does not have health reports (i.e. the weather icons) and it does not show a trend graph... I think I'm going to need a part 8 :-(

More praise for Hudson

2008-04-14T22:54:00.002+01:00

Over on the Maven2 Users list a recent poster asked what CI server was best, and Hudson was the only answer.

Then Jason van Zyl posted this praise for Hudson:

I know from my vantage point Hudson is the only system I will provide commercial support for at Sonatype because the battle is over. Hudson won by making developers lives' easier. Kohsuke will go to no end to make things easier for users.
...
At any rate I guarantee you that inside 3 months Hudson will have the best Maven integration of any CI/Build Server there is.

I'm personally biased towards Hudson, but I have to agree that it has won the war of the CI servers. I don't see anything coming close in ease of use or speed of startup.

Writing a Hudson plugin (Part 6 - Parsing the results)

2008-04-14T12:13:00.006+01:00

In some ways parsing the JavaNCSS results is the least interesting part of developing a Hudson plugin, as once I have implemented the parser, it is available for everyone. For that reason I will focus more on:

best practice techniques for parsing results
common gotchas
designing for extension

Getting started

First off, we need to analyse the results file format. In the case of JavaNCSS there are multiple ways that the results file can be generated: from the JavaNCSS program directly, from ANT or from Maven. This leads us onto gotcha #1

Gotcha #1:
Never assume that a build tool generates the same format of output when run from the command line, ANT or Maven.
A case in point for Gotcha #1 is Findbugs which generates one XML format from the command line and ANT, and generates a different format that appears similar at first glance when run from Maven (mail thread). In this case it turns out that Maven 1 used the different format output, and it is feared that some people came to depend on this Maven 1 format, so when the plugin for Maven 2 was developed, they kept the Maven 1 format. In any case, the moral is don’t assume, check!

So we use the sample projects from Part 1 and generate an ANT and a Maven 2 XML report. First off, here is the report from ANT:

<?xml version="1.0"?>
<javancss>
<date>2008-04-12</date>
<time>11:22:30</time>
<packages>
 <package>
   <name>com.onedash.common</name>
   <classes>1</classes>
   <functions>3</functions>
   <ncss>10</ncss>
   <javadocs>3</javadocs>
   <javadoc_lines>12</javadoc_lines>
   <single_comment_lines>0</single_comment_lines>
   <multi_comment_lines>0</multi_comment_lines>
 </package>
 <package>
    ...
 </package>
 ...
 <total>
   <classes>5</classes>
   <functions>8</functions>
   <ncss>46</ncss>
   <javadocs>9</javadocs>
   <javadoc_lines>37</javadoc_lines>
   <single_comment_lines>0</single_comment_lines>
   <multi_comment_lines>0</multi_comment_lines>
 </total>
 <table>
   <tr><td>Packages</td><td>Classes</td><td>Functions</td><td>NCSS</td><td>Javadocs</td><td>per</td></tr>
   <tr><td>4.00</td><td>5.00</td><td>8.00</td><td>46.00</td><td>9.00</td><td>Project</td></tr>
   <tr><td></td><td>1.25</td><td>2.00</td><td>11.50</td><td>2.25</td><td>Package</td></tr>
   <tr><td></td><td></td><td>1.60</td><td>9.20</td><td>1.80</td><td>Class</td></tr>
   <tr><td></td><td></td><td></td><td>5.75</td><td>1.13</td><td>Function</td></tr>
 </table>
</packages>

<objects>
 <object>
   <name>com.onedash.common.Factory</name>
   <ncss>7</ncss>
   <functions>3</functions>
   <classes>0</classes>
   <javadocs>3</javadocs>
 </object>
 <object>
   ...
 </object>
 ...
 <averages>
   <ncss>6.60</ncss>
   <functions>1.60</functions>
   <classes>0.00</classes>
   <javadocs>1.80</javadocs>
 </averages>
 <ncss>46.00</ncss>
</objects>

<functions>
 <function>
   <name>com.onedash.common.Factory.Factory()</name>
   <ncss>1</ncss>
   <ccn>1</ccn>
   <javadocs>1</javadocs>
 </function>
 <function>
   ...
 </function>
 ...
 <ncss>46.00</ncss>
</functions>
</javancss>

OK, first off, for those following the tutorial exactly, I have cheated a little. I added some more source files into the project to make sure that I have multiple classes is different packages. You can see the source code I built from here. Additionally, I have trimmed the output somewhat to highlight the interesting bits, removing the duplicate entries.

From this report file we can see a basic XML structure:

The root element is <javancss> and has child elements: <date>, <time>, <packages>, <objects>, and <functions>
The <date> and <time> elements are the timestamp when the report was generated with the date in YYYY-MM-DD format and the time in HH:MM:SS format
The <packages> element has child elements: <package>, <total>, and <table>. There are multiple <package>; elements, but only one <total> and <table> element.
- The <package> elements have child elements: <name>, <classes>, <functions>, <ncss>, <javadocs>, <javadoc_lines>, <single_comment_lines> and <multi_comment_lines>. The <name> element contains the name of the package as a String and the other elements contain totals as Integers.
- The <total> element has child elements: <classes>, <functions>, <ncss>, <javadocs>, <javadoc_lines>, <single_comment_lines> and <multi_comment_lines>. These elements are the sum of all the corresponding <package> children inside the <packages> parent
- The <table> element seems to be a HTML table.
The <objects> element has child elements: <object>, <averages> and <ncss>. There are multiple <object> elements, the <averages> element contains the average results for all the <object> elements and the <ncss> element providing some form of total or average.
The <functions> element has child elements: <function> and <ncss>. Again there are multiple <function> elements with the <ncss> element providing some form of total or average (interestingly the result appears to be the same as from <objects>).

Now, let’s take a look at what Maven 2 gives us:

<?xml version="1.0"?>
<?xml version="1.0"?>
<javancss>
<date>2008-04-12</date>
<time>11:43:06</time>
<packages>
 <package>
   <name>com.onedash.common</name>
   <classes>1</classes>
   <functions>3</functions>
   <ncss>10</ncss>
   <javadocs>3</javadocs>
   <javadoc_lines>12</javadoc_lines>
   <single_comment_lines>0</single_comment_lines>
   <multi_comment_lines>0</multi_comment_lines>
 </package>
 <package>
   ...
 </package>
 ...
 <total>
   <classes>5</classes>
   <functions>8</functions>
   <ncss>46</ncss>
   <javadocs>10</javadocs>
   <javadoc_lines>42</javadoc_lines>
   <single_comment_lines>3</single_comment_lines>
   <multi_comment_lines>3</multi_comment_lines>
 </total>
 <table>
   <tr><td>Packages</td><td>Classes</td><td>Functions</td><td>NCSS</td><td>Java
   <tr><td>4.00</td><td>5.00</td><td>8.00</td><td>46.00</td><td>10.00</td><td>P
   <tr><td></td><td>1.25</td><td>2.00</td><td>11.50</td><td>2.50</td><td>Packag
   <tr><td></td><td></td><td>1.60</td><td>9.20</td><td>2.00</td><td>Class</td><
   <tr><td></td><td></td><td></td><td>5.75</td><td>1.25</td><td>Function</td></
 </table>
</packages>

<objects>
 <object>
   <name>com.onedash.common.api.Namer</name>
   <ncss>2</ncss>
   <functions>1</functions>
   <classes>0</classes>
   <javadocs>1</javadocs>
 </object>
 <object>
   ...
 </object>
 ...
 <averages>
   <ncss>6.60</ncss>
   <functions>1.60</functions>
   <classes>0.00</classes>
   <javadocs>2.00</javadocs>
 </averages>
 <ncss>46.00</ncss>
</objects>

<functions>
 <function>
   <name>com.onedash.common.api.Namer.newName()</name>
   <ncss>1</ncss>
   <ccn>1</ccn>
   <javadocs>0</javadocs>
 </function>
 <function>
   ...
 </function>
 <ncss>46.00</ncss>
</functions>
</javancss>

Thankfully, this is the same format as for ANT. You will also be relieved to know that this is the format generated by the JavaNCSS program directly. Thus we only have to write one parser, and we do not have to detect what format we are parsing. But before I forget:

Best Practice #1:
When there are multiple formats of a report generated by different tools, make sure that your Hudson plugin can detect the different formats and can handle them appropriately (by either delegating to a different parser implementation, or by handling the differences on the fly).
One of the goals of Hudson is to minimise configuration. So when a plugin can detect an configuration option automatically, it should detect it automatically (possibly providing an “Advanced” option button to let users override the detection if Hudson gets it wrong)

Start small

Looking at the JavaNCSS output, I see that there is a lot of information... and I only have one more Part left in this series! So I am not going to parse everything. I am sure that in the future I will extend the Hudson plugin to parse all of the file, but for now I am just going to concentrate on the <pacakages> element. This gives users something useful and it’s better than nothing.

But what happens when I do get around to parsing the <objects> and <functions> elements? People may have lots of old builds and they will want to see the trends of the <objects> and <functions> results. I have two choices:

Tell them “Sorry, out of luck”
Save the results with the build, and then the newer parser can extract the results when people want the trend.

Choosing between these two options can be difficult. My preference is to go with option two, as long as the results are not a really big file.

Best Practice #2:
If you are not parsing everything in the results file, and the results file is not too big, and it can be parsed without reference to the source code, copy the results file to Hudson so that future versions of your plugin can read the information you are not currently parsing.

Don’t over parse

The results that we parse are going to be placed into an Action object. This Action object will be serialized. When Hudson starts up, it reads all the results of all the builds. If we place too much information in our Action object, this can have a detrimental effect on Hudson’s performance. When users have 50+ projects each with a couple of hundred builds, they will thank you for keeping your Action objects small.

Gotcha #2:
Don’t store too much in your Action objects.

Don’t under parse

OK, so I have just given out about storing too much in your Action objects. There is a second problem... not storing enough! Most reporting plugins try to present a trend graph to show progress over a number of builds. If we don’t store the information required to generate this trend graph inside our Action objects, then displaying the trend graph will require parsing all the result files for all builds of a project. This can have a detrimental effect on Hudson’s performance. When users have projects with a couple of hundred builds, they will thank you for keeping the information to generate the main trend graph inside your Action objects.

Gotcha #3:
Store the information for generating the Project level trend graph in your Action objects.
A case in point for Gotach #3 is the cobertura plugin, which at the time of writing, does not store the information for the main trend graph in the Action object. I fully intend to fix this situation once I have finished this series!

How to parse

Most of the result files that you will encounter are XML based. We are writing our plugins in Java, so that gives us a range of parsers to choose from, e.g.

SAX
DOM
StAX
Roll your own
Etc.

Given that report files can end up very big for very big projects, we need to be careful how we parse the results:

Gotcha #4:
Don’t parse XML results using DOM, as this will require reading the entire report file into memory.

I am going to stick my neck out and make a recommendation:

Best Practice #3:
Use an XML pull parser to parse XML report files.
They are generally faster, use less memory, and are better suited to a “hit-and-run” style of result extraction.

Be able to aggregate parsing results

You may think that there will only ever be one result file that you need to parse. Maven 2 usually throws a spanner into that model, and everyone has their own ANT build script, so:

Gotcha #5:
Don’t assume you only have to parse one report file for each project.
This gotcha arrives from the code coverage plugins (emma, clover, cobertura). Initially, you would think that people are only interested in one code coverage result, i.e. the coverage for the project... so they will only have one result file that we need to parse, right? Wrong! Some tools/build scripts generate a report for each module but only generate a summary report in non-conforming HTML. Some tools / build scripts generate a report for unit tests and integration tests separately. It’s a mess, and don’t get me started on using different tools for different test types...

The parsing engine

Ok, so here is the parsing engine:

package hudson.plugins.javancss.parser;

import hudson.model.AbstractBuild;
import hudson.util.IOException2;
import org.xmlpull.v1.XmlPullParser;
import org.xmlpull.v1.XmlPullParserException;
import org.xmlpull.v1.XmlPullParserFactory;

import java.io.*;
import java.util.*;

public class Statistic implements Serializable {

 private AbstractBuild<?, ?> owner;
 private String name;
 private long classes;
 private long functions;
 private long ncss;
 private long javadocs;
 private long javadocLines;
 private long singleCommentLines;
 private long multiCommentLines;

 public static Collection<Statistic> parse(File inFile)
         throws IOException, XmlPullParserException {
     Collection<Statistic> results = new ArrayList<Statistic>();
     FileInputStream fis = null;
     BufferedInputStream bis = null;
     try {
         fis = new FileInputStream(inFile);
         bis = new BufferedInputStream(fis);
         XmlPullParserFactory factory = XmlPullParserFactory.newInstance();
         factory.setNamespaceAware(true);
         factory.setValidating(false);
         XmlPullParser parser = factory.newPullParser();
         parser.setInput(bis, null);

         // check that the first tag is <javancss>
         expectNextTag(parser, "javancss");

         // skip until we get to the <packages> tag
         while (parser.getDepth() > 0
                && (parser.getEventType() != XmlPullParser.START_TAG
                    || !"packages".equals(parser.getName()))) {
             parser.next();
         }
         while (parser.getDepth() > 0
                && (parser.getEventType() != XmlPullParser.START_TAG
                    || !"package".equals(parser.getName()))) {
             parser.next();
         }
         while (parser.getDepth() >= 2
                && parser.getEventType() == XmlPullParser.START_TAG
                && "package".equals(parser.getName())) {
             Map<String, String> data = new HashMap<String, String>();
             String lastTag = null;
             String lastText = null;
             int depth = parser.getDepth();
             while (parser.getDepth() >= depth) {
                 parser.next();
                 switch (parser.getEventType()) {
                     case XmlPullParser.START_TAG:
                         lastTag = parser.getName();
                         break;
                     case XmlPullParser.TEXT:
                         lastText = parser.getText();
                         break;
                     case XmlPullParser.END_TAG:
                         if (parser.getDepth() == 4
                             && lastTag != null
                             && lastText != null) {
                             data.put(lastTag, lastText);
                         }
                         lastTag = null;
                         lastText = null;
                         break;
                 }
             }
             if (data.containsKey("name")) {
                 Statistic s = new Statistic(data.get("name"));
                 s.setClasses(Long.valueOf(data.get("classes")));
                 s.setFunctions(Long.valueOf(data.get("functions")));
                 s.setNcss(Long.valueOf(data.get("ncss")));
                 s.setJavadocs(Long.valueOf(data.get("javadocs")));
                 s.setJavadocLines(Long.valueOf(data.get("javadoc_lines")));
                 s.setSingleCommentLines(Long.valueOf(data.get("single_comment_lines")));
                 s.setMultiCommentLines(Long.valueOf(data.get("multi_comment_lines")));
                 results.add(s);
             }
             parser.next();
         }
     } catch (XmlPullParserException e) {
         throw new IOException2(e);
     } finally {
         if (bis != null) {
             bis.close();
         }
         if (fis != null) {
             fis.close();
         }
     }
     return results;
 }

 private static void skipTag(XmlPullParser parser)
         throws IOException, XmlPullParserException {
     parser.next();
     endElement(parser);
 }

 private static void expectNextTag(XmlPullParser parser, String tag)
         throws IOException, XmlPullParserException {
     while (true) {
         if (parser.getEventType() != XmlPullParser.START_TAG) {
             parser.next();
             continue;
         }
         if (parser.getName().equals(tag)) {
             return;
         }
         throw new IOException("Expecting tag " + tag);
     }
 }

 private static void endElement(XmlPullParser parser)
         throws IOException, XmlPullParserException {
     int depth = parser.getDepth();
     while (parser.getDepth() >= depth) {
         parser.next();
     }
 }

 public Statistic(String name) {
     this.name = name;
 }

 ...

 // Simple getters and setters for all the private fields

 ...

 // equals based on all private fields, hashCode based on
 // name and owner.

 ...

 // toString

 ...
}

Essentially the main work is done in the static parse method. It takes a File and tries to parse it. We get an XML Pull Parser for the stream and ensure that it is neither namespace aware nor validating as the file format does not use namespaces and we will be forgiving on the XML format.

The first tag should be <javancss> and after that we skip until we get a <pacakages> tag. Once we have found the <packages> tag we skip until we hit the first <package> tag.

We are reverse engineering the JavaNCSS file format, so we will not make any assumptions about the order of the child elements in the <package> element. We put all the child elements into a Map keyed by the element name, and then when we reach the end of the <package> element we pull out the information we were after from the Map and put it into a Statistic object and add that to the collection of results that we will return.

As soon as we hit the end of the <packages> element, we stop parsing.

Supporting aggregation

In order to support aggregation of multiple results, we'll add some utility methods to the Statistic class, first we need methods that allow us to calculate totals:

package hudson.plugins.javancss.parser;

...

public class Statistic implements Serializable {

 ...

 public static Statistic total(Collection<Statistic>... results) {
     Collection<Statistic> merged = merge(results);
     Statistic total = new Statistic("");
     for (Statistic individual : merged) {
         total.add(individual);
     }
     return total;
 }

 public void add(Statistic r) {
     classes += r.classes;
     functions += r.functions;
     ncss += r.ncss;
     javadocs += r.javadocs;
     javadocLines += r.javadocLines;
     singleCommentLines += r.singleCommentLines;
     multiCommentLines += r.multiCommentLines;
 }

 ...

}

The total method just calculates the total of all the statistics in a collection of statistics. We will also need to be able to merge different result sets. This should aggregate totals for each package separately and return a collection with one total statistic for each package:

package hudson.plugins.javancss.parser;

...

public class Statistic implements Serializable {

 ...

 public static Collection<Statistic> merge(
         Collection<Statistic>... results) {
     if (results.length == 0) {
         return Collections.emptySet();
     } else if (results.length == 1) {
         return results[0];
     } else {
         Map<String, Statistic> merged =
                 new HashMap<String, Statistic>();
         for (Collection<Statistic> result : results) {
             for (Statistic individual : result) {
                 if (!merged.containsKey(individual.name)) {
                     merged.put(individual.name,
                             new Statistic(individual.name));
                 }
                 merged.get(individual.name).add(individual);
             }
         }
         return merged.values();
     }
 }

 ...

}

That is pretty much it for the parser engine.

The Ghostwriter

Now we need to hook the engine into our publisher. We will need to configure the UI elements and the Actions... all tasks for the final part, but for now, we'll just hook it up. We want to run the parsing on the slave side so we implement Ghostwriter.SlaveGhostwriter.

package hudson.plugins.javancss;

import hudson.FilePath;
import hudson.model.AbstractBuild;
import hudson.model.BuildListener;
import hudson.plugins.helpers.BuildProxy;
import hudson.plugins.helpers.Ghostwriter;
import hudson.plugins.javancss.parser.Statistic;
import org.xmlpull.v1.XmlPullParserException;

import java.io.File;
import java.io.IOException;
import java.util.Collection;
import java.util.HashSet;
import java.util.Set;

public class JavaNCSSGhostwriter
     implements Ghostwriter,
     Ghostwriter.SlaveGhostwriter {

 private final String reportFilenamePattern;

 public JavaNCSSGhostwriter(String reportFilenamePattern) {
     this.reportFilenamePattern = reportFilenamePattern;
 }

 public boolean performFromSlave(
         BuildProxy build,
         BuildListener listener)
         throws InterruptedException, IOException {
     FilePath[] paths = build.getExecutionRootDir()
             .list(reportFilenamePattern);
     Collection<Statistic> results = null;
     Set<String> parsedFiles = new HashSet<String>();
     for (FilePath path : paths) {
         final String pathStr = path.getRemote();
         if (!parsedFiles.contains(pathStr)) {
             parsedFiles.add(pathStr);
             try {
                 Collection<Statistic> result =
                         Statistic.parse(new File(pathStr));
                 if (results == null) {
                     results = result;
                 } else {
                     results = Statistic.merge(results, result);
                 }

                 // TODO copy the parsed file to the master

             } catch (XmlPullParserException e) {
                 e.printStackTrace(listener.getLogger());
             }
         }
     }

     // TODO add the results into an Action an attach it to the
     // build.

     return true;
 }
}

Basically, we search the supplied wildcard-path for report files and merge all the results together into a collection of results. In the final part of this series we will create our Action to hold the results and wire everything together.

Writing a Hudson plugin (Part 5½ - Typos corrected)

2008-03-26T22:33:00.005Z

Ok, life has got in the way and I have not been able to update this series as quickly as I originally intended. In any case, there were a number of bugs in Hudson that I discovered and are now fixed, and there was a good deal of tidy-up needed in order for me to figure out what to do for parts 6 and 7. The good news is that I am getting closer to finishing writing these final two parts. The bad news is that there are a number of corrections to the previous posts (I have put some corrections in-line for parts 4 and 5). This post aims to ensure that, for parts 6 and 7, everyone is able to follow from the same code!

pom.xml

This needs to be updated to reference a Hudson version of at least 1.200 in order to have the bugs I identified fixed.

src/main/java/hudson/plugins/helpers/AbstractBuildAction.java

AFAIK I escaped all the HTML entities and this file is the same as constructed from the previous posts.

package hudson.plugins.helpers;

import hudson.model.AbstractBuild;
import hudson.model.HealthReportingAction;

public abstract class AbstractBuildAction<BUILD extends AbstractBuild<?, ?>>
       implements HealthReportingAction {

   private BUILD build = null;

   protected AbstractBuildAction() {
   }

   public synchronized BUILD getBuild() {
       return build;
   }

   public synchronized void setBuild(BUILD build) {
       if (this.build == null && this.build != build) {
           this.build = build;
       }
   }

   public boolean isFloatingBoxActive() {
       return false;
   }

   public boolean isGraphActive() {
       return false;
   }

   public String getGraphName() {
       return getDisplayName();
   }

   public String getSummary() {
       return "";
   }
}

src/main/java/hudson/plugins/helpers/AbstractMavenReporterImpl.java

AFAIK I escaped all the HTML entities and this file is the same as constructed from the previous posts.

package hudson.plugins.helpers;

import hudson.maven.MavenBuildProxy;
import hudson.maven.MavenReporter;
import hudson.maven.MojoInfo;
import hudson.model.BuildListener;
import hudson.model.Result;
import org.apache.maven.project.MavenProject;

import java.io.IOException;

public abstract class AbstractMavenReporterImpl extends MavenReporter {

   protected MojoExecutionReportingMode getExecutionMode() {
       return MojoExecutionReportingMode.ONLY_REPORT_ON_SUCCESS;
   }

   public boolean postExecute(MavenBuildProxy build,
                              MavenProject pom,
                              MojoInfo mojo,
                              BuildListener listener,
                              Throwable error)
           throws InterruptedException, IOException {
       if (!isExecutingMojo(mojo)) {
           // not a mojo who's result we are interested in
           return true;
       }

       final Boolean okToContinue = getExecutionMode().isOkToContinue(this, 
               build, listener, error);
       if (okToContinue != null) {
           return okToContinue;
       }

       build.registerAsProjectAction(this);

       return BuildProxy.doPerform(newGhostwriter(pom, mojo), build, pom, listener);
   }

   protected abstract boolean isExecutingMojo(MojoInfo mojo);

   protected abstract Ghostwriter newGhostwriter(MavenProject pom, MojoInfo mojo);

   public boolean preExecute(MavenBuildProxy build,
                             MavenProject pom,
                             MojoInfo mojo,
                             BuildListener listener)
           throws InterruptedException, IOException {
       return !isAutoconfMojo(mojo) || autoconfPom(build, pom, mojo, listener);
   }

   protected boolean autoconfPom(MavenBuildProxy build,
                                 MavenProject pom,
                                 MojoInfo mojo,
                                 BuildListener listener) {
       return true;
   }

   protected boolean isAutoconfMojo(MojoInfo mojo) {
       return false;
   }


   protected enum MojoExecutionReportingMode {
       ONLY_REPORT_ON_SUCCESS {
           Boolean isOkToContinue(MavenReporter reporter,
                                  MavenBuildProxy build,
                                  BuildListener listener,
                                  Throwable error) {
               return error == null ? null : Boolean.TRUE;
           }
       },
       ALWAYS_REPORT_STABLE {
           Boolean isOkToContinue(MavenReporter reporter,
                                  MavenBuildProxy build,
                                  BuildListener listener,
                                  Throwable error) {
               return null;
           }},
       REPORT_UNSTABLE_ON_ERROR {
           Boolean isOkToContinue(MavenReporter reporter,
                                  MavenBuildProxy build,
                                  BuildListener listener,
                                  Throwable error) {
               if (error != null) {
                   listener.getLogger().println("[HUDSON] "
                           + reporter.getDescriptor().getDisplayName()
                           + " setting build to UNSTABLE");
                   build.setResult(Result.UNSTABLE);
               }
               return null;
           }
       };

       abstract Boolean isOkToContinue(MavenReporter reporter,
                                       MavenBuildProxy build,
                                       BuildListener listener,
                                       Throwable error);
   }
}

src/main/java/hudson/plugins/helpers/AbstractProjectAction.java

AFAIK I escaped all the HTML entities and this file is the same as constructed from the previous posts.

package hudson.plugins.helpers;

import hudson.model.AbstractProject;
import hudson.model.Actionable;

abstract public class AbstractProjectAction<PROJECT extends AbstractProject<?, ?>> 
        extends Actionable {
   private final PROJECT project;

   protected AbstractProjectAction(PROJECT project) {
       this.project = project;
   }

   public PROJECT getProject() {
       return project;
   }

   public boolean isFloatingBoxActive() {
       return true;
   }

   public boolean isGraphActive() {
       return false;
   }

   public String getGraphName() {
       return getDisplayName();
   }

}

src/main/java/hudson/plugins/helpers/AbstractPublisherImpl.java

Some of the HTML entities were not properly escaped.

package hudson.plugins.helpers;

import hudson.tasks.Publisher;
import hudson.model.AbstractBuild;
import hudson.model.BuildListener;
import hudson.Launcher;

import java.io.IOException;

public abstract class AbstractPublisherImpl extends Publisher {

   protected abstract Ghostwriter newGhostwriter();

   public boolean perform(AbstractBuild<?, ?> build, Launcher launcher, 
           final BuildListener listener)
           throws InterruptedException, IOException {
       return BuildProxy.doPerform(newGhostwriter(), build, listener);
   }

   public boolean prebuild(AbstractBuild<?, ?> build, BuildListener listener) {
       return true;
   }
}

src/main/java/hudson/plugins/helpers/BuildProxy.java

Again, some of the HTML entities were not properly escaped. Additionally, this is the file that has had the most updates during this tutorial, so here it is in full:

package hudson.plugins.helpers;

import hudson.FilePath;
import hudson.maven.MavenBuildProxy;
import hudson.util.IOException2;
import hudson.model.Action;
import hudson.model.Result;
import hudson.model.AbstractBuild;
import hudson.model.BuildListener;

import java.util.Calendar;
import java.util.List;
import java.util.ArrayList;
import java.io.IOException;
import java.io.Serializable;

import org.apache.maven.project.MavenProject;

public final class BuildProxy implements Serializable {
   private final FilePath artifactsDir;
   private final FilePath projectRootDir;
   private final FilePath buildRootDir;
   private final FilePath executionRootDir;
   private final Calendar timestamp;
   private final List<AbstractBuildAction<AbstractBuild<?, ?>>> actions =
           new ArrayList<AbstractBuildAction<AbstractBuild<?, ?>>>();
   private Result result = null;
   private boolean continueBuild = true;


   public static boolean doPerform(Ghostwriter ghostwriter,
                                   AbstractBuild<?, ?> build,
                                   BuildListener listener)
           throws IOException, InterruptedException {

       // first, do we need to do anything on the slave

       if (ghostwriter instanceof Ghostwriter.SlaveGhostwriter) {

           // construct the BuildProxy instance that we will use

           BuildProxy buildProxy = new BuildProxy(
                   new FilePath(build.getArtifactsDir()),
                   new FilePath(build.getProject().getRootDir()),
                   new FilePath(build.getRootDir()),
                   build.getProject().getModuleRoot(),
                   build.getTimestamp());

           BuildProxyCallableHelper callableHelper = new BuildProxyCallableHelper(
                   buildProxy, ghostwriter, listener);

           try {
               buildProxy = buildProxy.getExecutionRootDir().act(callableHelper);

               buildProxy.updateBuild(build);

               // terminate the build if necessary
               if (!buildProxy.isContinueBuild()) {
                   return false;
               }
           } catch (Exception e) {
               throw unwrapException(e, listener);
           }
       }

       // finally, on to the master

       final Ghostwriter.MasterGhostwriter masterGhostwriter = 
               Ghostwriter.MasterGhostwriter.class.cast(ghostwriter);

       return masterGhostwriter == null
               || masterGhostwriter.performFromMaster(build, 
                       build.getProject().getModuleRoot(), listener);
   }

   private static RuntimeException unwrapException(Exception e,
                                                   BuildListener listener)
           throws IOException, InterruptedException {
       if (e.getCause() instanceof IOException) {
           throw new IOException2(e.getCause().getMessage(), e);
       }
       if (e.getCause() instanceof InterruptedException) {
           e.getCause().printStackTrace(listener.getLogger());
           throw new InterruptedException(e.getCause().getMessage());
       }
       if (e.getCause() instanceof RuntimeException) {
           throw new RuntimeException(e.getCause());
       }
       // How on earth do we get this far down the branch
       e.printStackTrace(listener.getLogger());
       throw new RuntimeException("Unexpected exception", e);
   }

   public void updateBuild(AbstractBuild<?, ?> build) {
       for (AbstractBuildAction<AbstractBuild<?, ?>> action : actions) {
           if (!build.getActions().contains(action)) {
               action.setBuild(build);
               build.getActions().add(action);
           }
       }

       if (result != null && result.isWorseThan(build.getResult())) {
           build.setResult(result);
       }
   }

   public static boolean doPerform(Ghostwriter ghostwriter,
                                   MavenBuildProxy mavenBuildProxy,
                                   MavenProject pom,
                                   final BuildListener listener)
           throws InterruptedException, IOException {

       // first, construct the BuildProxy instance that we will use

       BuildProxy buildProxy = new BuildProxy(
               mavenBuildProxy.getArtifactsDir(),
               mavenBuildProxy.getProjectRootDir(),
               mavenBuildProxy.getRootDir(),
               new FilePath(pom.getBasedir()),
               mavenBuildProxy.getTimestamp());

       // do we need to do anything on the slave

       if (ghostwriter instanceof Ghostwriter.SlaveGhostwriter) {
           final Ghostwriter.SlaveGhostwriter slaveGhostwriter = 
                   (Ghostwriter.SlaveGhostwriter) ghostwriter;

           // terminate the build if necessary
           if (!slaveGhostwriter.performFromSlave(buildProxy, listener)) {
               return false;
           }
       }

       // finally, on to the master

       try {
           return mavenBuildProxy.execute(new BuildProxyCallableHelper(
                   buildProxy, ghostwriter, listener));
       } catch (Exception e) {
           throw unwrapException(e, listener);
       }
   }

   private BuildProxy(FilePath artifactsDir,
                      FilePath projectRootDir,
                      FilePath buildRootDir,
                      FilePath executionRootDir,
                      Calendar timestamp) {
       this.artifactsDir = artifactsDir;
       this.projectRootDir = projectRootDir;
       this.buildRootDir = buildRootDir;
       this.executionRootDir = executionRootDir;
       this.timestamp = timestamp;
   }

   public List<AbstractBuildAction<AbstractBuild<?, ?>>> getActions() {
       return actions;
   }

   public FilePath getArtifactsDir() {
       return artifactsDir;
   }

   public FilePath getBuildRootDir() {
       return buildRootDir;
   }

   public FilePath getExecutionRootDir() {
       return executionRootDir;
   }

   public FilePath getProjectRootDir() {
       return projectRootDir;
   }

   public Calendar getTimestamp() {
       return timestamp;
   }

   public Result getResult() {
       return result;
   }

   public void setResult(Result result) {
       this.result = result;
   }

   public boolean isContinueBuild() {
       return continueBuild;
   }

   public void setContinueBuild(boolean continueBuild) {
       this.continueBuild = continueBuild;
   }

}

src/main/java/hudson/plugins/helpers/BuildProxyCallableHelper.java

Lost some HTML entities (again!)

package hudson.plugins.helpers;

import hudson.remoting.Callable;
import hudson.maven.MavenBuildProxy;
import hudson.maven.MavenBuild;
import hudson.model.BuildListener;

import java.io.IOException;

class BuildProxyCallableHelper implements Callable<BuildProxy, Exception>,
       MavenBuildProxy.BuildCallable<Boolean, Exception> {

   private final BuildProxy buildProxy;
   private final Ghostwriter ghostwriter;
   private final BuildListener listener;

   BuildProxyCallableHelper(BuildProxy buildProxy,
                            Ghostwriter ghostwriter,
                            BuildListener listener) {
       this.buildProxy = buildProxy;
       this.ghostwriter = ghostwriter;
       this.listener = listener;
   }

   public Boolean call(MavenBuild mavenBuild) throws Exception {
       buildProxy.updateBuild(mavenBuild);
       if (ghostwriter instanceof Ghostwriter.MasterGhostwriter) {
           final Ghostwriter.MasterGhostwriter masterBuildStep =
                   (Ghostwriter.MasterGhostwriter) ghostwriter;
           return masterBuildStep.performFromMaster(mavenBuild, 
                   buildProxy.getExecutionRootDir(), listener);
       }
       return true;
   }

   public BuildProxy call() throws Exception {
       if (ghostwriter instanceof Ghostwriter.SlaveGhostwriter) {
           final Ghostwriter.SlaveGhostwriter slaveBuildStep =
                   (Ghostwriter.SlaveGhostwriter) ghostwriter;
           try {
               buildProxy.setContinueBuild(
                       slaveBuildStep.performFromSlave(buildProxy, listener));
               return buildProxy;
           } catch (IOException e) {
               throw new Exception(e);
           } catch (InterruptedException e) {
               throw new Exception(e);
           }
       }
       return buildProxy;
   }
}

src/main/java/hudson/plugins/helpers/Ghostwriter.java

Java Generics are a real gotcha for HTML entities

package hudson.plugins.helpers;

import hudson.model.BuildListener;
import hudson.model.AbstractBuild;
import hudson.FilePath;

import java.io.Serializable;
import java.io.IOException;

public interface Ghostwriter extends Serializable {
   public static interface SlaveGhostwriter extends Ghostwriter {
       boolean performFromSlave(BuildProxy build, BuildListener listener) 
               throws InterruptedException, IOException;
   }

   public static interface MasterGhostwriter extends Ghostwriter {
       boolean performFromMaster(AbstractBuild<?, ?> build, 
               FilePath executionRoot, BuildListener listener) 
               throws InterruptedException, IOException;
   }

}

src/main/resources/hudson/plugins/helpers/AbstractBuildAction/enlargedGraph.jelly

I left a "css" attribute in the <l:layout> tag.

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
        xmlns:t="/lib/hudson" xmlns:f="/lib/form">
   <l:layout xmlns:plugin="/hudson/plugins/javancss/tags">
       <st:include it="${it.build}" page="sidepanel.jelly"/>
       <l:main-panel>
           <j:if test="${it.graphActive}">
               <h1>${it.graphName}</h1>
               <st:include page="largeGraph.jelly"/>
           </j:if>
       </l:main-panel>
   </l:layout>
</j:jelly>

src/main/resources/hudson/plugins/helpers/AbstractBuildAction/floatingBox.jelly

The <j:if> should be based on the from variable and not it

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
        xmlns:t="/lib/hudson" xmlns:f="/lib/form">
   <j:if test="${from.graphActive}">
       <div style="width:500px;">
           <div class="test-trend-caption">
               ${from.graphName}
           </div>

           <st:include page="normalGraph.jelly"/>

           <div style="text-align:right">
               <a href="enlargedGraph">enlarge</a>
           </div>
       </div>
   </j:if>
</j:jelly>

src/main/resources/hudson/plugins/helpers/AbstractBuildAction/index.jelly

I left a "css" attribute in the <l:layout> tag.

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
        xmlns:t="/lib/hudson" xmlns:f="/lib/form">
   <l:layout xmlns:plugin="/hudson/plugins/javancss/tags">
       <st:include it="${it.build}" page="sidepanel.jelly"/>
       <l:main-panel>
           <h1>${it.displayName}</h1>
           <j:if test="${it.graphActive}">
               <h4>${it.graphName}</h4>
               <st:include page="normalGraph.jelly"/>
           </j:if>
           <st:include page="reportDetail.jelly"/>
       </l:main-panel>
   </l:layout>
</j:jelly>

src/main/resources/hudson/plugins/helpers/AbstractBuildAction/largeGraph.jelly, src/main/resources/hudson/plugins/helpers/AbstractBuildAction/normalGraph.jelly, and src/main/resources/hudson/plugins/helpers/AbstractBuildAction/reportDetail.jelly

These are all the same content, and are basically empty placeholders to be overrided in classes that extend AbstractBuildAction. I do not think there are any corrections.

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
        xmlns:t="/lib/hudson" xmlns:f="/lib/form">
</j:jelly>

src/main/resources/hudson/plugins/helpers/AbstractBuildAction/summary.jelly

The <j:if> had an extra } in the expression

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
        xmlns:t="/lib/hudson" xmlns:f="/lib/form" xmlns:i="jelly:fmt">
   <t:summary icon="${it.iconFileName}">
       <a href="${it.urlName}">${it.displayName}</a>
       ${it.summary}
   </t:summary>
   <j:if test="${it.floatingBoxActive}">
       <div style="float:right">
           <st:include page="floatingBox.jelly"/>
       </div>
   </j:if>
</j:jelly>

src/main/resources/hudson/plugins/helpers/AbstractProjectAction/enlargedGraph.jelly

I left a "css" attribute in the <l:layout> tag.

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
        xmlns:t="/lib/hudson" xmlns:f="/lib/form">
   <l:layout xmlns:plugin="/hudson/plugins/javancss/tags">
       <st:include it="${it.project}" page="sidepanel.jelly"/>
       <l:main-panel>
           <j:if test="${it.graphActive}">
               <h1>${it.graphName}</h1>
               <st:include page="largeGraph.jelly"/>
           </j:if>
       </l:main-panel>
   </l:layout>
</j:jelly>

src/main/resources/hudson/plugins/helpers/AbstractProjectAction/floatingBox.jelly

The <j:if> should be based on the from variable and not it

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
        xmlns:t="/lib/hudson" xmlns:f="/lib/form" xmlns:i="jelly:fmt" xmlns:local="local">
   <j:if test="${from.graphActive}">
       <div style="width:500px;">
           <div class="test-trend-caption">
               ${from.graphName}
           </div>

           <st:include page="normalGraph.jelly"/>

           <div style="text-align:right">
               <a href="enlargedGraph">enlarge</a>
           </div>
       </div>
   </j:if>
</j:jelly>

src/main/resources/hudson/plugins/helpers/AbstractProjectAction/index.jelly

I left a "css" attribute in the <l:layout> tag.

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
        xmlns:t="/lib/hudson" xmlns:f="/lib/form">
   <l:layout xmlns:plugin="/hudson/plugins/javancss/tags">
       <st:include it="${it.project}" page="sidepanel.jelly"/>
       <l:main-panel>
           <h1>${it.displayName}</h1>
           <j:if test="${it.graphActive}">
               <h4>${it.graphName}</h4>
               <st:include page="normalGraph.jelly"/>
           </j:if>
           <st:include page="reportDetail.jelly"/>
       </l:main-panel>
   </l:layout>
</j:jelly>

src/main/resources/hudson/plugins/helpers/AbstractProjectAction/largeGraph.jelly, src/main/resources/hudson/plugins/helpers/AbstractProjectAction/normalGraph.jelly, and src/main/resources/hudson/plugins/helpers/AbstractProjectAction/reportDetail.jelly

These are all the same content, and are basically empty placeholders to be overrided in classes that extend AbstractProjectAction. I do not think there are any corrections.

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
        xmlns:t="/lib/hudson" xmlns:f="/lib/form">
</j:jelly>

Writing a Hudson plug-in (Part 5 – Reporting)

2008-02-20T19:12:00.010Z

Until now we have been generating classes that collect the results we want to display. We have not hooked into Hudson's methods of displaying reports. There are essentially four places that we could want to generate reports:

In each individual build (we'll call this a Single Build Report),
In the project (we'll call this a Single Project Report),
In the summary of all the Maven 2 module builds associated with a multi-module Maven 2 build (we'll call this an Aggregate Build Report), and finally
In the Maven 2 project itself (we'll call this an Aggregate Project Report).

To make matters more confusing, each of these reports usually to implement different classes and need to implement different interfaces:

Project Reports usually inherit from Actionable
Build Reports usually inherit from HealthReportingAction
Single Build Reports in Maven 2 projects need to implement AggregatableAction

What we need is to put some common framework around these reports to ensure that we are not repeating ourselves all the time. In general, each of these four reports will be doing mostly the same things.

The Build Reports will be displaying the results for a specific build
The Project Reports will be displaying the results for the latest build (and possibly a trend chart if that makes sense)
The Aggregate Reports will be displaying the aggregate of all the Single Reports

Sound's like a job for multiple inheritance! Fortune would have it that Java does not support multiple inheritance, so we will have to use some form of composition to get the same effect, and generics can help reduce the repetition too. But enough! On to the solution.

AbstractBuildAction

We'll start with a parent class for all our Build Reports. We'll make this class generic, parameterised by the Build class that it operates on, so that we can use the same core code for all the Build Reports:

package hudson.plugins.helpers;

import hudson.model.AbstractBuild;
import hudson.model.HealthReportingAction;

public abstract class AbstractBuildAction<BUILD extends AbstractBuild<?, ?>>
        implements HealthReportingAction {

    private BUILD build = null;

    protected AbstractBuildAction() {
    }

    public synchronized BUILD getBuild() {
        return build;
    }

    public synchronized void setBuild(BUILD build) {
        if (this.build == null && this.build != build) {
            this.build = build;
        }
    }
}

We store a reference to the build in a local variable and provide a getter for the build. Ideally we would like to the build variable to be final and initialize it in the constructor. However, Hudson's remoting interface will get in the way of this for any actions created on the slave. The solution is to use a setter to set the build. Additionally, we have some logic that makes this setter write-once. This is just a safety net to stop us from accidentally confusing Hudson. Strictly speaking, if you are careful the setter can be justa a simple setter and not write once.

[Correction] We also need to modify BuildProxy so that it calls our setter for us for actions added from slave side executions:

package hudson.plugsin.helpers;

...

public final class BuildProxy implements Serializable {

    ...

    private final List<AbstractBuildAction<AbstractBuild<?,?>>> actions = 
            new ArrayList<AbstractBuildAction<?,?>>>();

    ...

    public void updateBuild(AbstractBuild<,?> build) {
        for (AbstractBuildAction<AbstractBuild<?,?>> action: actions) {
            if (!build.getActions().contains(action)) {
                action.setBuild(build);
                build.getActions().add(action);
            }
        }

        if (result != null && result.isWorseThan(build.getResult())) {
            build.setResult(result);
        }
    }

    ...

    public List<AbstractBuildAction<AbstractBuild<?,?>>> getActions() {
        return actions;
    }

    ...

}

[/Correction]

In addition to these simple getters and setters, we want to provide a framework for displaying the report detail. Each different type of report has different ways of fitting into the Hudson UI. We are going to attempt to standardise these by using wrapper .jelly files to call a standard set which implementing classes can override. Jelly files are stored as resources in the hudson plugin, so with the Maven2 project structure for a Hudson plugin, the java source is /src/main/java/hudson/plugins/helpers/AbstractBuildAction.java and the jelly files for this java class are in /src/main/resources/hudson/plugins/helpers/AbstractBuildAction/. The basic things that all build reports want to do are as follows:

A main report detail page (e.g. package level summary of the number of lines of code)
A graph of some sort, with the option to enlarge it. (e.g. trend graph of the number of lines of code)
A simple summary of the report for the main page (i.e. "17,345 lines of code (+1,534)")

We will implement this functionality with some properties of the AbstractBuildAction, and some default .jelly files. First the additional properties:

...

public abstract class AbstractBuildAction<BUILD extends AbstractBuild<?, ?>> implements HealthReportingAction {

    ...

    public boolean isFloatingBoxActive() {
        return false;
    }

    public boolean isGraphActive() {
        return false;
    }

    public String getGraphName() {
        return getDisplayName();
    }

    public String getSummary() {
        return "";
    }
}

These four properties will be used to control how the action appears. isFloatingBoxActive allows us to enable the floating box on the build summary page. isGraphActive allows us to activate the graph inside the floating box. getGraphName should return the title that will be displayed above the graph. And finally, getSummary will control the summary text to display beside the build report icon on the build summary page.

Next we have some empty default place-holder jelly files. These will be the jelly files that we can override in our actual reports:

reportDetail.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
</j:jelly>

normalGraph.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
</j:jelly>

largeGraph.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
</j:jelly>

The reportDetail.jelly page will be used for the main report detail page, the normalGraph.jelly page will be used for the floating trend graph, while the largeGraph.jelly page will be used for the enlarged graph. To hook these pages into the Hudson framework, we need the following jelly pages:

index.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
    <l:layout xmlns:plugin="/hudson/plugins/javancss/tags" css="/plugin/javancss/css/style.css">
        <st:include it="${it.build}" page="sidepanel.jelly"/>
        <l:main-panel>
            <h1>${it.displayName}</h1>
            <j:if test="${it.graphActive}">
                <h2>${it.graphName}</h2>
                <st:include page="normalGraph.jelly"/>
            </j:if>
            <st:include page="reportDetail.jelly"/>
        </l:main-panel>
    </l:layout>
</j:jelly>

This page will also include the trend graph if it is active

floatingBox.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
    <j:if test="${it.graphActive}">
        <div style="width:500px;">
            <div class="test-trend-caption">
                ${from.graphName}
            </div>

            <st:include page="normalGraph.jelly"/>

            <div style="text-align:right">
                <a href="enlargedGraph">enlarge</a>
            </div>
        </div>
    </j:if>
</j:jelly>

summary.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form" xmlns:i="jelly:fmt">
    <t:summary icon="${it.iconFileName}">
        <a href="${it.urlName}">${it.displayName}</a>
        ${it.summary}
    </t:summary>
    <j:if test="${it.floatingBoxVisible}}">
        <div style="float:right">
            <st:include page="floatingBox.jelly"/>
        </div>
    </j:if>
</j:jelly>

Note that the build main page does not support floating boxes, so we have to hack it in via the summary.jelly page

enlargedGraph.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
    <l:layout xmlns:plugin="/hudson/plugins/javancss/tags" css="/plugin/javancss/css/style.css">
        <st:include it="${it.build}" page="sidepanel.jelly"/>
        <l:main-panel>
            <j:if test="${it.graphActive}">
                <h1>${it.graphName}</h1>
                <st:include page="largeGraph.jelly"/>
            </j:if>
        </l:main-panel>
    </l:layout>
</j:jelly>

AbstractProjectAction

This is similar to AbstractBuildAction, however, we don't have to deal with the write-once setter problem.

package hudson.plugins.helpers;

import hudson.model.AbstractProject;
import hudson.model.Actionable;

abstract public class AbstractProjectAction<PROJECT extends AbstractProject<?, ?>> extends Actionable {
    private final PROJECT project;

    protected AbstractProjectAction(PROJECT project) {
        this.project = project;
    }

    public PROJECT getProject() {
        return project;
    }

    public boolean isFloatingBoxActive() {
        return true;
    }

    public boolean isGraphActive() {
        return false;
    }

    public String getGraphName() {
        return getDisplayName();
    }

}

Again we have a set of jelly files, we'll repeat the same placeholders:

reportDetail.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
</j:jelly>

normalGraph.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
</j:jelly>

largeGraph.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
</j:jelly>

The other link in pages are similar, only changing from a reference for ${it.build} to ${it.project}

index.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
    <l:layout xmlns:plugin="/hudson/plugins/javancss/tags" css="/plugin/javancss/css/style.css">
        <st:include it="${it.project}" page="sidepanel.jelly"/>
        <l:main-panel>
            <h1>${it.displayName}</h1>
            <j:if test="${it.graphActive}">
                <h2>${it.graphName}</h2>
                <st:include page="normalGraph.jelly"/>
            </j:if>
            <st:include page="reportDetail.jelly"/>
        </l:main-panel>
    </l:layout>
</j:jelly>

floatingBox.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form" xmlns:i="jelly:fmt" xmlns:local="local">
    <j:if test="${it.graphActive}">
        <div style="width:500px;">
            <div class="test-trend-caption">
                ${from.graphName}
            </div>

            <st:include page="normalGraph.jelly"/>

            <div style="text-align:right">
                <a href="enlargedGraph">enlarge</a>
            </div>
        </div>
    </j:if>
</j:jelly>

The project page automatically includes the floating box, so we don;t require the summary hack

enlargedGraph.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
         xmlns:t="/lib/hudson" xmlns:f="/lib/form">
    <l:layout xmlns:plugin="/hudson/plugins/javancss/tags" css="/plugin/javancss/css/style.css">
        <st:include it="${it.project}" page="sidepanel.jelly"/>
        <l:main-panel>
            <j:if test="${it.graphActive}">
                <h1>${it.graphName}</h1>
                <st:include page="largeGraph.jelly"/>
            </j:if>
        </l:main-panel>
    </l:layout>
</j:jelly>

Summary

That's it for part 5, next time we will implement the javancss parser and get the results for a build. Part 7 will finish off the plugin with the reports based on these two base classes.

Writing a Hudson plug-in (Part 4 – Abstract Publishers and MavenReporters)

2008-02-11T22:12:00.001Z

This is the fourth installment, which covers the two abstract classes that call the Ghostwriter for us. When we implement our plugin we will extend from these two classes. These classes perform the work of calling the Ghostwriter at the appropriate points in the build, the concrete classes that we will develop in Part 5, however have the responsibility of creating a configured Ghostwriter and providing it to the superclass for execution.

AbstractPublisherImpl

In a way, this is the simplest of the two classes:

package hudson.plugins.helpers;

import hudson.tasks.Publisher;
import hudson.model.AbstractBuild;
import hudson.model.BuildListener;
import hudson.Launcher;

import java.io.IOException;

public abstract class AbstractPublisherImpl extends Publisher {

    protected abstract Ghostwriter newGhostwriter();

    public boolean perform(AbstractBuild build, 
                           Launcher launcher, 
                           final BuildListener listener)
            throws InterruptedException, IOException {
        return BuildProxy.doPerform(newGhostwriter(), build, listener);
    }

    public boolean prebuild(AbstractBuild build, 
                            BuildListener listener) {
        return true;
    }
}

Fairly simple, we just ask the BuildProxy to invoke the new Ghostwriter created by the concrete class.

AbstractMavenReporterImpl

This class needs to do a bit more work. First I'll present the abstract methods and protected methods that can be overrided by implementing classes:

package hudson.plugins.helpers;

import hudson.maven.MavenReporter;
import hudson.maven.MojoInfo;
import hudson.maven.MavenBuildProxy;
import hudson.model.BuildListener;
import hudson.model.Result;
import hudson.plugins.helpers.BuildProxy;
import hudson.plugins.javancss.PluginImpl;
import org.apache.maven.project.MavenProject;

import java.io.IOException;

public abstract class AbstractMavenReporterImpl extends MavenReporter {

    protected abstract boolean isExecutingMojo(MojoInfo mojo);

    protected abstract Ghostwriter newGhostwriter(MavenProject pom, 
                                                  MojoInfo mojo);

    protected boolean autoconfPom(MavenBuildProxy build,
                                  MavenProject pom,
                                  MojoInfo mojo,
                                  BuildListener listener) {
        return true;
    }

    protected boolean isAutoconfMojo(MojoInfo mojo) {
        return false;
    }

}

Maven project execution consists of invocations of various golas of the different mojos that are bound to the Maven lifecycle. Hudson inserts hooks that allow us to intercept mojos before they execute and after they have executed. Intercepting prior to execution can be useful to ensure that the plugin has been configured with the options we require (e.g. XML output enabled). Post execution is usually when we want to invoke our Ghostwriter. If the implementing class wants to be able to tweak the mojo configuration prior to execution it needs to override isAutoconfMojo in order to identify the mojo executions that need to be tweaked, and override autoconfPom to do the actual configuration.

The implementing class needs to provide:

a isExecutingMojo method to identify execution of the mojo that this publisher reports on.
a newGhostwriter method that constructs the Ghostwriter which will do the work for us. The new Ghostwriter can either be configured manually on the build page, and/or can be configured based on information in the pom.

The final thing that we need to decide is when to execute the publisher:

package hudson.plugins.helpers;

...

public abstract class AbstractMavenReporterImpl extends MavenReporter {

    ...
 
    protected MojoExecutionReportingMode getExecutionMode() {
        return MojoExecutionReportingMode.ONLY_REPORT_ON_SUCCESS;
    }

    ...

    protected enum MojoExecutionReportingMode {
        ONLY_REPORT_ON_SUCCESS {
            Boolean isOkToContinue(MavenReporter reporter,
                                   MavenBuildProxy build,
                                   BuildListener listener,
                                   Throwable error) {
                return error == null ? null : Boolean.TRUE;
            }
        },
        ALWAYS_REPORT_STABLE {
            Boolean isOkToContinue(MavenReporter reporter,
                                   MavenBuildProxy build,
                                   BuildListener listener,
                                   Throwable error) {
                return null;
            }},
        REPORT_UNSTABLE_ON_ERROR {
            Boolean isOkToContinue(MavenReporter reporter,
                                   MavenBuildProxy build,
                                   BuildListener listener,
                                   Throwable error) {
                if (error != null) {
                    listener.getLogger().println("[HUDSON] "
                            + reporter.getDescriptor().getDisplayName()
                            + " setting build to UNSTABLE");
                    build.setResult(Result.UNSTABLE);
                }
                return null;
            }
        };

        abstract Boolean isOkToContinue(MavenReporter reporter,
                                        MavenBuildProxy build,
                                        BuildListener listener,
                                        Throwable error);
    }
}

We define three different execution modes:

ONLY_REPORT_ON_SUCCESS - this ensures that we only run the publisher if the mojo executed without error.
ALWAYS_REPORT_STABLE - runs the publisher even if the mojo had an execution error, never marks the build as failed or unstable (note that Maven will most likely mark the build as failed or unstable)
REPORT_UNSTABLE_ON_ERROR - runs the publisher even if the mojo had an execution error. In the event of a mojo execution error the build will be marked UNSTABLE

All that's left is are the methods to tie these all together:

package hudson.plugins.helpers;

...

public abstract class AbstractMavenReporterImpl extends MavenReporter {

    ....

    public boolean preExecute(MavenBuildProxy build,
                              MavenProject pom,
                              MojoInfo mojo,
                              BuildListener listener)
            throws InterruptedException, IOException {
        return !isAutoconfMojo(mojo) 
                || autoconfPom(build, pom, mojo, listener);
    }

    public boolean postExecute(MavenBuildProxy build,
                               MavenProject pom,
                               MojoInfo mojo,
                               BuildListener listener,
                               Throwable error)
            throws InterruptedException, IOException {
        if (!isExecutingMojo(mojo)) {
            // not a mojo who's result we are interested in
            return true;
        }

        final Boolean okToContinue = getExecutionMode()
                .isOkToContinue(this, build, listener, error);
        if (okToContinue != null) {
            return okToContinue;
        }

        build.registerAsProjectAction(this);

        return BuildProxy.doPerform(newGhostwriter(pom, mojo), 
                                    build, 
                                    pom, 
                                    listener);
    }

}

The preExecute method checks if this is an mojo execution that we want to tweak, executing the tweaks if necessary. The postExecute method checks if this the the mojo we want to report on. Then it checks if there were execution errors, following the execution mode reported by getExecutionMode. We register this MavenReporter as a project action (thus signalling Hudson that it should call the getProjectAction() method of our reporter. We will have this method return null if we actually don't want a project action. This is safer than trying to be smart and only calling register if we want a project action!). Finally we invoke the Ghostwriter!

Summary

Well that's it for Part 4. In the next installment, I will introduce a DRY set of classes for the reports that the publisher will generate

Writing a Hudson plug-in (Part 3 – Subcontracting for Publisher and MavenReporter)

2008-01-31T22:44:00.001Z

In this, the third installment, I will introduce some helper classes that we can use to let Publisher and MavenReporter subcontract the job of creating the build reports. Firstly, we have the Ghostwriter interface. An implementation of a Ghostwriter is the work-horse that writes the build report. Secondly, we have the BuildProxy class. This class is a more general MavenBuildProxy that works for both Maven builds and non-Maven builds. In part 4 we will tie these helpers into the Hudson plugin framework with an AbstractMavenReporterImpl and AbstractPublisherImpl from which we can extend our plugin's MavenReporter and Publisher.

The classes presented here and in part 4 should be generic enough that most plugins can be implemented on the back of them, hence my reason for making them this generic.

Ghostwriter

A class or interface should be named after its function. PublisherDelegate, RealReporter, etc, just did not seem snappy enough, so I've picked Ghostwriter, as I think it encapsulates the idea of actually doing the writing without the consumer knowing who did the work!

package hudson.plugins.helpers;

import hudson.model.BuildListener;
import hudson.model.AbstractBuild;
import hudson.FilePath;

import java.io.Serializable;
import java.io.IOException;

public interface Ghostwriter extends Serializable {
    public static interface SlaveGhostwriter 
            extends Ghostwriter {
        boolean performFromSlave(BuildProxy build, 
                                 BuildListener listener) 
                throws InterruptedException, IOException;
    }

    public static interface MasterGhostwriter 
            extends Ghostwriter {
        boolean performFromMaster(AbstractBuild build, 
                                  FilePath executionRoot, 
                                  BuildListener listener) 
                throws InterruptedException, IOException;
    }
}

This interface is really a marker interface for a class that implements at least one of SlaveGhostwriter and MasterGhostwriter. The idea between these two child interfaces is that they encapsulate execution on the slave or the master.

Let's look at Ghostwriter.MasterGhostwriter the method for performing our plugin's action receives three parameters:

The build that this plugin is being executed on;
The file path location of the execution of the build (Note: this is a FilePath as the execution of the build is most likely remote from the master); and
The build listener where to allow us to add information to the build log

These are mostly the same parameters as hudson.tasks.BuildStep provides in its perform method, however we have substituted a file path for a launcher. The logic here is that with the Maven project type, the root of the maven module needs to be passed back to the plugin, so we will need this file path information. With the Free-style project type, the root easily and unambiguously available from the build parameter. The exceptions that we declare are needed as this method could be being called from the slave via the remoting support built into Hudson.

If we now turn to Ghostwriter.SlaveGhostwriter. In this case, the perform method takes two parameters

The build proxy, modelled after MavenBuildProxy, which contains the essential information required for our plugin; and
The build listener.

The idea with both of these interfaces is that our plugin implementation does some work figuring out what our Ghostwriter is supposed to do. If we are in a Maven project, we can determine some of the information from the pom.xml and the rest from Hudson. If we are in any other type of project, we determine all the information from Hudson. We capture that configuration in a Ghostwriter instance, and we send the Ghostwriter off to do our work. The Ghostwriter then goes off to the slave or the master as necessary and does it's job. We have separated our concerns as we have:

A publisher for determining the configuration for a freestyle project;
A maven reported for determine the configuration for a Maven2 project; and
A ghostwriter for getting the job done

Now it is time to look at our next helper class

BuildProxy

This is our workhorse class, it is designed to hide the different execution paths to calling the Ghostwriter and ensuring that there are no differences (as far as the Ghostwriter is concerned) between the Free-style project type and the Maven2 project type. As I said earlier, it is modelled on MavenBuildProxy, so we have getters for the important directories on the Master that are needed on the Slave. For example, if we need to copy a file to the artifacts directory of the master, the build directory of the master, or the project directory of the master.

package hudson.plugins.helpers;

import hudson.FilePath;
import hudson.maven.MavenBuildProxy;
import hudson.util.IOException2;
import hudson.model.Action;
import hudson.model.Result;
import hudson.model.AbstractBuild;
import hudson.model.BuildListener;

import java.util.Calendar;
import java.util.List;
import java.util.ArrayList;
import java.io.IOException;

import org.apache.maven.project.MavenProject;

public final class BuildProxy implements Serializable {
    private final FilePath artifactsDir;
    private final FilePath projectRootDir;
    private final FilePath buildRootDir;
    private final FilePath executionRootDir;
    private final Calendar timestamp;

    private BuildProxy(FilePath artifactsDir,
                       FilePath projectRootDir,
                       FilePath buildRootDir,
                       FilePath executionRootDir,
                       Calendar timestamp) {
        this.artifactsDir = artifactsDir;
        this.projectRootDir = projectRootDir;
        this.buildRootDir = buildRootDir;
        this.executionRootDir = executionRootDir;
        this.timestamp = timestamp;
    }

    public FilePath getArtifactsDir() {
        return artifactsDir;
    }

    public FilePath getBuildRootDir() {
        return buildRootDir;
    }

    public FilePath getExecutionRootDir() {
        return executionRootDir;
    }

    public FilePath getProjectRootDir() {
        return projectRootDir;
    }

    public Calendar getTimestamp() {
        return timestamp;
    }
}

There are additional methods, but I want to outline the key function of BuildProxy before introducing them. Also, the constructor is private because we will use these additional methods it instantiate the BuildProxy for us.

In my experience, the primary information that a plugin needs to know (when executing on the slave) are the five pieces of information that are stored in BuildProxy

The artifacts directory - the directory on the master where artifacts for this project are to be stored;
The build root directory - the directory on the master where build reports for this build are to be stored;
The project root directory - the directory on the master where project reports are to be stored;
The timestamp when the build started - for example to identify new artifacts produced during the build; and finally
The execution root directory - the directory either on the slave or the master from which the build was performed

Now, when our plugin is executing on the slave, it will need to pass back information to the real build. For example, if our plugin completes successfully, we will want to attach Action objects to the AbstractBuild in order to allow the user to see the results of our plugin on the build pages in Hudson. If there were problems when executing our plugin, we may want to fail the build, mark it as unstable, or stop any other plugins from executing. To handle these cases, we need to add some mutable properties to our BuildProxy:

...

public final class BuildProxy implements Serializable {

    ...

    private final List actions = new ArrayList();
    private Result result = null;
    private boolean continueBuild = true;

    ...

    public List getActions() {
        return actions;
    }

    public Result getResult() {
        return result;
    }

    public void setResult(Result result) {
        this.result = result;
    }

    public boolean isContinueBuild() {
        return continueBuild;
    }

    public void setContinueBuild(boolean continueBuild) {
        this.continueBuild = continueBuild;
    }

    ...

}

So now our Ghostwriter.SlaveGhostwriter has the ability to:

Add actions to the build;
Modify the result of the build; and
Flag that the build should finish as early as possible.

At this point we are ready to introduce the static methods that will invoke the Ghostwriter for our Publisher or MavenReporter.

First off, we have a doPerform method designed to be called from a Publisher:

...

public final class BuildProxy {

    ...

    public static boolean doPerform(Ghostwriter ghostwriter,
                                    AbstractBuild build,
                                    BuildListener listener)
            throws IOException, InterruptedException {

        // first, do we need to do anything on the slave

        if (ghostwriter instanceof Ghostwriter.SlaveGhostwriter) {

            // construct the BuildProxy instance that we will use

            BuildProxy buildProxy = new BuildProxy(
                    new FilePath(build.getArtifactsDir()),
                    new FilePath(build.getProject().getRootDir()),
                    new FilePath(build.getRootDir()),
                    build.getProject().getModuleRoot(),
                    build.getTimestamp());

            BuildProxyCallableHelper callableHelper = 
                new BuildProxyCallableHelper(buildProxy, ghostwriter, listener);

            try {
                buildProxy = buildProxy.getExecutionRootDir().act(callableHelper);

                buildProxy.updateBuild(build);

                // terminate the build if necessary
                if (!buildProxy.isContinueBuild()) {
                    return false;
                }
            } catch (Exception e) {
                throw unwrapException(e, listener);
            }
        }

        // finally, on to the master

        final Ghostwriter.MasterGhostwriter masterGhostwriter = 
            Ghostwriter.MasterGhostwriter.class.cast(ghostwriter);

        return masterGhostwriter == null
                || masterGhostwriter.performFromMaster(build, 
                        build.getProject().getModuleRoot(), 
                        listener);
    }

    private static RuntimeException unwrapException(Exception e,
                                                    BuildListener listener)
            throws IOException, InterruptedException {
        if (e.getCause() instanceof IOException) {
            throw new IOException2(e.getCause().getMessage(), e);
        }
        if (e.getCause() instanceof InterruptedException) {
            e.getCause().printStackTrace(listener.getLogger());
            throw new InterruptedException(e.getCause().getMessage());
        }
        if (e.getCause() instanceof RuntimeException) {
            throw new RuntimeException(e.getCause());
        }
        // How on earth do we get this far down the branch
        e.printStackTrace(listener.getLogger());
        throw new RuntimeException("Unexpected exception", e);
    }

    public void updateBuild(AbstractBuild build) {
        // update the actions
        build.getActions().addAll(actions);

        // update the result
        if (result != null && result.isWorseThan(build.getResult())) {
            build.setResult(result);
        }
    }

    ...
}

The method is initially called from the master. It takes a Ghostwriter, a build and a build listener. First off it checks if the Ghostwriter wants to execute some tasks on the slave, i.e. if it implements Ghostwriter.SlaveGhostwriter, sending the Ghostwriter over to the slave with a BuildProxy and updating the real build when the BuildProxy is returned. Finally, if the Ghostwriter wants to execute some tasks on the master, i.e. if it implements Ghostwriter.MasterGhostwriter we pass control over to the Ghostwriter again. Note: slave execution is pass-by-value.. Thus, if the Ghostwriter is executed on both the slave and the master, changes made to the Ghostwriter on the slave will not be picked up when executing on the master.

Now, for the method that will be called from a MavenReporter:

...

public final class BuildProxy implements Serializable {

    ...

    public static boolean doPerform(Ghostwriter ghostwriter,
                                    MavenBuildProxy mavenBuildProxy,
                                    MavenProject pom,
                                    final BuildListener listener)
            throws InterruptedException, IOException {

        // first, construct the BuildProxy instance that we will use

        BuildProxy buildProxy = new BuildProxy(
                mavenBuildProxy.getArtifactsDir(),
                mavenBuildProxy.getProjectRootDir(),
                mavenBuildProxy.getRootDir(),
                new FilePath(pom.getBasedir()),
                mavenBuildProxy.getTimestamp());

        // do we need to do anything on the slave

        if (ghostwriter instanceof Ghostwriter.SlaveGhostwriter) {
            final Ghostwriter.SlaveGhostwriter slaveGhostwriter = 
                (Ghostwriter.SlaveGhostwriter) ghostwriter;

            // terminate the build if necessary
            if (!slaveGhostwriter.performFromSlave(buildProxy, listener)) {
                return false;
            }
        }

        // finally, on to the master

        try {
            return mavenBuildProxy.execute(
                new BuildProxyCallableHelper(buildProxy, 
                                             ghostwriter, 
                                             listener));
        } catch (Exception e) {
            throw unwrapException(e, listener);
        }
    }

    ...
}

This method is initially called from the slave. It takes the Ghostwriter, the MavenBuildProxy, the pom and the build listener. We start off by creating the BuildProxy from the MavenBuildProxy and the pom, then we invoke the Ghostwriter if necessary (i.e. if it executes on the slave). Finally, we send the BuildProxy and the Ghostwriter over to the master in order to add any required build actions and, if necessary execute on the master.

Both of these two static helper methods make use of a Callable that is used by Hudson's remoting support to send tasks between the master and the slave:

package hudson.plugins.helpers;

import hudson.remoting.Callable;
import hudson.maven.MavenBuildProxy;
import hudson.maven.MavenBuild;
import hudson.model.BuildListener;

import java.io.IOException;

class BuildProxyCallableHelper implements Callable,
        MavenBuildProxy.BuildCallable {

    private final BuildProxy buildProxy;
    private final Ghostwriter ghostwriter;
    private final BuildListener listener;

    BuildProxyCallableHelper(BuildProxy buildProxy,
                             Ghostwriter ghostwriter,
                             BuildListener listener) {
        this.buildProxy = buildProxy;
        this.ghostwriter = ghostwriter;
        this.listener = listener;
    }

    public Boolean call(MavenBuild mavenBuild) throws Exception {
        buildProxy.updateBuild(mavenBuild);
        if (ghostwriter instanceof Ghostwriter.MasterGhostwriter) {
            final Ghostwriter.MasterGhostwriter masterBuildStep =
                    (Ghostwriter.MasterGhostwriter) ghostwriter;
            return masterBuildStep.performFromMaster(
                    mavenBuild, 
                    buildProxy.getExecutionRootDir(), 
                    listener);
        }
        return true;
    }

    public BuildProxy call() throws Exception {
        if (ghostwriter instanceof Ghostwriter.SlaveGhostwriter) {
            final Ghostwriter.SlaveGhostwriter slaveBuildStep =
                    (Ghostwriter.SlaveGhostwriter) ghostwriter;
            try {
                buildProxy.setContinueBuild(
                        slaveBuildStep.performFromSlave(buildProxy, listener));
                return buildProxy;
            } catch (IOException e) {
                throw new Exception(e);
            } catch (InterruptedException e) {
                throw new Exception(e);
            }
        }
        return buildProxy;
    }
}

Ok, that's enough for now. The full source code (including javadoc comments) for these classes is available in Hudson's CVS server underneath the /hudson/hudson/plugins/javancss/ directory. The next part of this tutorial will demonstrate how to use these methods by introducing an AbstractMavenReporterImpl and an AbstractPublisherImpl from which we can inherit our publisher and maven reporter.

Writing a Hudson plug-in (Part 2 – Understanding m2 and freestyle projects)

2008-01-22T22:53:00.000Z

Before we proceed with developing the plugin, there is an important concept to understand: How the different project types invoke their plugins.

Currently, one of the biggest source of confusion for end users of Hudson is that most of the plugins do not work with the "m2 project type". This is because there are essentially two completely different implementations of plugins.

"Normal" Plugins

"Normal" plugins work with all project types except the "m2 project type". Typically they will extend Publisher. The model of execution is that the Publisher is invoked on the Hudson Master and can invoke actions remotely on the slave.

The plugin author must be mindfull that they do not overload the master with excessive work, either by minimising the amount of work to be performed, or by sending work to the slave for execution. Best performance will be achieved if the heavy lifting is performed on the slave and then finally the results are sent to the master. Multiple trips back and forth between the master and the slave will not be performant.

"Maven" Plugins

"Maven" plugins only work with the "m2 project type". They must extend MavenReporter. Contrary to the "Normal" plugins, they are executed on the Slave. This is because they are invoked as call-backs from the Maven Embedded that is running on the slave. If the MavenReporter is able to invoke actions remotely on the Master.

The plugin author does not have to be as mindful about overloading the master, as by default, they are executing on the slave, however, the side effect is that the plugin does not have full access to all the Hudson objects, and must usually send work to the master, e.g. to add an Action to a build, it is necessary to serialize the Action, send it to the master along with a Callable that then takes the deserialized Action and adds it to the real Build. (Note: Adding an action is such a common task that BuildProxy provides convenience methods to do this for us)

Don't Repeat Yourself

The result of this split is that it can be difficult to keep the DRY principles when developing Hudson plugins. Looking at what all plugins do, though, there is a standard pattern for most Publishing plugins:

Determine where the information to be published is to be found
Process/extract the information on the slave
Send the persistent result of the build to the master
Finalize processing of the information on the master

Strictly speaking only one of the last two steps is required, however, in most cases there is a trade-off between tasks that are more performant when executed on the slave and tasks that must be performed on the master as either the required information is not available on the slave or the activity will affect the object instances on the master as an artifact of the serialization call-by-value that occurs on the slave.

Thus to keep the DRY principles we need to:

Publisher or MavenReporter constructs a Configuration object.
If the Configuration requires tasks to execute on the slave, ensure that it is sent to the slave and execute those tasks on the slave.
If the Configuration requires tasks to execute on the master, send it (if necessary back) to the master and execute those tasks on the master.

There are other issues with respect to Actions, i.e. Actions for a "m2 project type" must extend a specific subclass. However, this is not a problem, as there is nothing to stop us just having all the actions extend this type, the non-"m2 project type" usage will just ignore that it's extending the subclass.

What's next

In part 3, I will present a couple of wrapper classes that we can use to ensure that the DRY principle gives us plugin execution in one place. Part 4 will tie these wrapper classes into the Hudson plugin framework, giving us a MavenReporter and a Publisher that both use common code to deliver the plugin functionality. Part 5 will add aggregated reporting for m2 project. Part 6 will add simple trend graphs. Part 7 will close out by actually parsing the JavaNCSS files and implementing our plugin!

Writing a Hudson plug-in (Part 1 – Preparation)

2008-01-13T23:24:00.000Z

Writing a Hudson plug-in (Part 1 – Preparation)

This is the first part of a tutorial on writing a Hudson plug-in. The aim of this tutorial is to develop a Publisher for JavaNCSS. This publisher will work with both Hudson Freestyle projects and Hudson Maven2 projects. The publisher will also include a trend graph.

Before we can start plug-in development, we need to have a set of working projects to throw at our plugin. Ideally these projects should be simple and quick to build, but the most essential thing is that they build and run the tool that we want to publish the results of! We need a minimum of three projects:

Ant-based project
Maven2-based single module project
Maven2-based multiple module project

So we will develop these three basic projects, get them running the JavaNCSS tool, and then ensure they build in Hudson.

The Ant-based project

We’ll start with the Ant based project. First we’ll need a directory structure:

project/
   src/
       main/
       test/
   lib/
   tools/
       javancss/

Next we need to download the library dependencies to the lib folder (for example junit.jar) and the custom ant tasks to the appropriate sub-folder of tools.

You can get JUnit.jar from http://www.junit.org.

The JavaNCSS jars are available at http://www.kclee.de/clemens/java/javancss/ (Note this is the .de domain, not the .com domain). JavaNCSS comes as a zip file, so we’ll just extract the three jar files that we need: ccl.jar, javancss.jar and jhbasic.jar

At this point we should have the following

project/
   src/
       main/
       test/
   lib/
       junit.jar
   tools/
       javancss/
           ccl.jar
           javancss.jar
           jhbasic.jar

Next we need a build file, and some sample java files to build. Here is a sample build.xml that will do most of what we want:

<project name="SimpleAntProject" basedir="." default="dist">

   <path id="javancss.ant.task">
       <pathelement location="${basedir}/tools/javancss/ccl.jar"/>
       <pathelement location="${basedir}/tools/javancss/javancss.jar"/>
       <pathelement location="${basedir}/tools/javancss/jhbasic.jar"
   </path>

   <taskdef name="javancss"
           classname="javancss.JavancssAntTask"
           classpathref="javancss.ant.task"/>

   <property name="src.dir" value="src"/>
   <property name="java.dir" value="${src.dir}/main"/>
   <property name="lib.dir" value="lib"/>
   <property name="docs.dir" value="docs"/>
   <property name="src.junit" value="${src.dir}/test"/>
   <property name="manifest" value="${src.dir}/etc/manifest"/>
   <property name="resource.dir" value="${src.dir}/resources"/>
   <property name="build.dir" value="build"/>
   <property name="dist.dir" value="dist"/>

   <property name="build.classes" value="${build.dir}/classes"/>
   <property name="build.lib" value="${build.dir}/lib"/>
   <property name="build.javadocs" value="${build.dir}/javadocs"/>
   <property name="build.tests" value="${build.dir}/testcases"/>
   <property name="build.junit.reports" location="${build.tests}/reports"/>
   <property name="manifest.tmp" value="${build.dir}/optional.manifest"/>

   <!-- the absolute path -->
   <property name="build.tests.value" location="${build.tests}"/>

   <property name="debug" value="true"/>
   <property name="deprecation" value="false"/>
   <property name="optimize" value="true"/>
   <property name="javac.target" value="1.5"/>
   <property name="javac.source" value="1.5"/>
   <property name="junit.fork" value="false"/>
   <property name="junit.filtertrace" value="off"/>
   <property name="junit.summary" value="no"/>
   <property name="test.haltonfailure" value="false"/>
   <property name="junit.forkmode" value="once"/>

   <path id="classpath">
       <pathelement location="${lib.dir}"/>
   </path>

   <path id="tests-classpath">
       <pathelement location="${build.classes}"/>
       <pathelement location="${build.tests}"/>
       <pathelement location="${src.junit}"/>
       <pathelement location="${lib.dir}/junit.jar"/>
       <path refid="classpath"/>
   </path>
   <!-- turn this path into a string which is passed to the tests -->
   <property name="tests-classpath.value" refid="tests-classpath"/>

   <target name="clean" description="Delete all generated files">
       <delete dir="${build.dir}" failonerror="false"/>
       <delete dir="${dist.dir}" failonerror="false"/>
   </target>

   <target name="compile" description="Compiles ${ant.project.name}">
       <mkdir dir="${build.classes}"/>
       <javac srcdir="${java.dir}"
               destdir="${build.classes}"
               debug="${debug}"
               target="${javac.target}"
               source="${javac.source}"
               deprecation="${deprecation}">
           <classpath refid="classpath">
       </javac>
   </target>

   <target name="compile-tests" depends="compile">
       <mkdir dir="${build.tests}"></mkdir>
       <javac srcdir="${src.junit}"
               destdir="${build.tests}"
               debug="${debug}"
               target="${javac.target}"
               source="${javac.source}"
               deprecation="${deprecation}">
           <classpath refid="tests-classpath"/>
       </javac>
   </target>

   <target name="test" depends="compile-tests">
       <mkdir dir="${build.junit.reports}"/>
       <junit>
           <classpath refid="tests-classpath"/>
           <formatter type="brief" usefile="false"/>
           <batchtest fork="yes"
                   todir="${build.junit.reports}"
                   haltonerror="true"
                   haltonfailure="true">
               <fileset dir="${src.junit}"></fileset>
               <include name="**/*Test*.java"></include>
               <exclude name="**/AllTests.java"></exclude>
           </batchtest>
       </junit>
   </target>

   <target name="dist" description="Create the distribution" depends="compile, test, javancss">
       <mkdir dir="${dist.dir}"/>
       <jar destfile="${dist.dir}/${ant.project.name}.jar" basedir="${build.classes}"/>
   </target>

   <target name="javancss">
       <javancss srcdir="${java.dir}"
               generatereport="true"
               outputfile="${build.dir}/javancss-report.xml"
               format="xml"/>
   </target>
</project>

We can then write some simple java classes and test our build. We’ll start with a simple HelloWorld.java

package com.onedash.hello;

/**
* A basic hello world application.
*
* @author Stephen Connolly
*/
public class Hello {
   public String sayHello(String name) {
       return "Hello " + name;
   }

   public static void main(String[] args) {
       Hello instance = new Hello();
       if (args.length == 1) {
           System.out.println(instance.sayHello(args[0]));
       } else {
           System.out.println(instance.sayHello("world"));
       }
   }
}

At this point we should have the following project structure:

project/
   build.xml
   src/
       main/
           com/
               onedash/
                   hello/
                       Hello.java
       test/
   lib/
       junit.jar
   tools/
       javancss/
           ccl.jar
           javancss.jar
           jhbasic.jar

And running ANT should give output like:

C:\local\project>ant clean dist
Buildfile: build.xml

clean:
[delete] Deleting directory C:\local\project\build
[delete] Deleting directory C:\local\project\dist

compile:
 [mkdir] Created dir: C:\local\project\build\classes
 [javac] Compiling 5 source files to C:\local\project\build\classes

compile-tests:
 [mkdir] Created dir: C:\local\project\build\testcases
 [javac] Compiling 1 source file to C:\local\project\build\testcases

test:
 [mkdir] Created dir: C:\local\project\build\testcases\reports

javancss:
[javancss] Generating report

dist:
 [mkdir] Created dir: C:\local\project\dist
   [jar] Building jar: C:\local\project\dist\SimpleAntProject.jar

BUILD SUCCESSFUL
Total time: 3 seconds

And there should be a javancss-report.xml file in the build directory. That should be enough for an ANT project. Some extra finesse would be to add some more source code so we can examine the structure of the xml report.

The Maven2 based single module project

First we’ll need a directory structure:

project/
   src/
       main/
           java/
       test/
           java/

Next we need a pom.xml to describe the project for Maven2. Here is a simple pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xsi="http://www.w3.org/2001/XMLSchema-instance">
       xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
   <modelVersion>4.0.0</modelVersion>

   <!-- artifact identification -->

   <groupId>com.onedash.hudson</groupId>
   <artifactId>SimpleM2Project</artifactId>
   <name>SimpleM2Project</name>
   <version>1.0-SNAPSHOT</version>
   <packaging>jar</packaging>

   <!-- dependency specification -->

   <dependencies>
       <dependency>
           <groupId>junit</groupId>
           <artifactId>junit</artifactId>
           <version>4.4</version>
           <scope>test</scope>
       </dependency>
   </dependencies>

   <!-- build process and details specification -->

   <build>
       <defaultGoal>package</defaultGoal>
       <plugins>
           <plugin>
               <groupId>org.codehaus.mojo</groupId>
               <artifactId>javancss-maven-plugin</artifactId>
               <executions>
                   <execution>
                       <phase>process-sources</phase>
                       <goals>
                           <goal>report</goal>
                       </goals>
                   </execution>
               </executions>
           </plugin>
       </plugins>
   </build>
</project>

We can reuse the Hello.java from the Ant project, so at this point we should have the following directory structure:

project/
   pom.xml
   src/
       main/
           java/
               com/
                   onedash/
                       hello/
                           Hello.java
       test/
           java/

And running maven should give the following output:

C:\local\project>mvn clean package
[INFO] Scanning for projects...
[INFO] ----------------------------------------------------------------------------
[INFO] Building SimpleM2Project
[INFO]    task-segment: [clean, package]
[INFO] ----------------------------------------------------------------------------
[INFO] [clean:clean]
[INFO] Deleting directory C:\local\project\target
[INFO] Deleting directory C:\local\project\target\classes
[INFO] Deleting directory C:\local\project\target\test-classes
[INFO] Deleting directory C:\local\project\target\site
[INFO] [javancss:report {execution: default}]
[WARNING] Unable to locate Source XRef to link to - DISABLED
[INFO] [resources:resources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:compile]
[INFO] Compiling 1 source files to C:\local\project\target\classes
[INFO] [resources:testResources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:testCompile]
[INFO] Nothing to compile - all classes are up to date
[INFO] [surefire:test]
[INFO] No tests to run.
[INFO] [jar:jar]
[INFO] Building jar: C:\local\project\target\SimpleM2Project-1.0-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5 seconds
[INFO] Finished at: Sun Jan 13 22:06:20 GMT 2008
[INFO] Final Memory: 5M/10M
[INFO] ------------------------------------------------------------------------

And there should be a javancss-raw-report.xml file in the target directory.

The Maven2 based multi-module project

We can generate this project in its simplest form just by adding the simple project as a child of a project with <packaging>pom</packaging>. We’ll start off with the following directory structure:

project/
   SimpleM2Project
       pom.xml
       src/
           main/
               java/
                   com/
                       onedash/
                           hello/
                               Hello.java
           test/
               java/

Where the SimpleM2Project subtree is a copy of the previous project. All we now need is a pom.xml to nest this project in:

<project xmlns="http://maven.apache.org/POM/4.0.0"
       xsi="http://www.w3.org/2001/XMLSchema-instance"
       schemalocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
   <modelVersion>4.0.0</modelVersion>

   <!-- artifact identification -->

   <groupId>com.onedash.mvn.hudson</groupId>
   <artifactId>MultiM2Project</artifactId>
   <name>MultiM2Project</name>
   <version>1.0-SNAPSHOT</version>
   <packaging>pom</packaging>

   <!-- child modules -->
   <modules>
       <module>SimpleM2Project</module>
   </modules>

</project>

And running maven should give the following output:

C:\local\project>mvn clean package
[INFO] Scanning for projects...
[INFO] Reactor build order:
[INFO]   SimpleM2Project
[INFO]   MultiM2Project
[INFO] ----------------------------------------------------------------------------
[INFO] Building SimpleM2Project
[INFO]    task-segment: [clean, package]
[INFO] ----------------------------------------------------------------------------
[INFO] [clean:clean]
[INFO] Deleting directory C:\local\project\SimpleM2Project\target
[INFO] Deleting directory C:\local\project\SimpleM2Project\target\classes
[INFO] Deleting directory C:\local\project\SimpleM2Project\target\test-classes
[INFO] Deleting directory C:\local\project\SimpleM2Project\target\site
[INFO] [javancss:report {execution: default}]
[WARNING] Unable to locate Source XRef to link to - DISABLED
[INFO] [resources:resources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:compile]
[INFO] Compiling 1 source files to C:\local\project\SimpleM2Project\target\classes
[INFO] [resources:testResources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:testCompile]
[INFO] No sources to compile
[INFO] [surefire:test]
[INFO] No tests to run.
[INFO] [jar:jar]
[INFO] Building jar: C:\local\project\SimpleM2Project\target\SimpleM2Project-1.0-SNAPSHOT.jar
[INFO] ----------------------------------------------------------------------------
[INFO] Building MultiM2Project
[INFO]    task-segment: [clean, package]
[INFO] ----------------------------------------------------------------------------
[INFO] [clean:clean]
[INFO] Deleting directory C:\local\project\target
[INFO] Deleting directory C:\local\project\target\classes
[INFO] Deleting directory C:\local\project\target\test-classes
[INFO] Deleting directory C:\local\project\target\site
[INFO] [site:attach-descriptor]
[INFO]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] SimpleM2Project ....................................... SUCCESS [7.240s]
[INFO] MultiM2Project ........................................ SUCCESS [2.754s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10 seconds
[INFO] Finished at: Sun Jan 13 22:16:37 GMT 2008
[INFO] Final Memory: 7M/15M
[INFO] ------------------------------------------------------------------------

And there should be a javancss-raw-report.xml file in the SimpleM2Project/target directory.

The Hudson setup

We’ll start by running mvn hpi:create to create a base to work from:

C:\local>mvn hpi:create
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'hpi'.
[INFO] ----------------------------------------------------------------------------
[INFO] Building Maven Default Project
[INFO]    task-segment: [hpi:create] (aggregator-style)
[INFO] ----------------------------------------------------------------------------

...

[INFO] [hpi:create]
Enter the groupId of your plugin: org.jvnet.hudson.plugins
[INFO] Defaulting package to group ID: org.jvnet.hudson.plugins
Enter the artifactId of your plugin: javancss

...

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2 minutes 44 seconds
[INFO] Finished at: Sun Jan 13 22:34:58 GMT 2008
[INFO] Final Memory: 8M/15M
[INFO] ------------------------------------------------------------------------

The result should be a sub-directory called javancss with a pom.xml file and a basic Hudson plugin starting point. We’ll edit the pom.xml to update the version of Hudson to the latest (at the time of writing 1.170) and launch a live development instance to set our projects up in.

Open up the pom.xml file, and change the <version>1.153</version> tags for the hudson-core and hudson-war dependencies to 1.170.

The dependency section should now look like this:

<dependency>
   <groupId>org.jvnet.hudson.main</groupId>
   <artifactId>hudson-core</artifactId>
   <version>1.170</version>
   <scope>provided</scope>
</dependency>
<dependency>
   <groupId>org.jvnet.hudson.main</groupId>
   <artifactId>hudson-war</artifactId>
   <type>war</type>
   <version>1.170</version>
   <scope>test</scope>
</dependency>

Next launch a copy of Hudson by running mvn hpi:run

C:\local\javancss>mvn hpi:run
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'hpi'.
[INFO] ----------------------------------------------------------------------------
[INFO] Building Unnamed - org.jvnet.hudson.plugins:javancss:hpi:1.0-SNAPSHOT
[INFO]    task-segment: [hpi:run]
[INFO] ----------------------------------------------------------------------------
[INFO] Preparing hpi:run

...

[INFO] Started Jetty Server
[INFO] Starting scanner at interval of 1 seconds.

Now fire up a web browser for http://localhost:8080/

We want to create five projects:

Freestyle project for the Simple Ant project
Freestyle project for the Single module Maven2 project
Freestyle project for the Multi-module Maven2 project
Maven2 project for the Single module Maven2 project
Maven2 project for the Multi-module Maven2 project

For extra finesse, we can add a slave executor, and create a second set of these projects tied to the slave. An final extra project that does nothing other than trigger all the other projects to build is the icing on the cake.

JPA, equals() and hashCode()

2007-12-14T21:25:00.001Z

http://burtbeckwith.com/blog/?p=53

Combining Hibernate and Facelets with Maven, Netbeans and Glassfish

2007-11-21T16:54:00.000Z

Note to self, for later reading

http://technology.amis.nl/blog/?p=2610

Not sure why netbeans is essential on this!

Sventon and Hudson

2007-11-14T15:02:00.000Z

This morning I discovered sventon (http://www.sventon.org) a Subversion (http://subversion.tigris.org/) repository browser written in Java. And it's free! Ten minutes later I had this integrated as a Repository browser in Hudson (https://hudson.dev.java.net/ will be available in build 1.157) and I'm loving it. It's not quite as good as fisheye... but it does everything I need it to!

Hudson is really starting to have kick-ass features that leave everything else for dust.

Hudson Coverage plugin

2007-11-06T09:09:00.000Z

Note to self, here are some code coverage tools for .NET that are free to use... and therefore good candidates for having the Coverage plugin for Hudson support off the shelf:

http://sourceforge.net/projects/partcover/

http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=881a36c6-6f45-4485-a94e-060130687151

Stumbled upon

2007-10-08T11:38:00.000+01:00

A UML renderer written in Java

http://snipsnap.org/space/SnipGraph/UML+Example

A Sequence Diagram renderer written in Java

http://sdedit.sourceforge.net/

A more complex set of UML digrams from UMLet

http://www.umlet.com/

A JavaScript code formatter

http://code.google.com/p/google-code-prettify/

JAX-WS 2.1 madness

2007-09-05T08:42:00.000+01:00

I am sick of the fun that is getting JAX-WS 2.1 to work on JVM 1.6.

Oh, copy these four jars into the endorsed directory and then you can use JAX-WS 2.1... oh but sometimes it won't work for some unknown reason and then it will work again.

How you are supposed to explain this to end users, I don't know.

So next you need a platform specific installer to put those jars into the correct location, or a platform specific start script to tell the JVM about my alternate endorsed lib folder... or do I write a self-extracting jar file that exctracts the libs and forks a second JVM... no that won't work for people wanting to use my library..

We went through all this pain with SAX and DOM. Did Sun learn nothing?

pom for jsf-comp.sf.net's chartcreator

2007-08-23T21:45:00.001+01:00

Not perfect, but enough to get you going.

To build with this pom:
You will need to grab the sources from http://downloads.sourceforge.net/jsf-comp/chartcreator-1.2.0.source.zip and extract into src/main/java
You will need to grab the jar and extract the three files in the META-INF (not MANIFEST.MF) into src/main/resources/META-INF

Then you can install away to your hearts content.

<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>net.sf.jsf-comp</groupId>
    <artifactId>chartcreator</artifactId>
    <version>1.2.0-mavenized</version>
    <packaging>jar</packaging>
    <name>ChartCreator</name>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.4</source>
                    <target>1.4</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>javax.faces</groupId>
            <artifactId>jsf-api</artifactId>
            <version>1.2-b19</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>javax.faces</groupId>
            <artifactId>jsf-impl</artifactId>
            <version>1.2-b19</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>com.sun.facelets</groupId>
            <artifactId>jsf-facelets</artifactId>
            <version>1.1.11</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>javax.servlet.jsp</groupId>
            <artifactId>jsp-api</artifactId>
            <version>2.1</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>javax.portlet</groupId>
            <artifactId>portlet-api</artifactId>
            <version>1.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>servlet-api</artifactId>
            <version>2.5</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>jfree</groupId>
            <artifactId>jfreechart</artifactId>
            <version>1.0.5</version>
        </dependency>
    </dependencies>
</project>

Near minimal Maven 2 pom.xml for Jetty development of Facelets applications using JSF RI 1.2

2007-08-23T21:12:00.001+01:00

Spent ages trying to get close to this... gave up looking at what others had done, here is my version from scratch:

<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>....</groupId>
    <artifactId>....</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>war</packaging>
    <name>....</name>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.5</source>
                    <target>1.5</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.mortbay.jetty</groupId>
                <artifactId>maven-jetty-plugin</artifactId>
                <version>6.1H.5-beta</version>
                <configuration>
                    <contextPath>/</contextPath>
                    <scanIntervalSeconds>10</scanIntervalSeconds>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>javax.faces</groupId>
            <artifactId>jsf-api</artifactId>
            <version>1.2-b19</version>
        </dependency>
        <dependency>
            <groupId>javax.faces</groupId>
            <artifactId>jsf-impl</artifactId>
            <version>1.2-b19</version>
        </dependency>
        <dependency>
            <groupId>com.sun.facelets</groupId>
            <artifactId>jsf-facelets</artifactId>
            <version>1.1.11</version>
        </dependency>
        <dependency>
            <groupId>commons-digester</groupId>
            <artifactId>commons-digester</artifactId>
            <version>1.7</version>
        </dependency>
        <dependency>
            <groupId>commons-beanutils</groupId>
            <artifactId>commons-beanutils</artifactId>
            <version>1.7.0</version>
        </dependency>
        <dependency>
            <groupId>commons-collections</groupId>
            <artifactId>commons-collections</artifactId>
            <version>3.2</version>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>jstl</artifactId>
            <version>1.1.0</version>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>servlet-api</artifactId>
                <version>2.5</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>
</project>

JPA Entity exerciser

2007-07-20T20:15:00.000+01:00

Working on this to add to EasyGloss.

There are a number of rules that JPA entities must obey:

equals and hashCode must only be based on the persistent fields that are @Id annotated.
annotations must be applied to either fields or getters, no mix & match (although future versions of the spec may provide for such)
In general, getters and setters should be simple methods (i.e. no complex processing)

I want to have a JPA Entity excerciser that will check these rules for you (and can be included in your unit tests)

hashCode() pitfalls with HashSet and HashMap

2007-02-26T21:52:00.000Z

I have been reading a book on C# recently, and it got me thinking about Java's hashCode() in a little bit more detail than I had before.

Consider the following Java class.


public class Person {
  private String firstName;
  private String surname;

  public Person(String firstName, String surname) {
    this.firstName = firstName;
    this.surname = surname;
  }

  public boolean equals(Object other) {
    /* proper equals checking firstName and surname */
  }

  public int hashCode() {
    int code = surname.hashCode();
    code = 31 * code + firstName.hashCode();
    return code;
  }

  public String getFirstName() { /* getter */ }
  public void setFirstName(String firstName) { /* setter */ }
  public String getSurname() { /* getter */ }
  public void setSurname(String surname) { /* setter */ }
}

What is wrong with the above? Ignore that the hash code may not be well designed given that names are usually A-Z only and the prime factor may not be the most efficient algorithm for calculating hash codes for our data set.

OK, so I have cheated... the problem is the setters that I glossed over.

Our hash code is not calculated on the basis of immutable values.

Let's try using it with a HashSet...


Person joe = new Person("Joe", "Bloggs");
Set people = new HashSet();
people.add(joe);

assert people.contains(joe); // this works
assert people.contains(new Person("Joe", "Bloggs")); // this works

joe.setSurname("Smith");

assert !people.contains(new Person("Joe", "Bloggs")); // this works
assert people.contains(new Person("Joe", "Smith")); // this fails!!!
assert people.contains(joe); // this fails!!!

boolean found = false;
for (Person person: people) {
  if (person == joe) {
    found = true;
    break;
  }
}
assert found; // this works!!!

So what is going on?

When we put joe into the HashSet, the HashSet selects a bucket based on the hash code for joe. When we ask the HashSet if it contains a Person, it computes the hash code of the object that it's looking for and searches only in that bucket.

As our hashCode() is based on non-final fields, the hash code can change behind the scenes of our HashSet (and it won't find out until it has too few buckets and needs some more to optimise searching). Thus completely invalidating the basic assumption of the HashSet.

A correct implementation for the Person class would either have the names as final, or have hashCode like so


public int hashCode() {
  return 0; // we have no non-final identity fields to calculate the hashCode from
}

How em.merge actually works

2006-06-14T13:23:00.000+01:00

Let's have a look at some em.merge related fun.

From discussions on a number of forums, here is my explanation for what goes on when you call em.merge.

We will use the following classes as an example:

@Entity
public class Parent {
    ...
    private List<Child> children;
    ...
    public List<Child> getChildren() { return this.children; }
    public void setChildren(List<Child> children) { this.children = children; }
    ...
}


@Entity
public class Child {
    ...
    private Parent parent;
    ...
    public Parent getParent() { return this.parent; }
    public void setParent(Parent parent) { this.parent = parent; }
    ...
}

To aid in understanding, we will assume that our client has a UserTransaction ut
(this is to allow us to force entity instances to become detached. Entity instances can become
detached without using a UserTransaction, however, we want to show what happens to the entities;

So our client goes something like:

  Parent parent = null;
  ...
  ut.begin();
  parent = em.find(Parent.class, aParentId);
  ...

At this point the client has a reference to a managed instance. We can illustrate this like so:

Now at some point we have to commit the transaction and the transaction ends.

  ...
  ut.commit();
  ...

So now parent still points to an instance of the class Parent that still holds the data as
before, however, the instances have lost their connection to the entity manager (as the
connection depended on the transaction.

We can illustrate this like so:

Later on, we want to synchronise the parent object back to the database, so we call em.merge:

  ...
  ut.begin();
  Parent managedParent = em.merge(parent);
  ...

Now, the transaction is still active, so the instances pointed to by managedParent are still
managed instances, i.e. changes made to fields will be reflected back into the database.

The instances pointed to by parent are still detached instances, i.e. changes made to fields
do not get persisted back to the database. em.merge has done two things for us:

Fetched new managed instances of the objects we already have

Synchronised all the changes in the instances we have back to the managed instances (and therefore back to the database

We can illustrate this like so:

Once we commit or otherwise end the transaction, we will now have two sets of detached instances.

Some people want to not have references to our old detached instance lying around... so they will
typically reuse the parent instance variable like so:

  ...
  ut.begin()
  parent = em.merge(parent);
  ...

This is OK, provided that there are no other references to the detached entities lying around.

If you had done something like

  ...
  Child child = parent.getChildren().get(0);
  ut.begin();
  parent = em.merge(parent);
  ut.commit();
  ...

Then you are in a dangerous situation as child points to a different detatched instance of the same persisted entity!!!

The moral of the story is that when you call em.merge, you need to know what instances you are merging and what you will do with the detatched instances after you have finished merging

Another view on unit testing annotated entities

2006-04-13T13:32:00.000+01:00

Borys Burnayev has an interesting article with some useful ideas on testing annotated entities and EJB3 classes requiring injection to work.
http://www-128.ibm.com/developerworks/java/library/j-ejb3jpa.html

Unit testing of annotated classes (part 2)

2006-04-12T13:05:00.000+01:00

EasyGloss has been accepted as an incubator project on dev.java.net

https://easygloss.dev.java.net/

Unit testing of annotated classes

2006-04-11T13:23:00.000+01:00

I ran into some fun while trying to unit test my annotated entities as some of the annotations were on private fields.

Rather than create constructors to do the injection for me, or change the access type on the fields, I came up with this little framework that people might be interested in. (Or be able to help improve)

Temporary home: http://www.stvconsultants.com/community/easygloss/

Permanent home: hopefully on dev.java.net

Has anyone any comments?

JUnit testing strategies for EJB3 entities and beans

2006-04-07T10:12:00.000+01:00

Here are my initial thoughts (which I posted as a forum comment elsewhere)

Layer 0. Test the entity beans are deployable (You'll need some of the framework from Layer 4 for this). Basically, you need to know that all your annotations work. Things to watch out for are multiple @Id fields in one class or @EmbeddedID or @IdClass in conjunction with @ManyToOne, @ManyToMany, @OneToMany, @OneToOne and fun with @JoinTable, @JoinColumn and @JoinColumns. Once you know how these are supposed to work with the specification, it's not too bad to write it correctly each time. But there are some gotchas that will break things later on.

Layer 1. Do the functions in the classes that don't depend on annotations work as expected. Typically, this is just going to be the getters and setters in your entity classes. Of course JUnit best practice says we don't bother testing functions that look like:

public T getX() {
return this.x;
}

or

public void setX(T x) {
this.x = x;
}

as there is nothing that can go wrong with them. So in that case, your level 1 tests will just be initial values specified from constructors and verifying that the non-get/set pairs work, and that the getters you have tagged @Transient work (because you've likely put some logic in them)

Layer 2. Test the session bean methods that don't require injection to work.

Layer 3. Test the session bean methods that require injection (Mock Objects). Simulate the injection for yourself, injecting Mock Objects for the entity manager. Then you can confirm that the correct methods are being called in the correct sequences, etc.
[Note this may require some skill in designing the mock. I'm working on developing my own entitymanager mock, and if it looks useful I'll release it to the world.

Layer 4. Test the session bean methods that require injection (Real entity manager) (See Layer 0)
For this you will need an out of container persistence implementation. Currently Hibernate and Glassfish provide beta versions. You will need a different persistence.xml file that lists all the entities. You will have to use reflection to inject the entity manager(s) that you create from an entity manager factory unless you provide a constructor that takes an EntityManager as a parameter. You may need to use reflection to call any @PostConstruct method if you made it private.

Layer 5. Navigate the relationships in the objects returned from Layer 4 using a database that has been loaded with test data.

I am currently using Layers 0, 1, 2 & 4 to test my session beans and entity beans.

Has anyone else any other ideas?

Nope, it's in the spec

2006-03-20T09:33:00.000Z

From the spec:

The deployment tool must first read the Java EE application deployment descriptor from the application .ear file (META-INF/application.xml). If the deployment descriptor is present, it fully specifies the modules included in the application. If no deployment descriptor is present, the deployment tool uses the following rules to determine the modules included in the application.

It would be nice if glassfish gave some warning that your application.xml might not be completely specified!

application.xml related stuff

2006-03-20T09:21:00.000Z

I have been going nuts trying to trace problems with deployment of
enterprise applications and resource injection. I think the issue is
around the application.xml file in a .ear! Without an
application.xml file, everything is fine and dandy. As
soon as you add an application.xml file, things start
breaking... it seems that when you have defined an
application.xml file only those modules that are defined in
the application.xml file are loaded. Without a
application.xml file, all the .jar files are scanned for
@Stateless and @Statefull annotations and the
modules are inferred. It would help if there was some notification that
maybe your application.xml file was incomplete, or else
some way of restoring the scanning behaviour if you want it. Now to go
digging through the spec to see if I'm just stupidly missing something
or if I need to file a bug report.

Rolling your own ELResolver

2006-03-08T22:52:00.000Z

ELResolvers are where you get to extend the new unified EL for your own pages. They are not that difficult to extend, it's just a case of:

Have your class extend from ELResolver
Implement the abstract methods
Add an <el-resolver> element to your <application> section in faces-config.xml
Start using your resolver!

OK, so why would you want to do this? Well, for one it can make some things a lot easier. I was driven to find this solution in order to dynamically generate h:dataTables with variable numbers of columns. Ordinarily you would need to bind the h:dataTable to a backing bean and have the backing bean add in the extra columns, but with c:forEach now being compatible with JSF, we have an alternative.

here's the model of the data we are trying to present: (the classes are coming from EJB entities which are mapping to a legacy database)

public class Result {
    ...
    private Sample sample;
    private ResultType resultType;
    private String value;
    ...
}

public class Sample {
    ...
    private String name;
    private Collection<Result> results;
    ...
}

public class Project {
    ...
    private String name;
    private Collection<Sample> samples;
    ...
}

We want to display a results table for all the samples from the project object we have been passed... it should look something like

Sample   TypeA   TypeB   TypeC   ...
1   ...   ...   ...
2   ...   ...   ...
...

where there are as many columns as there are types.

We add a method to the project class that returns Collection<ResultType>

and what we'd like to do is:

<h:dataTable value="#{project.samples}" var="sample">
    <h:column>
       <h:outputText value="#{sample.name}"/>
    </h:column>
    <c:forEach items="#{project.resultTypes}" var="resultType">
       <h:column>
          <h:outputText value="#{sample.results[resultType].value}"/>
       </h:column>
    </c:forEach>
</h:dataTable>

Normally this would only work if sample.results is a Map<String,Result> and each ResultType can be converted into a string that happens to be the key of the map. We end up writing a wrapper class for a Map and since we've been given a collection and not a map, efficiency is not the best.

but a custom ELResolver can help us:

I'll post more when I have simplified my resolver to fit this example

Note to self

2006-03-03T16:06:00.000Z

The path to the solution to authorization woes could start here

http://www.thoughtsabout.net/blog/archives/000033.html

Keeping up with the latest glassfish builds

2006-02-17T08:00:00.000Z

Are you keeping up with the latest glassfish builds? Are you running windows 2k or higher? Are you fed up having to manually reinstall the server whenever a new weekly build comes out?

Well I was, so here's a very basic Windows NT command script.

@ECHO OFF REM Change these environment variables to your own config SET GLASSFISH_DOWNLOADS=c:\java\downloads SET GLASSFISH_DRIVE=c: SET GLASSFISH_PARENT_DIR=c:\java REM Make sure we are in the correct location %GLASSFISH_DRIVE% CD %GLASSFISH_PARENT_DIR% REM Find the latest build that has been downloaded SET BUILD=aaa FOR %%i IN (%GLASSFISH_DOWNLOADS%\glassfish-installer-9.0-*.jar) DO IF %%i GTR %BUILD% SET BUILD=%%i IF A%BUILD%A==AaaaA GOTO no_build CLS ECHO . ECHO . ECHO Newest build found is: %BUILD% IF EXIST glassfish.bld (TYPE glassfish.bld) ELSE (ECHO Current installed build: I don't know) ECHO . ECHO . ECHO . ECHO [%0] About to remove current installation of Glassfish... ECHO . ECHO This will delete any currently deployed applications, so you would want to ECHO either not care about them or have made a backup already! ECHO . ECHO . ECHO This is your last chance to press Ctrl+C to abort otherwise ECHO press any other key to continue... PAUSE >> NUL ECHO . ECHO . ECHO . ECHO [%0] Stopping domain1... ECHO . CALL ASADMIN stop-domain domain1 ECHO . ECHO . ECHO . ECHO [%0] Removing current installation... ECHO . RMDIR /S /Q %GLASSFISH_PARENT_DIR%\glassfish ECHO . ECHO . ECHO . ECHO [%0] Starting installer... ECHO . JAVA -Xmx256m -jar %BUILD% ECHO Current installed build: %BUILD% > glassfish.bld ECHO . ECHO . ECHO . ECHO [%0] Setting up domain1... ECHO . CD glassfish CALL ANT -f setup.xml ECHO . ECHO . ECHO . ECHO [%0] Starting domain1... ECHO . CALL ASADMIN start-domain domain1 ECHO . ECHO . ECHO . ECHO [%0] Glassfish reinstalled! ECHO . GOTO end :no_build ECHO . ECHO . ECHO . ECHO Cannot find any glassfish builds in %GLASSFISH_DOWNLOADS% ECHO . :end ECHO . ECHO . ECHO . ECHO [%0] I'm done, press a key and I'll be out of here... ECHO . PAUSE >> NUL

Save it as, e.g. REINSTALL-GLASSFISH.CMD and all you need to do is run it every time you download a new build!

Todo:
1. Add some stuff to make it go fetch the latest build from the download page
2. Add some stuff to pull out all the applications from the autodeploy directory before removing the current build and put them back in afterwards.