Maven Plugin Testing Tools

Obviously your MOJOs (read Maven Plugins) need to be tested as detailed as all your other classes. Here are some details on using the Maven Plugin Testing Tools and how I used them for the Macker Maven Plugin.

Unit Testing
To unit test a MOJO create a JUnit test case and extend AbstractMojoTestCase. It comes with the maven-plugin-testing-harness:

<dependency>
  <groupId>org.apache.maven.plugin-testing</groupId>
  <artifactId>maven-plugin-testing-harness</artifactId>
  <version>1.2</version>
  <scope>test</scope>
</dependency>

Note that version 1.2 of the maven-plugin-testing-harness is still Maven 2.0 compatible, as will be version 1.3. The alpha release of version 2.0 is already based on Maven 3. In the test case the MOJO is initialised with a pom fragment and executed. The basic usage is well documented in the Maven Plugin Harness documentation. The usual folder structure for plugin unit tests is

.
 |-- pom.xml
 \-- src
     |-- main
     \-- test
         |-- java
         |   \-- MojoTest.java
         \-- resources
             \-- unit
                 |-- test-configuration1
                 |   \-- resources for this test if any
                 |-- test-configuration1-plugin-config.xml
                 \-- test-configuration2-plugin-config.xml

The class MojoTest contains methods testConfiguration1() and testConfiguration2(). There are several Maven projects using this approach, just do a code search.

Stubs
If your MOJO needs more complex parameters, e.g. a MavenProject or an ArtifactRepository, these have to be provided as stubs. Stubs are simple implementations of real Maven objects, e.g.

public class org.apache.maven.plugin.testing.stubs.ArtifactStub
  implements org.apache.maven.artifact.Artifact

and have to be configured in the pom fragment (test-configuration1-plugin-config.xml) as described in the Maven Plugin Testing Harness Cookbook. See also MackerMojoTest in the Macker Maven Plugin for an extensive example. There are several stubs to simulate Maven objects such as ArtifactHandler or ArtifactResolver. Creating stubs gets cumbersome when you need more of Maven's internals. Every object and every method has to be stubbed out.

Integration Testing
Integration testing is done with the maven-plugin-testing-tools:

<dependency>
  <groupId>org.apache.maven.plugin-testing</groupId>
  <artifactId>maven-plugin-testing-harness</artifactId>
  <version>1.2</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.apache.maven.plugin-testing</groupId>
  <artifactId>maven-plugin-testing-tools</artifactId>
  <version>1.2</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.codehaus.plexus</groupId>
  <artifactId>plexus-utils</artifactId>
  <version>1.5.6</version>
  <scope>test</scope>
</dependency>

Note that the testing tools 1.2 specifically need plexus-utils version 1.5.6. The usual folder structure for plugin integration tests is

.
 |-- pom.xml
 \-- src
     |-- main
     \-- test
         |-- java
         |   \-- MojoIT.java
         \-- resources
             \-- it
                 |-- test-case1
                 |   |-- pom.xml
                 |   \-- main
                 |       \-- classes and data for this test
                 \-- test-case2

The class MojoIT contains the test methods testCase1() and testCase2(). Folder test-case1 contains a full Maven module that uses the MOJO under test in some way.

The test uses PluginTestTool, BuildTool and the other tools from maven-plugin-testing-tools to create a local repository, install the Maven module under test into it and execute the Maven build of test-case1/pom.xml. The Plugin Testing Tools site provides more detail about this process and the various tools.

Setup
The Plugin Testing Tools do not provide an abstract test case. Each test has to create it's own AbstractPluginITCase. A good example is the AbstractEclipsePluginIT of the Eclipse Plugin. It contains methods to build the module, execute poms against it and verify created artefacts. As far as I know this is the only example available. AbstractOuncePluginITCase is a modified copy as well as AbstractMackerPluginITCase.

Execution
Integration tests should be executed in the integration-test phase.

<build>
  <plugins>
    <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <executions>
        <execution>
          <phase>integration-test</phase>
          <goals>
            <goal>test</goal>
          </goals>
          <configuration>
            <includes>
              <include>**/*IT.java</include>
            </includes>
            <excludes>
              <exclude>specified only to override config
                       from default execution</exclude>
            </excludes>
          </configuration>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

The PluginTestTool modifies the pom.xml to skip all tests, so the tests are not invoked again when the module is build by the integration test.

Help Mojo Workaround
Unfortunately there is a problem with the above approach. It works fine for modules which do not contain Maven Plugins. But it fails during integration test preparation when the help-mojo of maven-plugin-plugin is executed. Fortunately the guys from the Eclipse Plugin found a workaround:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-plugin-plugin</artifactId>
  <!-- lock down to old version as newer version aborts
       build upon no mojos as required during ITs -->
  <version>2.4.3</version>
  <executions>
    <!-- disable execution, makes IT preparation using
         maven-plugin-testing-tools fail (see
         target/test-build-logs/setup.build.log) -->
    <execution>
      <id>help-mojo</id>
      <configuration>
        <extractors>
          <extractor />
        </extractors>
      </configuration>
    </execution>
  </executions>
</plugin>

But now the plugin can't be built from scratch any more. So they used a profile to run the integration tests and disable help-mojo execution.

<profiles>
  <profile>
    <id>run-its</id>
    <build>
      <plugins>
        <plugin>
          <artifactId>maven-surefire-plugin</artifactId>
          ...
        </plugin>
        <plugin>
          <artifactId>maven-plugin-plugin</artifactId>
          ...
        </plugin>
      </plugins>
    </build>
  </profile>
</profiles>

Wrap-up
So far so good. The BuildTool has to activate the profile run-its (as it has to skip help-mojo execution). This could be done by setting a certain property, let's call it ProjectTool:packageProjectArtifact. Then the profile would only be activated during integration test preparation.

<profiles>
  <profile>
    <id>run-its</id>
    ...
    <activation>
      <property>
        <name>ProjectTool:packageProjectArtifact</name>
      </property>
    </activation>
  </profile>
</profiles>

I've submitted a patch for that, but in the meantime I had to copy the BuildTool into my own plugin, ugh. (I was working towards a clean solution throughout this post but in the end all gets messed up.) The whole plugin testing can be seen in action in the Macker Maven Plugin.

Acknowledgement
Experimenting with the Maven Plugin Testing Tools was part of a System One Research Day. Thank you System One for supporting Open Source :-)