Eclipse Plugin Unit Testing

I will describe the set-up of a unit testing infrastructure for a big RCP application consisting of several plugins. In the first part I will write about how to run JUnit tests inside Eclipse.

Test Plugins
First we need a place to put the tests. We do not want to put the tests in the same project as the production code because we do not want to ship the tests. So for every plugin which we want to write tests for, we create a separate plugin with the same name suffixed with ".test". For example let's assume we want to write a JUnit test for a class in the company.product.general plugin. First we look for a plugin named company.product.general.test. Does it exist? Maybe we never used it and need to check it out from version control. Still nothing? OK, so we need to create it.

For a new test plugin we create a fragment project. We do that by selecting File > New > Project > Plug-in Development > Fragment Project. A fragment allows us to add code to another plug-in, known as the host. The fragment is merged with its host and has full access to all the packages declared in its host, not only the exported ones. Among other ways to set up tests for plugins, fragments generally are the best solution. So our unit tests have access to all classes of the host plugin. The fragment inherits all dependencies from the host. The only explicit required dependency of the test plugin is org.junit4. (Remember that various files in the build system might need to be updated for new (test) plugins to be part of the build. These might include some kind of loading rules, certain Ant build files or test features.)

The above set-up is enough to run simple JUnit tests which do not use any feature of Eclipse's OSGI container. To enable PDE JUnit tests the host plugin needs to define Eclipse-ExtensibleAPI: true in its MANIFEST.MF file. This allows the fragment to access classes from the host plugin and tells the PDE tooling to add the fragment to the classpath of the host plugin.

Test Packages
We use the plugin's name as base package of the plugin's source code to avoid name collisions. For example all packages of the plugin company.product.general are sub packages of company.product.general. This is not the case for fragments, as fragments are merged with the host. All packages of the company.product.general.test fragment are still sub packages of company.product.general.

The package with the same name as the test fragment is used for common test set-up or custom assertion logic that might be used for more than one test. For example the package company.product.general.test might contain a class called UsLocale, which sets the current locale to US, as needed to test some business logic. Common test code can be reused by other test plugins if it is exported from the fragment. This works the same way as for normal plugins.

Warning about "Add Required Plugins"
Sometimes it is necessary to add new plugins to a launch configuration in order for the application to start. When you do that by using the "Add Required Plugins" button in the launch configuration, make sure that all the test plugins are not selected afterwards. We had some issues when test-only functionality somehow "leaked" into production.

Simple JUnit Tests
When the test plugin is ready, we can start writing simple JUnit tests. They will not necessarily be "simple", they are just called like that to distinguish them from PDE JUnit tests which have a more complex set-up. Even when developing RCP applications, these simple JUnit tests have their use. We can use them to test any class that is not dependent on the Eclipse Platform API. They are particularly useful for testing business logic and model classes.

JUnit tests go into the same Java package as the class under test. So the test plugin has the same packages as the host plugin. Test classes are named like the class under test suffixed with "Test". For example let's assume we want to test the class company.product.general.api.CatchHandlers in the plugin company.product.general. The related test is company.product.general.api.CatchHandlersTest in the company.product.general.test plugin.

Mocking
Unit testing is supposed to test classes or methods in isolation but classes usually depend on other classes. For more complex scenarios we want to use different objects than the live ones to get rid of these dependencies. These objects are provided to the class under test. By using them we can verify logic independently of the depended-on objects. Such objects are called Test Doubles.

Dummy objects, fakes and stubs are created easily with custom classes. But true mocks need some more logic to verify the order of method invocations (which needs some kind of state machine). Fortunately there are several frameworks that bring mocks to Java, for example EasyMock. To use it in our RCP application we need to create on OSGI bundle (i.e. an Eclipse plugin) from its jars. Let the plugin company.product.test.mock contain the mocking framework's jars. To use its classes in tests add it to the dependencies of the test fragment.

PDE JUnit Tests
PDE or Plugin JUnit tests are executed by a special test runner that launches another Eclipse instance and executes the test methods within that workbench. So PDE JUnit tests can be used for classes that are dependent on Eclipse Platform API. Parts of the Eclipse platform are available outside a running workbench as well, so a normal JUnit test might still be an option. Additionally some other parts, for example the extension registry can be mocked. But there are cases where a running workbench is necessary. For example you may assume a particular behaviour of Plugin.getStateLocation(), which would not be available if you were just using an ordinary JUnit test. Besides that the tests themselves look exactly like any other test.

When automating the build these PDE tests should be distinguished from basic ones. So we follow a different naming convention and call them *PlatformTests. Also note that the usual class extension mechanism might not work inside the workbench. You need to introduce interfaces for the types you want to mock.

GeeCON 2012

Last week one of my favourite Java conferences GeeCON took place in Poznan, Poland. I was there before and liked it a lot. Despite that it took me 13 hours train travel to Poznan, it was a very good conference. The organisers were caring for everything (even free beer ;-), the presentations were interesting and some of the attendees were really passionate about our craft.

I am too lazy to write a full round-up of all sessions that I attended, but at least I will give you a list of my favourite sentences heard during the sessions:

• Maintenance starts with the first check-in. (Thomas Sundberg)
• 40 errors, 38 warnings. That looks bad. Let's commit that. ;-) (Thomas Sundberg)
• object null extends Nothing() (Ceylon Language)
• Threads do not work. (Bruce Eckel)
• It's not getting easier being a developer. [... because there is more and more we need to know.] (Ivar Jacobson)
• An architecture without executable code is a hallucination, but code without architecture is shit. (Ivar Jacobson)
• ... f*cking ... f*cking ... f*ck ... [He really does not like Java any more ;-)] (Gavin King)
• I write crap on daily basis. ;-) (Tomek Kaczanowski)
• @Seed("deadbeef") [This is a valid hex number.] (Dawid Weiss)
• Every time you write a for loop, god kills a kitten. (Keith Braithwaite)
• Simple design is for simple systems. [... making fun of overly complex solutions.] (Thomas Sundberg)
• The more green it is, the more green it is. [... the build.] (Wojciech Seliga)
• Heal or Kill. [... these flaky tests.] (Wojciech Seliga)
• ... 50.000 LoC. - That are five operating systems. Does it do that much? (Kevlin Henney)

Thank you all for the great show!

Mocking the Eclipse Extension Registry

Some classes of a RCP application might make use of Extension Points, calling Platform.getExtensionRegistry() and evaluating the returned configuration elements. It is possible to mock (in fact stub) the default registry provider by calling RegistryFactory.setDefaultRegistryProvider() with a registry implementation. By using a fake registry some PDE JUnit test cases can be converted to plain JUnit tests, which speeds up their execution.

A Changing Registry Factory
The default registry provider can only be set once. So we need to register a wrapper, which delegates to a RegistryFactory so it can be changed among tests.

import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.IExtensionRegistry;
import org.eclipse.core.runtime.RegistryFactory;
import org.eclipse.core.runtime.spi.IRegistryProvider;

public class ChangingRegistryFactory {

  private static IRegistryProvider registryProvider;
  private static boolean defaultRegistryHasBeenSet;

  public static void setDefaultRegistryProvider(IRegistryProvider provider)
                                                       throws CoreException {
    setRegistryProvider(provider);
    synchronized (ChangingRegistryFactory.class) {
      setDefaultRegistryProviderOnce();
    }
  }

  private static void setRegistryProvider(IRegistryProvider provider) {
    registryProvider = provider;
  }

  private static void setDefaultRegistryProviderOnce() throws CoreException {
    if (!defaultRegistryHasBeenSet) {
      setDefaultRegistryProvider();
      defaultRegistryHasBeenSet = true;
    }
  }

  private static void setDefaultRegistryProvider() throws CoreException {
    RegistryFactory.setDefaultRegistryProvider(new IRegistryProvider() {
      public IExtensionRegistry getRegistry() {
        return registryProvider.getRegistry();
      }
    });
  }

}

An Extension Point Mockery
Using EasyMock or any other mock framework we can create a mock registry and register it to the RegistryFactory (in fact to the ChangingRegistryFactory). To hide the mocking details we provide a specific builder like API in the helper class ExtensionPointMockery which helps mocking Extension Points:

import static org.easymock.EasyMock.*;

public class ExtensionPointMockery {

  private final IExtensionRegistry registry = createRegistry();

  @SuppressWarnings("unused")
  public ExtensionPointMockery() throws CoreException {
    // allow CoreException to be thrown from initialiser blocks
  }

  private IExtensionRegistry createRegistry() {
    return createMock(IExtensionRegistry.class);
  }

  public IConfigurationElement confWith(Object specificInstance)
                                                       throws CoreException {
    IConfigurationElement confElement = createMock(IConfigurationElement.class);
    expect(confElement.createExecutableExtension("class")).
          andReturn(specificInstance).anyTimes();
    replay(confElement);
    return confElement;
  }

  public IConfigurationElement mockFor(Class<?> type) throws CoreException {
    Object mockInstance = createNiceMock(type);
    replay(mockInstance);
    return confWith(mockInstance);
  }

  public void set(String key, IConfigurationElement... elements) {
    expect(registry.getConfigurationElementsFor(key)).
          andReturn(elements).anyTimes();
  }

  public void setAll(IConfigurationElement... elements) {
    expect(registry.getConfigurationElementsFor(anyObject(String.class))).
          andReturn(elements).anyTimes();
  }

  public void close() throws CoreException {
    replay(registry);

    IRegistryProvider provider = createMock(IRegistryProvider.class);
    expect(provider.getRegistry()).andReturn(registry).anyTimes();
    replay(provider);

    ChangingRegistryFactory.setDefaultRegistryProvider(provider);
  }

}

Usage
Its typical usage looks like

new ExtensionPointMockery() {{
  set("preferenceSettings", mockFor(IPreferenceSettings.class));
  set("filterManager", confWith(new FilterManagerDelegate()));
  set("postProcessor");
}}.close();

where confWith() puts the given object directly into the registry, simulating its contribution and mockFor() creates a "nice mock" (a fake) of the given interface and adds it to the configuration. The call of set() without any parameter initialises the extension registry for the given name but without any contributions. Finally close() creates the mock registry and registers it to the platform.

Happy mocking!