28 May 2012

Eclipse Plugin Unit Testing

I will describe the set-up of a unit testing infrastructure for a big RCP application consisting of several plugins. In the first part I will write about how to run JUnit tests inside Eclipse.

PlugsTest Plugins
First we need a place to put the tests. We do not want to put the tests in the same project as the production code because we do not want to ship the tests. So for every plugin which we want to write tests for, we create a separate plugin with the same name suffixed with ".test". For example let's assume we want to write a JUnit test for a class in the company.product.general plugin. First we look for a plugin named company.product.general.test. Does it exist? Maybe we never used it and need to check it out from version control. Still nothing? OK, so we need to create it.

For a new test plugin we create a fragment project. We do that by selecting File > New > Project > Plug-in Development > Fragment Project. A fragment allows us to add code to another plug-in, known as the host. The fragment is merged with its host and has full access to all the packages declared in its host, not only the exported ones. Among other ways to set up tests for plugins, fragments generally are the best solution. So our unit tests have access to all classes of the host plugin. The fragment inherits all dependencies from the host. The only explicit required dependency of the test plugin is org.junit4. (Remember that various files in the build system might need to be updated for new (test) plugins to be part of the build. These might include some kind of loading rules, certain Ant build files or test features.)

The above set-up is enough to run simple JUnit tests which do not use any feature of Eclipse's OSGI container. To enable PDE JUnit tests the host plugin needs to define Eclipse-ExtensibleAPI: true in its MANIFEST.MF file. This allows the fragment to access classes from the host plugin and tells the PDE tooling to add the fragment to the classpath of the host plugin.

FragmentsTest Packages
We use the plugin's name as base package of the plugin's source code to avoid name collisions. For example all packages of the plugin company.product.general are sub packages of company.product.general. This is not the case for fragments, as fragments are merged with the host. All packages of the company.product.general.test fragment are still sub packages of company.product.general.

The package with the same name as the test fragment is used for common test set-up or custom assertion logic that might be used for more than one test. For example the package company.product.general.test might contain a class called UsLocale, which sets the current locale to US, as needed to test some business logic. Common test code can be reused by other test plugins if it is exported from the fragment. This works the same way as for normal plugins.

Warning about "Add Required Plugins"
Sometimes it is necessary to add new plugins to a launch configuration in order for the application to start. When you do that by using the "Add Required Plugins" button in the launch configuration, make sure that all the test plugins are not selected afterwards. We had some issues when test-only functionality somehow "leaked" into production.

Simple JUnit Tests
When the test plugin is ready, we can start writing simple JUnit tests. They will not necessarily be "simple", they are just called like that to distinguish them from PDE JUnit tests which have a more complex set-up. Even when developing RCP applications, these simple JUnit tests have their use. We can use them to test any class that is not dependent on the Eclipse Platform API. They are particularly useful for testing business logic and model classes.

JUnit tests go into the same Java package as the class under test. So the test plugin has the same packages as the host plugin. Test classes are named like the class under test suffixed with "Test". For example let's assume we want to test the class company.product.general.api.CatchHandlers in the plugin company.product.general. The related test is company.product.general.api.CatchHandlersTest in the company.product.general.test plugin.

Unit testing is supposed to test classes or methods in isolation but classes usually depend on other classes. For more complex scenarios we want to use different objects than the live ones to get rid of these dependencies. These objects are provided to the class under test. By using them we can verify logic independently of the depended-on objects. Such objects are called Test Doubles.

Dummy objects, fakes and stubs are created easily with custom classes. But true mocks need some more logic to verify the order of method invocations (which needs some kind of state machine). Fortunately there are several frameworks that bring mocks to Java, for example EasyMock. To use it in our RCP application we need to create on OSGI bundle (i.e. an Eclipse plugin) from its jars. Let the plugin company.product.test.mock contain the mocking framework's jars. To use its classes in tests add it to the dependencies of the test fragment.

PDE JUnit Tests
PDE or Plugin JUnit tests are executed by a special test runner that launches another Eclipse instance and executes the test methods within that workbench. So PDE JUnit tests can be used for classes that are dependent on Eclipse Platform API. Parts of the Eclipse platform are available outside a running workbench as well, so a normal JUnit test might still be an option. Additionally some other parts, for example the extension registry can be mocked. But there are cases where a running workbench is necessary. For example you may assume a particular behaviour of Plugin.getStateLocation(), which would not be available if you were just using an ordinary JUnit test. Besides that the tests themselves look exactly like any other test.

When automating the build these PDE tests should be distinguished from basic ones. So we follow a different naming convention and call them *PlatformTests. Also note that the usual class extension mechanism might not work inside the workbench. You need to introduce interfaces for the types you want to mock.


Tomek Bujok said...

That's a nice write-up. But, if you have more test fragments there's no way to share utils among them. Either you put them in to the host bundle, or copy them everytime to the fragment (sic!). I wished RCP was a more friendly development environment.,

Peter Kofler said...

Thank you Tomek. Regarding re-use of test utils, we added a test fragment to a core plugin that all of our plugins depend on. Due to Eclipse-ExtensibleAPI: true all other test fragments have access to the utils defined in the core test fragment. Another way would be to define your own test utils plugin and require it in all your test fragments. This is already done for JUnit and the mocking framework (if needed). Does this help you?

Tomek Bujok said...

Hi Peter,

The Eclipse-ExtensibleAPI enables you to add API from the fragments to the Host, so I can see the API from the fragments by the inclusion of the host in other bundle. It does not enable you to share API among fragments of the same host.

I also tried the second option. But in my case the utils plugin depends on the Host plugin (that is supposed to be tested).
When I include then this utils plugin in the fragments of the Host (that the utils depends on) eclipse complains that there is a cycle (although it's not really a cycle, let's say a semi cycle), but I cannot proceed with this either.


Peter Kofler said...

Tomek, that's interesting, I did not know that you have different test fragments for a single plugin, but I understand that it may be useful, e.g. unit test fragment, integration test fragment etc. My number one rule to resolve issues like that is "decompose". Maybe your plugin is doing too much? Splitting it into several smaller ones with clear dependencies might make it easier to avoid duplication. But of course, I do not know your scenario, so this is just an idea...

Unknown said...

Using Eclipse-ExtensibleAPI to re-use test fragment code in other test fragments has a side-effect. Production code importing packages now also have access to test code. How do you handle this?

Peter Kofler said...

yes the Eclipse-ExtensibleAPI makes the test fragments visible to all users, that is the wanted behaviour for the other test fragments but not the production code. I did not handle this in a technical way. I just made sure that during all kind of testing except the unit testing run, the fragments were not loaded.