27 November 2012

See the new test fail

The Wikipedia article about Test-driven development describes the process of TDD. It says that after adding a new test, you should run all tests and see if the new one fails. This is part of the red-green-refactor cycle. During my remote pairing activities with Thomas I noticed that I tend to forget this step. To fix this I wrote a tiny Eclipse plugin that complains if a test run does not fail after adding a new test.
Alert! New Test Did Not FailThe plugin attaches itself to the org.eclipse.jdt.junit.testRunListeners extension point and records the names of all tests during JUnit test execution. (RedGreenListener.java) When the test session is finished, it compares these test statistics against the previous run. In fact the only thing we need to know is if any test cases have been added. (TestStats.java) Based on the comparison the plugin decides if to show an annoying popup or not. (TestRunDiff.java)

The current mechanism is simple and likely to be wrong for special cases:
public boolean firstTestOk() {
    return newAdded.size() == 0 || secondFailed;
but it works great on katas and small hobby projects.

The org.eclipse.jdt.junit.testRunListeners extension point is available in Eclipse Europa (version 3.3), but it does not work there. The schema testRunListeners.exsd for the extension point seems to be missing from the JDT/JUnit bundles. The situation changes in Helios (3.6) and JUnit starts to notify the declared listeners. As the current release of Eclipse is version 3.8/4.2, I believe that two versions of backwards-compatibility should be enough.

(Download the plugin from Bitbucket.)


Sebastian Dietrich said...

Whow - that's a serious code cop :-) Penalties for abbreviations could help software development a lot.

Just think about what we could achieve when developers are forced to write tests / refactor... No more excuses like "management would not have let me do it right"...

Peter Kofler said...

That is actually a cool idea. Thank you Sebastian. Maybe I can extend the plugin to hide the production source until a test failed, and then hide the test until it passes. And then wait until a refactoring from the menu was triggered, at least local rename three times. Crude and brutal ;-)

Some time ago I was also wandering if it is possible to come up with a language that can only be coded in a clean way. But this seems to be a difficult one...