KDiff3 Merge Tool for RTC

"3-way merges still remain one of the more taxing tasks of any software development team." (Wikipedia)

Rational Team Concert
For my current work I use Rational Team Concert, an Eclipse based IDE. RTC has a built-in compare tool that works well for comparing files or reviewing changes and furthermore Jazz Version Control offers various ways to resolve conflicts. But recently I had to merge a branch with more than a year worth of changes (more than 1000 commits) and the RTC merge tool showed certain deficiencies which had a negative impact on my productivity. So I looked for alternatives.

3-way Merge Tools
An ideal merge tool would be free and should support all platforms. Based on my experience and googling for 3-way merge, the following tools showed up in that particular order: KDiff3, P4Merge and DiffMerge.

KDiff3
KDiff3 is a free diff and merge program. It works on single files and whole directories. It runs on MS-Windows, Mac OSX and any Un*x that is supported by QT. It is GNU licensed which is ok for me but troubles our legal department ;-) Its installation is straight forward and it has direct Explorer integration on Windows systems. I have been using it to compare files for years. It is really cool, go check it out!

Integration with RTC
RTC allows a standalone merging tool to be used as a replacement of the internal one. To configure KDiff3 in RTC perform the following steps:

• Open the menu for Window -> Preferences.
• Select Team -> Jazz Source Control -> External Compare Tools.
• Choose <<custom>> in the drop down.
• Check off to use the external compare tool as the default open action.
• Browse to your KDiff3 install location for the executables.
• Use "${file1Path}" "${file2Path}" for the 2-Way Compare.
• Use "${ancestorFilePath}" "${file2Path}" "${file1Path}" -o "${mergeFilePath}" for the for the 3-Way Conflict Compare.

Usage
An external merge tool starts much slower than the integrated one. I do not recommend using it as the default open action. I use the internal one to see differences and only when I have to merge conflicts I choose Open in External Compare Tool from the context menu of the unresolved change:
Then KDiff3 starts and (hopefully) greets you with the message that all conflicts have been merged ;-) Often this is the case because the merge capabilities of KDiff3 are much stronger than of RTC, its Auto-Merge rarely works for me.
The user interface of KDiff3 is a bit crowded with windows. The top left diff-window (A) shows the base version, i.e. the common ancestor of both changed files. The middle window (B) shows the proposed changes and the right window (C) contains the current version of the file. The bottom panel is editable and allows you to solve conflicts, while showing the final output. KDiff3 immediately positions the cursor at the first unresolved conflict where you can use ctrl-1, 2 or 3 to do the merging. You can also use the ctrl-arrows to navigate the diffs.
Usually manual merging is a matter of a few key strokes. After saving and exiting KDiff3, RTC shows the changed file. Now select Resolve as Merged from the context menu of the unresolved change and the merge is finished.
If no changed file appears after saving the merge in KDiff3, that means that the merged version is the same as the current version. In this case select Resolve with Mine from the context menu of the unresolved change.

Other Tools
As mentioned in the beginning, there are two similar tools available: P4Merge and DiffMerge. P4Merge, the Perforce Visual Merge Tool, is part of the version control system Perforce. To only install P4Merge, deselect everything except Visual Merge Tool during install. P4Merge compares files, folders and images. It is much like KDiff3, shows three diff windows and the bottom pane is editable. DiffMerge is from SourceGear, another vendor of version control systems. It compares files and folders and has Windows Explorer menu integration like KDiff3. Both tools can be found in the drop-down list of supported RTC external compare tools and the arguments should not require any changes.

Waterfall and Requirements

We are doing Waterfall. "They" want it to be called Agile using Scrum, but it really is Waterfall. We have nine month release cycles including two months requirements gathering followed by four months development and three months of testing and defect fixing. Before we took over the project it had no process at all, so it is OK now, at least it follows some process. Waterfall is not that bad after all ;-) Currently we are at the end of the development phase and our last sprint will end next Thursday.

Some time ago, the customer suddenly found additional budget and decided to have another requirement implemented. (I have no idea how big organizations can suddenly find more budget, maybe under the mattress of the CEO?) We (the development team) were already booked up with requirements, so he brought in some special people from "Lab X". Lab X is located in off-shoring country Y, but that is not the point here, so let's just assume he found them under a stone. People from lab X had no opportunity to get to know our application, which is eally complex, a Big Ball of Mud with cryptic use cases. Finding your way around our one million lines code base usually takes more than six months for experienced developers.

I had heard about that additional requirement some time ago, but was busy and did not pay much attention. Recently things went bad. The senior developer from Lab X left the company or got rotated to another project, which had already happened before (in another release cycle - another story). I heard that it is common for developers to switch companies in country Y if they get a better offer from a competitor. One or two new junior developers were brought in to continue his work. From what I saw of their code, I do not think they deserve the word "junior" at all: someJavaString == "" is a clear sign that someone has no idea how Java works nor did he or she test the code. I do not blame them, it is not their fault. If you cannot find experienced developers, you have to hire new ones and train them, coach them and let them grow. Maybe they are experienced in another language, I do not know. I just know to expect these things from cheap labour service centres of country Y, where nothing is a problem and everything will be dealt with. "No problem Sir, it will be handled. Everything is OK now."

Right from the beginning two members of our team were asked to team up with the lab to help integrating their code into our product, which - of course - would not need much time. As far as I know they already worked five times as much on the integration as estimated and one went so far to implement parts of the solution on his own time and give it to the lab people to use it instead of their crap, which had not worked. I believe that he should not do that, but I respect my team member to have his reasons, so let him work double shifts if he feels like.

Early on the cycle, some executive or a project manager had signed a document to approve the inclusion of this new feature, so we have no option to escape this mess. I was told that the project manager keeps reminding the customer that his actions have been proven to be problematic but from what I know, the customer's representative is a true alpha-being and will not listen to anything he does not like. (I once had a phone call with a similar executive where I wanted to discuss a certain problem, but during the one hour call I was unable to say a single sentence, so obviously there was no problem and no actions were taken. I really need to improve my communication skills ;-)

So everybody works hard and the current release will be a success including the extra feature, the customer will be happy and nobody will learn anything. As soon as the new feature will be in production, the resources from lab X will vanish and we will be stuck in the quicksand a bit deeper. I hope that the world ends this Xmas as predicted by the Maya calendar and all this ends.

Eclipse Plugin Development

My friend Piotr asked me where to start with Eclipse/RCP development. I am not an expert on Eclipse but it is the main platform of my current employer, and I collected some information while digging deeper into the topic myself. Following an advice from Scott Hanselman I write this blog post instead of an email. This is my list for developers starting with Eclipse development.

• The first pages to read are the Eclipse Plugin and Eclipse RCP Tutorials by Lars Vogella. These are short tutorials with good content which are highly relevant. You might want to start here.
  
  
• Another source for tutorials is G. Prakash's Eclipse Tips, especially his top ten mistakes in Eclipse Plug-in development are highly recommended.
  
  
• More details can be found in the book about Eclipse, Eclipse Rich Client Platform by Jeff McAffer, Jean-Michel Lemieux and Chris Aniszczyk. Throughout the book the authors build an entire application, set up the automated build, create an update site and everything else. This is a comprehensive end to end description.
  
  
• The Eclipse website itself hosts a lot of specialized information. My favourite article is about how to use the JFace Tree Viewer. This does not only show how to use the Tree Viewer, it also explains how to "think in JFace" in general.
  
  
• Finally a lot of articles can be found on IBM developerWorks, just search for RCP. These articles are older, mainly from 2006 to 2008, but most things discussed there are still relevant.

That should be more than enough to get you started ;-)

See the new test fail

The Wikipedia article about Test-driven development describes the process of TDD. It says that after adding a new test, you should run all tests and see if the new one fails. This is part of the red-green-refactor cycle. During my remote pairing activities with Thomas I noticed that I tend to forget this step. To fix this I wrote a tiny Eclipse plugin that complains if a test run does not fail after adding a new test.


The plugin attaches itself to the org.eclipse.jdt.junit.testRunListeners extension point and records the names of all tests during JUnit test execution. (See RedGreenListener.java) When the test session is finished, it compares these test statistics against the previous run. In fact the only thing we need to know is if any test cases have been added. (See TestStats.java) Based on the comparison the plugin decides if to show an annoying popup or not. (See TestRunDiff.java)

The current mechanism is simple and likely to be wrong for special cases:

public boolean firstTestOk() {
    return newAdded.size() == 0 || secondFailed;
}

but it works great on katas and small hobby projects.

The org.eclipse.jdt.junit.testRunListeners extension point is available in Eclipse Europa (version 3.3), but it does not work there. The schema testRunListeners.exsd for the extension point seems to be missing from the JDT/JUnit bundles. The situation changes in Helios (3.6) and JUnit starts to notify the declared listeners. As the current release of Eclipse is version 3.8/4.2, I believe that two versions of backwards-compatibility should be enough.

Download the plugin here. Copy it into your plugins folder and restart Eclipse.

BaDaDam Testing Framework

Some time ago I had a look into BDD - no, not Bullshit Driven Development ;-) - but Behaviour-Driven Development. I started with JBehave but it seemed heavyweight to me. Further it had more than ten dependencies to other libraries, some with possibly problematic licences. Still I wanted to know how that kind of framework worked, so instead of learning by taking it apart I decided to build my own.

For some time I struggled to find a nice name for the project and discussed my problem with Michael. I was mumbling as usual and he mistook BDD as BaDaDam. Voila - I present to you BaDaDam, a minimalistic BDD framework for Java. Obviously it follows the spirit of JBehave, is lightweight, self sufficient and depends only on JUnit. It allows you to write stories in plain text, implement them in Java classes and run them using JUnit. The first version is finished since some time and available in my Maven repository here.

Java 7 aka Dolphin

Four years after the last major release, Java 7 was released July 28th 2011. It brought several new features, the most commonly mentioned were the Diamond Operator, using strings in switch statements, automatic resource management, numeric literals with underscores and improved exception handling. Further there were additions to NIO, the Fork/Join framework and invokedynamic. You have probably heard of all these things by now. There were also some minor additions, but the previous ones have been talked about since long before Java 7 was released.

A good start to check the new features of Java 7 is the presentation about the 55 New Features You (Probably) Didn't Hear About. It also covers some lesser known additions. For a complete list of all changes see the Java 7 Release Notes. They are really worth reading, and include links into the Java Tutorial for more complex topics.

Classlist
Always when a new version of Java comes out, I take a look at its new classes, using my Java Class File Parser. The list of new classes complements the list of new features. Java 7 adds 211 public classes to rt.jar, where Java 6 had 490 and Java 5 even added 547. So it looks like a smaller release.

invokedynamic
I heard that a lot of work went into the invokedynamic byte code instruction which was added to the JVM. It comes with a small, new package java.lang.invoke.

java.lang.invoke.CallSite
java.lang.invoke.ConstantCallSite
java.lang.invoke.MethodHandle
java.lang.invoke.MethodHandleProxies
java.lang.invoke.MethodHandles
java.lang.invoke.MethodType
java.lang.invoke.MutableCallSite
java.lang.invoke.SwitchPoint
java.lang.invoke.VolatileCallSite
java.lang.invoke.WrongMethodTypeException

While this package contains just ten classes, there are another 16 package access classes (not shown here), so invokedynamic is definitely more complex than it looks.

NIO Channels
A more practical addition are some new NIO channels, most noteworthy the AsynchronousChannels, which allow network and file access in an asynchronous way, together with their exceptions.

java.nio.channels.AcceptPendingException
java.nio.channels.AlreadyBoundException
java.nio.channels.AsynchronousByteChannel
java.nio.channels.AsynchronousChannel
java.nio.channels.AsynchronousChannelGroup
java.nio.channels.AsynchronousFileChannel
java.nio.channels.AsynchronousServerSocketChannel
java.nio.channels.AsynchronousSocketChannel
java.nio.channels.CompletionHandler
java.nio.channels.IllegalChannelGroupException
java.nio.channels.InterruptedByTimeoutException
java.nio.channels.MembershipKey
java.nio.channels.MulticastChannel
java.nio.channels.NetworkChannel
java.nio.channels.ReadPendingException
java.nio.channels.SeekableByteChannel
java.nio.channels.ShutdownChannelGroupException
java.nio.channels.WritePendingException

java.nio.channels.spi.AsynchronousChannelProvider

NIO.2
The new java.nio.file package is the largest addition, it contains a brand new API for file access together with file attributes like POSIX permissions, file system and file storage abstractions.

java.nio.file.AccessDeniedException
java.nio.file.AccessMode
java.nio.file.AtomicMoveNotSupportedException
java.nio.file.ClosedDirectoryStreamException
java.nio.file.ClosedFileSystemException
java.nio.file.ClosedWatchServiceException
java.nio.file.CopyOption
java.nio.file.DirectoryIteratorException
java.nio.file.DirectoryNotEmptyException
java.nio.file.DirectoryStream
java.nio.file.FileAlreadyExistsException
java.nio.file.Files
java.nio.file.FileStore
java.nio.file.FileSystem
java.nio.file.FileSystemAlreadyExistsException
java.nio.file.FileSystemException
java.nio.file.FileSystemLoopException
java.nio.file.FileSystemNotFoundException
java.nio.file.FileSystems
java.nio.file.FileVisitOption
java.nio.file.FileVisitor
java.nio.file.FileVisitResult
java.nio.file.InvalidPathException
java.nio.file.LinkOption
java.nio.file.LinkPermission
java.nio.file.NoSuchFileException
java.nio.file.NotDirectoryException
java.nio.file.NotLinkException
java.nio.file.OpenOption
java.nio.file.Path
java.nio.file.PathMatcher
java.nio.file.Paths
java.nio.file.ProviderMismatchException
java.nio.file.ProviderNotFoundException
java.nio.file.ReadOnlyFileSystemException
java.nio.file.SecureDirectoryStream
java.nio.file.SimpleFileVisitor
java.nio.file.StandardCopyOption
java.nio.file.StandardOpenOption
java.nio.file.StandardWatchEventKinds
java.nio.file.Watchable
java.nio.file.WatchEvent
java.nio.file.WatchKey
java.nio.file.WatchService

java.nio.file.attribute.AclEntry
java.nio.file.attribute.AclEntryFlag
java.nio.file.attribute.AclEntryPermission
java.nio.file.attribute.AclEntryType
java.nio.file.attribute.AclFileAttributeView
java.nio.file.attribute.AttributeView
java.nio.file.attribute.BasicFileAttributes
java.nio.file.attribute.BasicFileAttributeView
java.nio.file.attribute.DosFileAttributes
java.nio.file.attribute.DosFileAttributeView
java.nio.file.attribute.FileAttribute
java.nio.file.attribute.FileAttributeView
java.nio.file.attribute.FileOwnerAttributeView
java.nio.file.attribute.FileStoreAttributeView
java.nio.file.attribute.FileTime
java.nio.file.attribute.GroupPrincipal
java.nio.file.attribute.PosixFileAttributes
java.nio.file.attribute.PosixFileAttributeView
java.nio.file.attribute.PosixFilePermission
java.nio.file.attribute.PosixFilePermissions
java.nio.file.attribute.UserDefinedFileAttributeView
java.nio.file.attribute.UserPrincipal
java.nio.file.attribute.UserPrincipalLookupService
java.nio.file.attribute.UserPrincipalNotFoundException

java.nio.file.spi.FileSystemProvider
java.nio.file.spi.FileTypeDetector

The new abstraction for files is the Path, and the helper classes Files and Paths are the main entry points for using the new functionality. The good old java.io.File is not deprecated but considered legacy from now on. Now plain Java can do everything and we do not need to include commons-io any more.

JSR 166y Concurrency Additions
JSR 166y added the Fork/Join framework and a few concurrency related classes which did not make it into Java 5 on time, where the original JSR 166 went in.

java.util.concurrent.ConcurrentLinkedDeque
java.util.concurrent.ForkJoinPool
java.util.concurrent.ForkJoinTask
java.util.concurrent.ForkJoinWorkerThread
java.util.concurrent.LinkedTransferQueue
java.util.concurrent.Phaser
java.util.concurrent.RecursiveAction
java.util.concurrent.RecursiveTask
java.util.concurrent.ThreadLocalRandom
java.util.concurrent.TransferQueue

The main class is the ForkJoinPool together with its two workers, RecursiveAction and RecursiveTask.

Other Classes
There are several new classes all over the JDK,

java.awt.SecondaryLoop
java.beans.Transient
java.lang.AutoCloseable
java.lang.BootstrapMethodError
java.lang.ClassValue
java.lang.ReflectiveOperationException
java.lang.SafeVarargs
java.lang.management.BufferPoolMXBean
java.lang.management.PlatformLoggingMXBean
java.lang.management.PlatformManagedObject
java.net.ProtocolFamily
java.net.SocketOption
java.net.StandardProtocolFamily
java.net.StandardSocketOptions
java.nio.charset.StandardCharsets
java.security.AlgorithmConstraints
java.security.CryptoPrimitive
java.security.cert.CertificateRevokedException
java.security.cert.CRLReason
java.security.cert.Extension
java.security.cert.PKIXReason
java.sql.PseudoColumnUsage
java.util.IllformedLocaleException
java.util.Objects
javax.crypto.AEADBadTagException
javax.crypto.spec.GCMParameterSpec
javax.net.ssl.ExtendedSSLSession
javax.net.ssl.X509ExtendedTrustManager
javax.print.attribute.standard.DialogTypeSelection
javax.security.auth.kerberos.KeyTab
javax.sound.midi.MidiDeviceReceiver
javax.sound.midi.MidiDeviceTransmitter
javax.sql.rowset.RowSetFactory
javax.sql.rowset.RowSetProvider
javax.swing.border.StrokeBorder
javax.swing.JLayer
javax.swing.Painter
javax.swing.plaf.nimbus.AbstractRegionPainter
javax.swing.plaf.nimbus.NimbusLookAndFeel
javax.swing.plaf.nimbus.NimbusStyle
javax.swing.plaf.nimbus.State
javax.xml.bind.JAXBPermission
javax.xml.ws.EndpointContext
javax.xml.ws.spi.Invoker
javax.xml.ws.spi.http.HttpContext
javax.xml.ws.spi.http.HttpExchange
javax.xml.ws.spi.http.HttpHandler

The changes cover additions to UI capabilities, e.g. NimbusLookAndFeel, new security algorithms and updated JDBC, XML and WS packages. The most interesting classes are AutoCloseable for the new try-with-resources statement and Objects with its requireNonNull(T obj) method.

Summary
Java 7 has some nice additions to core Java, but still feels like a smaller release, not as ground breaking as Java 5. The most useful thing might be the Diamond Operator because it saves a lot of typing.

See the complete list of all classes available in Java 7. Each class name is annotated with [release] showing the release it first appeared, e.g. java.lang.annotation.Annotation [5]. Package access classes are indicated by a lowercase p, e.g. java.lang.CharacterName [7p].

Jakarta Commons Cookbook

This summer I finished reading the Jakarta Commons Cookbook. I bought it several years ago and it was sitting on my shelf since then. It survived the last "spring cleaning" when I gave away most of my books to the local Java user group, regardless if I had read them or not. I was not very enthusiastic about it, but felt the need to read it as it covers some of the most important Java libraries available today - the Apache Commons.

Review
Timothy chose a way to organize the book around common tasks, so the book is in the usual cookbook/recipes style. To me the cookbook style is rather verbose and there are many examples on minute details which make the book suitable for beginners. If you are experienced and have used Apache Commons before, the book is a quick read.

Although the book covers a lot of material, it is just a glimpse of the whole Apache Commons universe. An extensive description of all the Commons projects is impossible so squeeze into a book. Still Timothy wrote about the most important projects like Lang, Collections, IO, Math as well as about some Apache top-level projects like Velocity and Lucene.

The cookbook has been published in 2004 and shows its age. A few Apache projects described in it have retired in the meantime and some parts of the shown APIs are deprecated. Still only a few changes were needed to compile all examples against the newest versions of Commons libraries. So the content in the book is still relevant.

Examples
I wanted to keep all the source code of the book as a quick reference and as a source to copy from. Unfortunately O'Reilly did not provide the code for download or I just did not find it. So I extracted the code samples and expected output from the eBook based on their markup. My script parsed the eBook and saved the Java examples into packages derived from the chapter names and classes derived from the section names. It added a main method and appended the expected output as a comment after the code. For example the extracted code for recipe 1.11 about finding items in an array looked like that

package com.discursive.jccook.supplements;

import org.apache.commons.lang.ArrayUtils;

public class FindingItemsInArray {

   public static void main(String[] args) {
      String[] stringArray = { "Red", "Orange", "Blue", "Brown", "Red" };

      boolean containsBlue = ArrayUtils.contains(stringArray, "Blue");
      int indexOfRed = ArrayUtils.indexOf(stringArray, "Red");
      int lastIndexOfRed = ArrayUtils.lastIndexOf(stringArray, "Red");

      System.out.println("Array contains 'Blue'? " + containsBlue);
      System.out.println("Index of 'Red'? " + indexOfRed);
      System.out.println("Last Index of 'Red'? " + lastIndexOfRed);
   }

   // Array contains 'Blue'? true
   // Index of 'Red'? 0
   // Last Index of 'Red'? 4

}

And then I went overboard with this little project. For a week I worked to have all examples compile and run. This was hard work because the examples contained many syntax errors like missing semicolons or typos in variable names. These mistakes were no problem for human readers, but the compiler complained a lot. For some examples I had to second guess the code not shown, e.g. used Java beans or factory methods which had been omitted for brevity. In hindsight it was a stupid idea, but after successfully cleaning up more than half of the examples, I just had to finish the work. Remember, I am a completionist ;-)

In the end I won and now all the examples are mine! I would like to share them with you, but O'Reilly does not allow that. The examples are distributed under some special "fair use" license that prohibits "reproducing a significant portion of the code".

Design Lessons

Just finished my weekly cycle workout. As usual I watched a presentation, this time it happened to be Deep Design Lessons by Michael "Legacy Code" Feathers, recorded at NDC Oslo 2012. In this talk he outlines some of the gems that he feels are less understood today. You really need to see this. There is so much to learn from his few slides. Now go, watch it! (Thanks to Claus for sending me the link.)

When writing about talks by Michael Feathers, I always need to mention my all-time favourite Deep Synergy Between Testability and Good Design. I am repeating myself, but I highly recommend watching it. It is an excellent talk examining the relationship among test-pains, code smells and design principles.

If you want to follow what I watch, here are the presentations I liked.

Remote Pair Practice

Deep in my notes of past conferences I keep a quote that says Writing code together is the only true way that programmers communicate. Even if you do not agree, pair programming is still a great way to share with one another. Whenever I program in a pair, I learn a lot. Unfortunately I have a very little opportunity to do so as my team is distributed around the globe. After attending a Test Driven Development workshop at this year's GeeCON I decided to do something about it.

Schedule
I plan for a pair coding activity each week. This seems like a good target to aim for. To free one or two hours each week for deliberate practice should be possible for everybody. Unfortunately I do not reach my target because I might be busy, might still look for a pair or my pair might be busy as well. In the past I only managed to participate in a pairing session twice a month. By writing this post I hope to save time on explaining and to improve the frequency of my pair programming. I found it better to schedule throughout the day instead of the evening. It also seems easier to find a suitable time at work days instead of weekends. Keeping away from evenings and weekends helps because there are still people with some private life ;-). An appointment at a fixed day and time is an option when pairing with the same person again and again, but to meet with different people, one has to be flexible. I do not care for the time as long as it is somewhere in my (European) time zone. To find suitable times in advance I use Doodle.

Pair Programming
A lot of people are afraid of pair programming because they never did it. To get started you should read James Shore's description of Pair Programming from his book The Art of Agile Development. There is even a short summary consisting of 99 words for the people who do not have time to read. In the end the rules how to behave during pair programming, are about practising everyday civility, because all I really need to know about pair programming I learned in Kindergarten. So be especially courteous. Pair programming teaches us to collaborate. I recommend watching Angela Harms' presentation about the (social) anti patterns in pair programming, Does pair programming have to suck? Note that pair programming may be difficult and uncomfortable for the first few times.

Get-Together
Obviously we have to meet in order to pair. In the past I managed a few times to meet during lunch breaks and have a short pairing session, much like Code & Coffee. Now I always bring my laptop when I meet my friends for lunch, maybe he is interested in a spontaneous pairing session while eating?

Remote Setup
Meeting in person is preferable but not always possible. Remote pairing is mainly about the additional tooling and all rules from regular pairing apply as well. My friend Thomas wrote a good post about Pair Programing from which I took all the material of this section. We use Skype to talk and TeamViewer to share our screens. It helps to have Skype contacts exchanged in advance so one can start the remote session by sending a chat message to the other one. As soon as the line is established we discuss who will host the session. A good headset is recommended, otherwise the voice quality is poor.

Then the host starts his TeamViewer and sends his TeamViewer id in Skype. I never had any problems with TeamViewer and it works great, even when connecting Windows and Mac machines. TeamViewer is a commercial tool but free for non-commercial use. A pairing session is definitely for educational purposes, so I do not see any problems with the license. After connecting I recommend changing the resolution of the host machine to the maximum resolution supported by the client. As a client I use TeamViewer's options View > Original and and View > Full Screen to get rid of all these scroll bars. If I have an additional screen I put the Skype window and a browser window there, if I want to look something up quickly. This setup is much like Level Four Pair-adise, except that it is remote.

It happens that I would like to scribble something on a white board. There is a nice Web Whiteboard, which allows drawing and shared editing. It is as simple as a real white board. If you need something more advanced, create a new Scribble in Google Docs which allows shared editing as well. All these and even more advanced topics of remote pairing are discussed by Joe Moore in his excellent talk about Remote Pair Programming: People and Technology.

Code Kata
An educational pairing session looks much like a Coderetreat. We might do a code kata or whatever coding problem seems adequate to work on. A code kata is an exercise in programming which helps a programmer improve his or her skills through practice and repetition. When I meet someone new, I do not impose any special rules. We would just work on a kata, preferably one that we both know, using ping-pong styled Test Driven Development. Todd Sedano compiled a list of code katas sorted by how easy it is to apply TDD. Fizz Buzz and Prime Factors are too short, but all other katas beginning with String Calculator are suitable. In fact the size of the kata does not matter, because it is not about finishing, it is about improving.

The choice of programming language is not important to me. Although my primary language is Java, I have done katas in Ruby, Scala, JavaScript and C# as well as some other, less common languages. After choosing the kata, the language and the development environment to use, we need to agree on the focus of the training session. There are some variations of katas possible, e.g. to use a more functional programming style or not to us any primitives or conditionals, up to the (at least for me) extremely difficult TDD as if you meant it.

With Tomatoes
It happened that we would lose track of the time while focusing on the problem. Later, for example after working on the kata for two hours, we would "wake" up and be totally exhausted. To prevent this, and to create some space for chit-chat, I use the Pomodoro Technique. There is a break of 5 minutes every 25 minutes. I just watch out to stop the Pomodoro with a failing test, so we know where to continue after the break.

Conclusion
As shown my remote pair practice contains everything that is good and holy: pair programming, cool tools like video conferencing and screen sharing, coding, TDD, code katas and Pomodoros. Can you resist?

Code Cop Kofler

Summer began last month in Austria and I went abroad and got myself a lovely souvenir: a carved wooden desk name plate. I was not sure what to write there and thought about my twitter name but did not trust the artificer to cope with the @-sign and ended with some kind of hybrid name:


In hindsight I was clever not to ask for something complicated because the guy messed it up twofold and did not even get the letters of my name right. Nevertheless it is an icon of my office and I love it. If only I had a desk to put it on.

My Open Source Involvement

My current employer is refreshingly serious about software licenses. For example if an open source library is GPL licensed, it just cannot be used in a product or service offering. End of story. All other companies, I have ever worked for, just did not care. I like being serious about software. But it is not just about using the software, in fact it is about being contaminated with someone else's intellectual property, whatever this might be. To comply with my employer's bureaucracy I have to report my open source involvement and get written permission from my manager. Ok, so let's collect all my active open source projects and list them here as well. Some advertising never hurts.

• It began with PMD (BSD License), a source code analyser for Java, which I used a lot. In 2004 I started submitting patches and later I created my own set of custom rules as Maven module. Unfortunately the new PMD 5.0 is not backward compatible, so I have some pending work to do here. (Source Code)
  
  
• In 2008 I created a system tray notifier to monitor build servers under BSD license. I use it myself and have to create maintenance releases from time to time. (Ah delicious dog food ;-)
  
  
• javaclass-rb is a Java class file parser in Ruby. I created it in 2009 and I there is a small release once a year. Although it is not finished, it is particularly useful to analyse custom Java code. I hope that I will have some time in the future to describe it in detail.
  
  
• Since last year I am organizing Hackergarten Vienna. Hackergarten is a craftsman's workshop, a classroom, a laboratory, a social circle, a writing group, a playground and an artist's studio. We meet once a month and work on Open Source. During this time I have submitted patches with working code to Commons Exec, Castor, Commons Email and JMeter (all under Apache License).
  
  
• My latest project is BaDaDam, an experimental BDD framework for Java. It allows you to write stories in plain text, implement them in Java classes and just run them using JUnit. Some people from Hackergarten have helped me with contributions.
  
  
• Occasionally I submit pieces of code to Rosetta Code, a programming chrestomathy site.

T-shirt Update

I love t-shirts and keep wearing them regardless of my employer's dress code. I had only two complaints about them in the last year, so things worked out quite well ;-) The last time I wrote about Code Cop t-shirts was more than two years ago, so it is definitely time for an update.

Don't Repeat Yourself
The Don't Repeat Yourself or DRY principle states that every piece of knowledge must have a single, unambiguous, authoritative representation within a system. In simple terms this means that there should be no duplicates or repetition. The dark blue DRY shirt reminds me to write things once and only once. It is also my best selling shirt ever, as a friend bought one after he was exposed to horrible duplicated code at the client's site. Unfortunately I did not get rich as I bought him a beer in return.

Keep the bar green to keep the code clean
Keeping the bar green is the primary motto of JUnit, the popular unit-testing framework for Java. This grey shirt displays a simplified image of the Eclipse JUnit runner together with a green bar. I had problems finding the right dimensions for the characters and the green check marks and threw away at least three versions until I got it right. One day I wore it in the office and one of my colleagues, a project manager, asked me "green is good, right?" I just love them.

Red-Green-Refactor
Red-Green-Refactor is the cycle of Test Driven Development. You write a failing test which results in a red bar. Then you write some code until the bar is green. Finally you clean up the code you just wrote, e.g. improve the names or remove duplication. (Remember DRY?) For the white shirt I used typefaces according to these phases: Red is scary, I do not like it, so I used a Halloween typeface for it. Then I go for a green bar as fast as possible, so I used a dynamic typeface for that. My result after refactoring is sorted and tidy symbolized by a regular typeface.

(Buy Code Cop shirts)

Eclipse Plugin Unit Testing

I will describe the set-up of a unit testing infrastructure for a big RCP application consisting of several plugins. In the first part I will write about how to run JUnit tests inside Eclipse.

Test Plugins
First we need a place to put the tests. We do not want to put the tests in the same project as the production code because we do not want to ship the tests. So for every plugin which we want to write tests for, we create a separate plugin with the same name suffixed with ".test". For example let's assume we want to write a JUnit test for a class in the company.product.general plugin. First we look for a plugin named company.product.general.test. Does it exist? Maybe we never used it and need to check it out from version control. Still nothing? OK, so we need to create it.

For a new test plugin we create a fragment project. We do that by selecting File > New > Project > Plug-in Development > Fragment Project. A fragment allows us to add code to another plug-in, known as the host. The fragment is merged with its host and has full access to all the packages declared in its host, not only the exported ones. Among other ways to set up tests for plugins, fragments generally are the best solution. So our unit tests have access to all classes of the host plugin. The fragment inherits all dependencies from the host. The only explicit required dependency of the test plugin is org.junit4. (Remember that various files in the build system might need to be updated for new (test) plugins to be part of the build. These might include some kind of loading rules, certain Ant build files or test features.)

The above set-up is enough to run simple JUnit tests which do not use any feature of Eclipse's OSGI container. To enable PDE JUnit tests the host plugin needs to define Eclipse-ExtensibleAPI: true in its MANIFEST.MF file. This allows the fragment to access classes from the host plugin and tells the PDE tooling to add the fragment to the classpath of the host plugin.

Test Packages
We use the plugin's name as base package of the plugin's source code to avoid name collisions. For example all packages of the plugin company.product.general are sub packages of company.product.general. This is not the case for fragments, as fragments are merged with the host. All packages of the company.product.general.test fragment are still sub packages of company.product.general.

The package with the same name as the test fragment is used for common test set-up or custom assertion logic that might be used for more than one test. For example the package company.product.general.test might contain a class called UsLocale, which sets the current locale to US, as needed to test some business logic. Common test code can be reused by other test plugins if it is exported from the fragment. This works the same way as for normal plugins.

Warning about "Add Required Plugins"
Sometimes it is necessary to add new plugins to a launch configuration in order for the application to start. When you do that by using the "Add Required Plugins" button in the launch configuration, make sure that all the test plugins are not selected afterwards. We had some issues when test-only functionality somehow "leaked" into production.

Simple JUnit Tests
When the test plugin is ready, we can start writing simple JUnit tests. They will not necessarily be "simple", they are just called like that to distinguish them from PDE JUnit tests which have a more complex set-up. Even when developing RCP applications, these simple JUnit tests have their use. We can use them to test any class that is not dependent on the Eclipse Platform API. They are particularly useful for testing business logic and model classes.

JUnit tests go into the same Java package as the class under test. So the test plugin has the same packages as the host plugin. Test classes are named like the class under test suffixed with "Test". For example let's assume we want to test the class company.product.general.api.CatchHandlers in the plugin company.product.general. The related test is company.product.general.api.CatchHandlersTest in the company.product.general.test plugin.

Mocking
Unit testing is supposed to test classes or methods in isolation but classes usually depend on other classes. For more complex scenarios we want to use different objects than the live ones to get rid of these dependencies. These objects are provided to the class under test. By using them we can verify logic independently of the depended-on objects. Such objects are called Test Doubles.

Dummy objects, fakes and stubs are created easily with custom classes. But true mocks need some more logic to verify the order of method invocations (which needs some kind of state machine). Fortunately there are several frameworks that bring mocks to Java, for example EasyMock. To use it in our RCP application we need to create on OSGI bundle (i.e. an Eclipse plugin) from its jars. Let the plugin company.product.test.mock contain the mocking framework's jars. To use its classes in tests add it to the dependencies of the test fragment.

PDE JUnit Tests
PDE or Plugin JUnit tests are executed by a special test runner that launches another Eclipse instance and executes the test methods within that workbench. So PDE JUnit tests can be used for classes that are dependent on Eclipse Platform API. Parts of the Eclipse platform are available outside a running workbench as well, so a normal JUnit test might still be an option. Additionally some other parts, for example the extension registry can be mocked. But there are cases where a running workbench is necessary. For example you may assume a particular behaviour of Plugin.getStateLocation(), which would not be available if you were just using an ordinary JUnit test. Besides that the tests themselves look exactly like any other test.

When automating the build these PDE tests should be distinguished from basic ones. So we follow a different naming convention and call them *PlatformTests. Also note that the usual class extension mechanism might not work inside the workbench. You need to introduce interfaces for the types you want to mock.

GeeCON 2012

Last week one of my favourite Java conferences GeeCON took place in Poznan, Poland. I was there before and liked it a lot. Despite that it took me 13 hours train travel to Poznan, it was a very good conference. The organisers were caring for everything (even free beer ;-), the presentations were interesting and some of the attendees were really passionate about our craft.

I am too lazy to write a full round-up of all sessions that I attended, but at least I will give you a list of my favourite sentences heard during the sessions:

• Maintenance starts with the first check-in. (Thomas Sundberg)
• 40 errors, 38 warnings. That looks bad. Let's commit that. ;-) (Thomas Sundberg)
• object null extends Nothing() (Ceylon Language)
• Threads do not work. (Bruce Eckel)
• It's not getting easier being a developer. [... because there is more and more we need to know.] (Ivar Jacobson)
• An architecture without executable code is a hallucination, but code without architecture is shit. (Ivar Jacobson)
• ... f*cking ... f*cking ... f*ck ... [He really does not like Java any more ;-)] (Gavin King)
• I write crap on daily basis. ;-) (Tomek Kaczanowski)
• @Seed("deadbeef") [This is a valid hex number.] (Dawid Weiss)
• Every time you write a for loop, god kills a kitten. (Keith Braithwaite)
• Simple design is for simple systems. [... making fun of overly complex solutions.] (Thomas Sundberg)
• The more green it is, the more green it is. [... the build.] (Wojciech Seliga)
• Heal or Kill. [... these flaky tests.] (Wojciech Seliga)
• ... 50.000 LoC. - That are five operating systems. Does it do that much? (Kevlin Henney)

Thank you all for the great show!

Mocking the Eclipse Extension Registry

Some classes of a RCP application might make use of Extension Points, calling Platform.getExtensionRegistry() and evaluating the returned configuration elements. It is possible to mock (in fact stub) the default registry provider by calling RegistryFactory.setDefaultRegistryProvider() with a registry implementation. By using a fake registry some PDE JUnit test cases can be converted to plain JUnit tests, which speeds up their execution.

A Changing Registry Factory
The default registry provider can only be set once. So we need to register a wrapper, which delegates to a RegistryFactory so it can be changed among tests.

import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.IExtensionRegistry;
import org.eclipse.core.runtime.RegistryFactory;
import org.eclipse.core.runtime.spi.IRegistryProvider;

public class ChangingRegistryFactory {

  private static IRegistryProvider registryProvider;
  private static boolean defaultRegistryHasBeenSet;

  public static void setDefaultRegistryProvider(IRegistryProvider provider)
                                                       throws CoreException {
    setRegistryProvider(provider);
    synchronized (ChangingRegistryFactory.class) {
      setDefaultRegistryProviderOnce();
    }
  }

  private static void setRegistryProvider(IRegistryProvider provider) {
    registryProvider = provider;
  }

  private static void setDefaultRegistryProviderOnce() throws CoreException {
    if (!defaultRegistryHasBeenSet) {
      setDefaultRegistryProvider();
      defaultRegistryHasBeenSet = true;
    }
  }

  private static void setDefaultRegistryProvider() throws CoreException {
    RegistryFactory.setDefaultRegistryProvider(new IRegistryProvider() {
      public IExtensionRegistry getRegistry() {
        return registryProvider.getRegistry();
      }
    });
  }

}

An Extension Point Mockery
Using EasyMock or any other mock framework we can create a mock registry and register it to the RegistryFactory (in fact to the ChangingRegistryFactory). To hide the mocking details we provide a specific builder like API in the helper class ExtensionPointMockery which helps mocking Extension Points:

import static org.easymock.EasyMock.*;

public class ExtensionPointMockery {

  private final IExtensionRegistry registry = createRegistry();

  @SuppressWarnings("unused")
  public ExtensionPointMockery() throws CoreException {
    // allow CoreException to be thrown from initialiser blocks
  }

  private IExtensionRegistry createRegistry() {
    return createMock(IExtensionRegistry.class);
  }

  public IConfigurationElement confWith(Object specificInstance)
                                                       throws CoreException {
    IConfigurationElement confElement = createMock(IConfigurationElement.class);
    expect(confElement.createExecutableExtension("class")).
          andReturn(specificInstance).anyTimes();
    replay(confElement);
    return confElement;
  }

  public IConfigurationElement mockFor(Class<?> type) throws CoreException {
    Object mockInstance = createNiceMock(type);
    replay(mockInstance);
    return confWith(mockInstance);
  }

  public void set(String key, IConfigurationElement... elements) {
    expect(registry.getConfigurationElementsFor(key)).
          andReturn(elements).anyTimes();
  }

  public void setAll(IConfigurationElement... elements) {
    expect(registry.getConfigurationElementsFor(anyObject(String.class))).
          andReturn(elements).anyTimes();
  }

  public void close() throws CoreException {
    replay(registry);

    IRegistryProvider provider = createMock(IRegistryProvider.class);
    expect(provider.getRegistry()).andReturn(registry).anyTimes();
    replay(provider);

    ChangingRegistryFactory.setDefaultRegistryProvider(provider);
  }

}

Usage
Its typical usage looks like

new ExtensionPointMockery() {{
  set("preferenceSettings", mockFor(IPreferenceSettings.class));
  set("filterManager", confWith(new FilterManagerDelegate()));
  set("postProcessor");
}}.close();

where confWith() puts the given object directly into the registry, simulating its contribution and mockFor() creates a "nice mock" (a fake) of the given interface and adds it to the configuration. The call of set() without any parameter initialises the extension registry for the given name but without any contributions. Finally close() creates the mock registry and registers it to the platform.

Happy mocking!

JUnit Tutorials

I prepared a short list of tips and links for some colleagues to get started with JUnit.

• A good start for an absolute beginner is the JUnit Cookbook.
• JUnit is integrated in Eclipse. Lars Vogel's JUnit Tutorial shows how to write and execute tests using Eclipse. Talking about Eclipse, I always use the keyboard short-cut ALT-SHIFT-X and then key T to launch the test open in the editor window. After that I use the keyboard short-cut CTRL-F11, which runs the last launched test or application, to re-run my JUnit test until it succeeds.
• JUnit comes pre-packaged with Hamcrest, a framework for writing matcher objects. These matchers improve the readability of tests and provide better failure messages. Consider writing a custom Matcher if you need to compare large objects with one another based on complex state.
• A mistake that I see quite often is the handling of expected exceptions in tests. Szczepan Faber has written down five rules how to do that and to avoid any problems.
• To explore more advanced topics of JUnit see my presentation on Practical Unit Testing (June 2009).
• Also make sure that your tests are deterministic. Non-deterministic tests are a serious threat to the discipline of unit testing.

All these links focus on using the JUnit technology, but do not explain how to write good tests. If you have some spare time and would like to know more about JUnit and unit testing in general I recommend the book Pragmatic Unit Testing in Java with JUnit.

Behaviour - the "New" Way
By the rise of BDD the common understanding of unit tests has changed. Even if you do not do TDD or BDD these things apply to any unit test.

• They are now called Micro tests to distinguish them from traditional unit tests. Tests with a larger scope, e.g. integration or end-to-end tests, are no unit tests even if they make use of JUnit.
• Further the names of test methods should be full sentences focusing on behaviour. JUnit 4.x removes the need to prefix test methods with test, and usually the sentence of the expected behaviour starts with the word should.
• The test methods should be created using the Arrange-Act-Assert or even better the Given-When-Then pattern.

Recommended Watching
Finally I highly recommend the recording of The Deep Synergy Between Testability and Good Design by Michael Feathers. It's an excellent talk examining relationship among test-pains, code smells and design principles. Go, watch it!

The Diaries Continue

My employer, or to be more specific the department I am working for, is hiring senior Java developers. This is a rare opportunity as usually IT related jobs move from high-wage economies to places where wages are lower. So people keep asking me how am I doing. Here is an update of my diaries. As usual all my writing is fictional and has no resemblance to any real people or companies.

Finding a Friend
In August I made sort-of a friend. He is from another team and our teams are not related, but we share the same version control repository. By accident I saw some crazy things in his changes so I mailed him. After a nice discussion he started reviewing my changes and sent me an email whenever he spotted something that looked suspicious. In the same way I have annoyed people all over my own team with their code, so no one hesitates to send me a note whenever I misspell something. Sweet Revenge Huh.

Consequences of Diaries
A few days after I had published my first diary, my boss approached me and asked me about it. I was frightened. Had I gone too far and pissed someone off? I checked the blog post several times to make sure it was neither aggressive nor diminishing. My fear was unfounded. As I had been wearing my own shirts with my URL code-cop.org on it, a colleague had visited my blog and read it. He was concerned because my writing seemed depressed. So my manager took me aside to confirm that I was still the right person for the job and that he had no doubts about me. It was good to know that someone cared about me, especially at the beginning of a new job.

How to Order a Book
After buying a book about some technology being used in the current project I asked my manager for a refund. I used to ask for a refund of book expenses. It was not about the money but more about a gesture of my manager that he or she appreciated and supported me reading technical books on my personal time. Unfortunately, during most of my jobs, I was not supported and my requests were rejected with ridiculous excuses. So eventually I stopped asking. Refreshed by the new job I did ask my new manager without expecting much. Again he surprised me. First he checked how to fill the form for a refund, then he helped me, in person, to fill it. Later he kept me updated about the state of the approval process without even me asking. That made me feel supported like never before.

Error Handling
It turned out that our application had serious problems in its error handling. Pieces of code like

catch (Throwable t) {
   throw new Error(t.getMessage());
}

or my favourite code snippet to throw all exceptions,

for (Throwable exception : exceptionList) {
   throw exception;
}

were not uncommon. And I am not talking about the 636 empty catch blocks placed throughout the application.

Strong Language
Nevertheless things went well. The crappy code did not improve on its own but we grew a team and started cleaning up the mess. I kept finding awful heresies of code which had been freshly perpetrated and did not hesitate to point them out. Once a particular change made be very unhappy - well not only unhappy, but it made me sick. It contained everything you would not like to see in your code, for example strange boolean expressions like if (isChanged == true) instead of if (isChanged), useless field names like _dpL, magic numbers, large amounts of commented code, long lines mindlessly formatted and even more problems. I spent an entire hour summing up the problems and proposing ways to fix them. I contained myself but I might not have been particularly polite. Afterwards I had a chat with the developer to make sure that he was neither insulted nor angry with me and everything looked fine. But the next day my manager showed me an email, he received from the developer complaining about my strong language that had made him feel uncomfortable. (Strong language is a markedly or unwarrantedly forcible or vehement manner of expression or choice of words.) I admit that I am direct and I understand that do not and wrong might be considered strong language by certain people, but I have also strong opinions about what you just do not do in your code and when some functionality is in the wrong place. He should consider himself lucky that I did not follow Alberto Savoia's Management Approach.

CV Wizard
We are a service organization and every employee is asked to enter his or her curriculum vitae into an internal database. When a proposal for a customer is prepared, sales people browse this database and choose suitable people for the project. Although I was on an active project the reminder emails of the CV wizard kept pestering me. After a month I gave in and started to fill in my data. It looked easy but was not. For each passed project I had to figure out my role, my tasks, responsibilities, contributions and accomplishments. "I created code, it was clean and worked" did not quite fulfil these requirements. Additionally I had to choose my words carefully as the CV was supposed to help sales people and should communicate "I deliver client value" at least in every third sentence. It took me 11 hours to finish my CV but in the end I liked the result. I have never known I did so many great things ;-)

Incoming Changes
I used to review incoming changes. I did not do formal reviews, just browsed through the changes before accepting them, read the descriptions and had a look at the differences if I felt like it. Some day in November I stopped doing that as it was pointless. Usually I managed to read three or four changes before I fired up my email client and started writing about the given architecture, coupling, separation of concerns or similar things. It just took too much time, therefore I could not afford to spend several hours each day on informal code reviews. So I had to "trust" my colleagues instead.

Learning Plans
To get to know the company, new hires were supposed to work through special learning plans each consisting of several hours online presentations about various topics related to the company's policies, workplace habits and offerings. End of November I finished the first one. Finally I started getting an idea what the company was all about.

Goodbye Eclipse
In the middle of December I wrote my last piece of code. Caring about the overall product, development process and architecture I (was) moved more and more into backlog definition, sprint planning, setting up the development infrastructure, code reviews and acting as team lead. I spent most of my time attending meetings and writing emails. It makes me unhappy.

Desk Sharing
I do not like desk sharing. Setting up my keyboard and aligning my monitor each day is just a waste of time. Further most of the screens are so dirty that I need two cleaning pads to wipe them. It looks like I am the only person caring for clean screens in my area of the building. With the beginning of January things got worse. The company started a big project and lots of new people were joining. The office got crowded and - in the end - was running out of space. Some colleagues started working from home several days a week. While working from home may be desirable under certain circumstances, it was not helping us at all. Informal communication stopped happening and the team started falling apart. As a late worker it made my life especially difficult. When I arrived at the office most desks were already occupied. There was no way I could sit near my team (or at least what was left of it). Well, almost no way. I chose to sit on a coffee table in a break area next to my team for several weeks. That situation really annoyed me. Since 2005 I have been working with two screens all the time, even at home, and suddenly I was downgraded to using a small Laptop screen and no proper place to sit.

No Time to Write Unit Tests
As I said before, there were several teams developing a family of applications and sharing the same version control repository. There was no separation of these applications and the teams were connected and somehow depending on one another. I was always happy to spread the idea of clean code and did not stop inside my team. A colleague from a smaller, recently introduced application showed interest in raising the quality of his work. He asked me for help to introduce unit tests into his development work and I was more than eager to help him. He is smart and understands the concepts. Short after that he started writing useful tests. Unfortunately things did not go well and in the end he was forced by his team lead to abandon writing unit tests. Of course the team lead did not tell him to stop writing tests. He just asked my colleague to get more done and that he should not even implement the requirements completely, just implement the happy path of as many features as possible. That made me angry and depressed at the same time.

Resource Action
Last month I overheard my manager talking about a serious situation in the company's US branch. It was about some 'resource action'. I did not understand what it was about and did not care, US was far away. At least it was far away until I opened the intra-net page the next day. I guy that I knew was affected by the resource action (read layoff) and had been axed. I was shocked because Robi B. was no ordinary employee, rather one of the more dedicated ones. We did not share any work but I kept noticing his posts and comments all over the place. Well, probably not all over the place, as "this place" was huge, but at least in the areas that mattered to me. Recognizing him as a community builder and enthusiastic individual, I got in touch. Robi believed in innovation and organized Hackday beside his regular work. (Hackday was a community of people engaged in creating new tools and brainstorming great ideas about how to make their work lives better. For more information follow @hackday.) Robi had been with the company for almost 15 years and had started several innovative projects.

Prime Factors XSLT

Here is a short conversation we had at Hackergarten Vienna yesterday:

I did the Prime Factors Kata in XSLT. Would you like to review it?
You did WHAT?
You know, this small program. It's calculating the prime factors of a number.
But what? In XSLT? Why XSLT?
I know that it sounds a bit crazy. I had performed the kata in all programming languages which I know, e.g. Java, Ruby and even old BASIC or Turbo Pascal, but not using XSL transformations. So I just had to do it.

Initial Version
Using XSLTunit I created a test case that would apply the template to <value> to calculate the prime factors of the given source.

<xsltu:test id="test-template-factors-one">
  <xsl:variable name="source">
  <value>1</value>
  </xsl:variable>
  <xsl:call-template name="xsltu:assertEqual">
  <xsl:with-param name="id" select="'template factors 1'" />
  <xsl:with-param name="nodes1">
    <xsl:apply-templates select="exsl:node-set($source)/value" />
  </xsl:with-param>
  <xsl:with-param name="nodes2"></xsl:with-param>
  </xsl:call-template>
</xsltu:test>

...

<xsltu:test id="test-template-factors-four">
  <xsl:variable name="source">
  <value>9</value>
  </xsl:variable>
  <xsl:call-template name="xsltu:assertEqual">
  <xsl:with-param name="id" select="'template factors 4'" />
  <xsl:with-param name="nodes1">
    <xsl:apply-templates select="exsl:node-set($source)/value" />
  </xsl:with-param>
  <xsl:with-param name="nodes2">3 3</xsl:with-param>
  </xsl:call-template>
</xsltu:test>

and so on. The result looked like that:

<xsl:template match="value">
  <xsl:call-template name="generate">
  <xsl:with-param name="number" select="number(current())" />
  <xsl:with-param name="candidate" select="number(2)" />
  </xsl:call-template>
</xsl:template>

<xsl:template name="generate">
  <xsl:param name="number" />
  <xsl:param name="candidate" />
  <xsl:choose>
  <!-- one is no prime and does not have any factors -->
  <xsl:when test="$number = 1"></xsl:when>
  <!-- candidate == number, so number is a prime -->
  <xsl:when test="$candidate = $number">
    <xsl:value-of select="$number" />
  </xsl:when>
  <!-- number is factored by the candidate, factor it -->
  <xsl:when test="$number mod $candidate = 0">
    <xsl:value-of select="$candidate" />
    <xsl:text> </xsl:text>
    <xsl:call-template name="generate">
    <xsl:with-param name="number" select="$number div $candidate" />
      <xsl:with-param name="candidate" select="$candidate" />
      </xsl:call-template>
    </xsl:when>
    <!-- try again with the next factor -->
    <xsl:otherwise>
      <xsl:call-template name="generate">
        <xsl:with-param name="number" select="$number" />
        <xsl:with-param name="candidate" select="$candidate + 1" />
      </xsl:call-template>
    </xsl:otherwise>
  </xsl:choose>
</xsl:template>

StackOverflowError
Using javax.xml.transform.TransformerFactory from Java 6 to run the XSLT transformation, the last prime that was processed successfully was 7351. Calling generate with the following prime (7369) caused a StackOverflowError inside Apache Xalan due to the recursion. As there was no need to try candidates larger than the square root of number, I replaced the test for $candidate = $number with $candidate * $candidate > $number. This brought me as far as 54184313.

Reducing the Stack Depth
XSLT is a functional language, there are no loops, the only way to iterate is recursion. So the size of the largest prime to process is limited by the size of the stack. Still half of the recursions are useless because no even number larger than two can be a prime.

<!-- try again with the next factor -->
  <xsl:otherwise>
    <xsl:choose>
      <xsl:when test="$candidate = 2">
        <xsl:call-template name="generate">
          <xsl:with-param name="number" select="$number" />
          <xsl:with-param name="candidate" select="$candidate + 1" />
        </xsl:call-template>
      </xsl:when>
      <xsl:otherwise>
        <xsl:call-template name="generate">
          <xsl:with-param name="number" select="$number" />
          <xsl:with-param name="candidate" select="$candidate + 2" />
        </xsl:call-template>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:otherwise>

This can be improved further by unrolling the (recursive) loop. If number is not factored by candidate + 2 then the next one is candidate + 4 and so on.

<!-- try again with the next factor -->
  <xsl:otherwise>
    <xsl:choose>
      <xsl:when test="$candidate = 2">
        <xsl:call-template name="generate">
          <xsl:with-param name="number" select="$number" />
          <xsl:with-param name="candidate" select="$candidate + 1" />
        </xsl:call-template>
      </xsl:when>
      <xsl:when test="$number mod ($candidate + 2) = 0">
        <xsl:call-template name="generate">
          <xsl:with-param name="number" select="$number" />
          <xsl:with-param name="candidate" select="$candidate + 2" />
        </xsl:call-template>
      </xsl:when>
      <xsl:when test="$number mod ($candidate + 4) = 0">
        <xsl:call-template name="generate">
          <xsl:with-param name="number" select="$number" />
          <xsl:with-param name="candidate" select="$candidate + 4" />
        </xsl:call-template>
      </xsl:when>
      <xsl:otherwise>
        <xsl:call-template name="generate">
          <xsl:with-param name="number" select="$number" />
          <xsl:with-param name="candidate" select="$candidate + 6" />
        </xsl:call-template>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:otherwise>

Required Reading: Clean Code

Here is an email that I wrote to my team earlier this year. The team members are able to deliver new features on time but do not care for code quality (at least not as much as I do ;-). This is going to change.

Raising the bar. ⁽¹⁾

Dear team,
as mentioned in the kick-off, we need to get into the right attitude for the upcoming refactoring effort. Many items on our code clean-up list are ongoing changes, e.g.

• use proper names for fields and methods
• clean up magic numbers
• split large classes
• add JavaDoc to core classes
• fix compiler warnings
• remove duplicated code
• remove dead code
• add JUnit tests

All of these changes are in fact rules how to produce code that is easy to read and maintain. All these rules are part of a coding style sometimes called "clean code".

The Pragmatic Programmers once wrote that you should read at least four technical books a year to stay sharp and relevant in our fast paced industry. Remember that the half-life of our technical knowledge in only 18 months. (Heinz Kabutz) So as first technical book to read in 2012 I highly recommend Clean Code by Bob Martin. It's a great book and has raised a new wave of code-consciousness. Read one of its reviews if you do not believe me.

Sooner or later you will have to read it, so why not start now? Go ahead, buy it, read it! Or at least browse it and see what is inside. We cannot afford to have any more cryptic variable names or huge classes.

Regards,
Code Cop

Lowering the bar.

Of course I did not attach the PDF of the book. That would have been illegal but it might lower the bar to get my colleagues into reading it. I know that not many of the team will read it. I would consider my email successful if one of two read the table of contents or browse a chapter.

⁽¹⁾ "Raising the bar" is the subtitle of the Software Craftsmanship Manifesto.

Literature for Java Developers

Jonny Andersson asked for a list of good books that one needs to read to learn how to create proper and useful Java code. He knew a few reasonable books, like the SCJP (Sun Certified Programmer) Study Guide, and I immediately came up with some recommendations. Still he encouraged me to share more literature tips in a discussion thread because as he said, there are so many books on Java development and software engineering available that it is impossible to know them all. We cannot afford to waste time on other books than the best ones and have to relay on recommendations. My last reading list is a bit outdated so I decided to repost my recommendations here.

First I would like to differentiate the topic of Java into three phases: First learn how to program, second learn how to program in an object oriented way and only then learn whatever framework you like. Unfortunately I see that most junior developers have serious deficiencies in the first two areas, whereas frameworks are well known. I met developers who knew the Spring Framework pretty well, but had never heard of Test Driven Development and had no idea what the Java keywords volatile or transient are supposed to do.

For the first phase I refer to the most influential book every programmer should read. There you see that the most important book (as seen by the StackOverflow community) is Code Complete. The second important book is The Pragmatic Programmer. I think it's even more important than Code Complete. This book is the base of all development work. I'm sure you will like it. It has around 350 pages and for an experienced developer only a few chapters will be new. But on the other hand, I still meet people that do not use version control or build automation. That's one reason that everybody has to read it. Now go, read it! Even if you are an expert, just go and read it! ;-)

The SCJP Study Guide is a good start for Java. There is no way around knowing operator precedence and other low level details of the language. Another book on Java basics like TIJ (Thinking in Java) is necessary to deepen the knowledge after the SCJP guide, which is just focusing on the very basics. Then I highly recommend Java Generics and Collections to comprehend Java Generics. Of course no list of Java books is complete without Effective Java which is also very important. To round up the basic Java knowledge I recommend the Java Language Specification. It's a specification but it's not that bad to read.

After the basics I would definitely add Kent's Beck Test Driven Development by Example, which is a short and excellent introduction to JUnit and TDD. And of course you must read Design Patterns, Refactoring, Clean Code - argh, more and more books get piling up. Think about it, you will not become a professional (Java) developer by reading three books, you probably need more than that, maybe 30 ;-) So if you feel like more books, just check out my Goodreads reading list. All books I've read are rated and tagged accordingly. I encourage to have a look. I'm looking forward to see your recommendations.

Why Singletons Are Evil

Currently I'm working on enabling automated unit testing in a legacy code base but I am seriously hindered by the singleton design pattern which is very common in the code base in the form of managers. A quick sweep with a static code analysis tool reveals that 1207 out of 4787 classes depend on such (singleton) managers. See the nice graph generated with GSD to get an idea of these dependencies:

Singletons Are Evil
This means that 25% of all classes in the code base are virtually impossible, or at least very hard to unit test. Testing gets difficult as the singletons' global state has to be initialised for each test. This slows down test execution and makes the tests more brittle. That's one of many reasons I just hate the singleton pattern. It's an object oriented anti-pattern. In case you do not believe me, I am sure you will believe well known people like Miško Hevery, Uncle Bob or J.B. Rainsberger to name just a few. At least read Steve Yegge's take on the singleton which is my favourite article on the topic.

Pure Evilness
Some years ago I worked on a code base which was similar both in size and the number of singletons involved. In fact it was not that bad, only 15% of all classes were depending on a singleton, still it was nasty to work with.So I took the liberty to educate my team mates on the singleton's evilness. The following list was taken from my Design Patterns reading group notes and contains all negative effects of singletons that I'm aware of. Singletons are evil because they ...

... introduce global state/global variables.

• ... hurt modularity and readability.
• ... make concurrent programming hard.
• ... encourage you to forget everything about object oriented design.

... break encapsulation/increase coupling.

• ... are a throwback to non object oriented programming.
• ... allow anyone to access them at any time. (They ignore scope.)
• Finding the right balance of exposure and protection for an object is critical for maintaining flexibility.
• They typically show up like star networks in a coupling diagram.
• ... make assumptions about the applications that will use them.
• If nobody is going to use them they are basically just memory leaks.

... cause coding/design problems.

• Signatures of methods (inputs) don't show their dependencies, because the method could pull a singleton out of thin air.
• All singletons have to stick in boilerplate code.
• Everyone who uses it has to stick in boilerplate code, too.

... make code hard to test.

• When classes are coupled, it is possible only to test a group of classes together, making bugs more difficult to isolate.
• ... can prevent a developer from testing a class at all.
• Two tests that actually depend on each other by modifying a shared resource (the singleton) can produce blinking tests.

... prevent you from using other code in place of production implementations.

• Mocking would make unit tests run more quickly.
• It is simpler to simulate behaviour than to recreate behaviour.

So always remember that Singletons are Evil!