20 April 2015

Maven Integration Tests in Extra Source Folder

On one of my current projects we want to separate the fast unit tests from the slow running integration and acceptance tests. Using Maven this is not possible out of the box because Maven only supports two source folders, main and test. How can we add another source and resource folder, e.g. it (for integration test)? Let's assume a project layout like that:
project
|-- pom.xml
`-- src
    |-- main
        `-- java
    |-- test\java
        `-- java
            `-- UnitTest.java
    `-- it\java
        `-- java
            `-- IntegrationIT.java
We use the regular Maven Surefire to execute our unit tests, i.e. all the tests in the src/test/java folder. The plugin definition in the pom.xml is as expected.
        <plugin>
            <!-- run the regular tests -->
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.18</version>
        </plugin>
And we use Maven Failsafe to execute the integration tests. If you do not know Failsafe, it is much like the Surefire plugin, but with different defaults and usually runs during the integration test phase.
        <plugin>
            <!-- run the integration tests -->
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.18.1</version>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
By default the Failsafe plugin does not look for class names ending with Test but ending with IT, making it possible to mix both kind of tests in the same src/test/java folder, but we do not want that. The "trick" to have another source folder for integration tests is to use the Build Helper Maven Plugin and add it as test source.
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>build-helper-maven-plugin</artifactId>
            <version>1.9.1</version>
            <executions>
                <execution>
                    <id>add-integration-test-source-as-test-sources</id>
                    <phase>generate-test-sources</phase>
                    <goals>
                        <goal>add-test-source</goal>
                    </goals>
                    <configuration>
                        <sources>
                            <source>src/it/java</source>
                        </sources>
                    </configuration>
                </execution>
            </executions>
        </plugin>
Now src/it/java is added as test source as well, as seen during Maven execution. After the compile phase Maven logs
[INFO] [build-helper:add-test-source {execution: add-integration-test-source-as-test-sources}]
[INFO] Test Source directory: .\src\it\java added.
There is still only one test source for Maven but at least we have two folders in the file system. All test sources get merged and during the test compile phase we see
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Compiling 2 source files to .\target\test-classes
showing that all classes in both src/test/java and src/it/java are compiled at the same time. This is important to know because class names must not clash and tests still need different naming conventions like *Test and *IT. Now mvn test will only execute fast unit tests and can be rerun many times during development.
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: .\target\surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running UnitTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in UnitTest

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
Only if we want, using mvn verify, the integration tests are executed.
[INFO] [failsafe:integration-test {execution: default}]
[INFO] Failsafe report directory: .\target\failsafe-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running IntegrationIT
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in IntegrationIT

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] [failsafe:verify {execution: default}]
[INFO] Failsafe report directory: .\target\failsafe-reports
Now we can run the slow tests before we check in the code or after major changes a few times during the day.

30 November 2014

TDDaiymi vs. Naked Primitives

For our recent session Brutal Coding Constraints at the Agile Testing Days 2014, Martin Klose and I experimented with combinations of constraints for code katas and coding dojos. A constraint, also known as an activity, is a challenge during a kata, coding dojo or code retreat designed to help participants think about writing code differently than they would otherwise. Every activity has a specific learning goal in mind. Most coding constraints are an exaggeration of a fundamental rule of clean code or object oriented design. We tried to find particularly difficult combinations that would challenge expert developers and masters of object orientation. A brutal mixture of constraints is TDD as if you Meant it, No IFs and No naked primitives.

Goya's Naked Maja - pug styleTDD as if you Meant it
Several years ago Keith Braithwaite noticed that many developers use some kind of pseudo-TDD by thinking of a solution and creating classes and functions that they just know they need to implement it. Then their tests keep failing and much time is spent debugging. But instead they should just follow Kent Beck's original procedure of adding a little test, seeing it fail and making it pass by some small change. To help developers practise these original principles Keith defined TDDaiymi as an exercise to really do TDD. The exercise uses a very strict interpretation of the practice of TDD, avoiding design up front and helping people getting used to emergent design by refactoring. It is a difficult exercise and I still remember the first time I did it in a session run by Keith himself. For almost two hours my pair and I were struggling to create something useful. While TDDaiymi is a bit extreme, some people believe it to be the next step after TDD. German "Think Tank" Ralf Westphal even claims that following TDD as if you Meant it is the only way that TDD can give you good design results.

No naked primitives
No naked primitives is another nice constraint. All primitive values, e.g. booleans, numbers or strings need to be wrapped and must not be visible at object boundaries. Arrays, all kinds of containers like lists or hash-tables and even Object (the root class of the language's class hierarchy) are considered primitive as well. Similar to Keith's TDDaiymi this rule is designed to exercise our object orientation skills. A string representing a name is an under-engineered design because many strings are no valid names. In an object oriented system we would like to represent the concept of a name with a Name class. Usually Value Objects are used for this purpose. Also a list of shopping items is not a shopping basket. A general purposes list implementation offers operations that do not make sense for a shopping basket. So containers need to be encapsulated. While it is exaggerated to wrap all primitives (see Primitive obsession obsession), I have seen too many cases of Primitive Obsessions that I rather see a few additional value objects than another map holding maps holding strings. (To avoid that I created a custom PMD rule PrimitiveObsession that flags all primitives in public signatures of Java classes.)

Strange ConditionsNo IFs
No IFs is a great constraint of the Missing Feature catalogue. All conditionals like if, while, switch and the ternary operator (e.g. _ ? _ : _) are forbidden. Like No naked primitives this is exaggerated but many code bases are full of abysmal-deeply nesting, like the PHP Streetfighter. This has been recognised as a problem and there are campaigns to reduce the usage of ifs. After all, excessive usage of if and switch statements is a code smell and can be replaced by polymorphism and other structures. In order to practice the constraint is taken to the extreme. I have seen many developers struggle with the missing concept in the beginning, but after some thought each and everyone of them found a way to express his or her concepts in another, less nested way.

Combining Constraints
Some constraints like TDDaiymi are already difficult on their own, but obeying several of them at the same time is hard. We stick to our (coding-) habits even if we planned to do otherwise. While working the experiments Martin an I would notice a constraint violation only after several times switching roles, several minutes after introducing the "forbidden" concept. This is especially true for conditionals because they are most fundamental to our (imperative styled) coding. It seems that tool support for constraints would help, but not all constraints can be enforced automatically and definitely not for all programming languages.

Refactoring Towards Constraints
It is difficult to find solutions with constraints - after all that is one of their primary purposes - so we decided to first make it green and then refactor towards the constraint. This seems natural as TDD itself has a similar rule: After creating the next failing test (red phase), try to get it green as fast as possible, creating dirty code and duplication as needed (green phase). Then clean up the mess relying on the safety net of the already existing tests (refactor phase). This results in more refactoring than usual, and we spent more than half of the time refactoring. As TDDaiymi puts much more focus on the refactoring step, this might have been due to the first constraint alone. 50-50 for a code kata with a refactoring focused constraints is not much, given that some people report three refactoring commits per one feature commit in their production code.

Escapologist, Covent Garden, LondonDesigning Up Front?
When refactoring towards the constraint, it is often hard to change the current ("green") solution because it is rooted on primitive data types or conditionals or whatever violates the constraints. Sometimes we had no clue how to refactor and left the violation in the code for several red-green-refactor cycles until it became clear how to proceed. Being hard is not a problem because this is an exercise after all but allowing violations for extended periods of time made me unhappy as the code was not completely clean regarding our standards after each cycle. Also the code looked weird but some constraints are weird and therefore force odd solutions. In following TDDaiymi we did not think about any design up front, but maybe we should have done so. Thinking about structures (i.e. design) that support the constraint, e.g. the absence of conditionals, greatly helps the code and such "designed" solutions look much better. While we should think about the problem, the technology-agnostic solution and its acceptance criteria (test cases) before starting to write code, classic TDD discourages creating a design up front, and TDDaiymi does even more so. I guess more practise is needed to resolve this issue.

Evolving Structures
Another, more direct contradiction is the combination of TDDaiymi and No naked primitives. While following the former constraint, we want to avoid premature design decisions and sometimes use Object as a place-holder for a concept as long as there is no structure yet. Maybe this can be avoided but I do not know how. (If you know how to TDDaiymi without plain objects, I am more than happy to pair with you on some exercises.) In the end, when the design is finished, the Object has been replaced, but as the final structure evolves late, it may stick around for some time. On the other hand Object is a primitive and therefore forbidden. Maybe it is special and should not count as primitive. I have never seen any under-engineered code base using plain objects, on the contrary they usually contain lots of strings with the occasional boolean or number, but no naked Object. So I believe it is safe to allow Object as primitive because it will be replaced eventually.

Conclusion
When I started to write this article I thought its conclusion would be that some coding constraints are just not compatible. As constraints are exaggerated principles, this would be no surprise. But now, after writing thinking about it, I am not sure it is a problem. On the contrary the discussion helped me understand the constraints, their origins and intent. My real conclusion is that doing katas with constraints sparks interesting discussions and is a great way of learning about one's own coding habits and many aspects of coding in general. I strongly encourage you to do so,

Credits
Thanks to Martin Klose for the countless hours of pairing and discussing these constraints with me.

28 September 2014

Advanced Personal Branding

This February I wrote three articles on personal branding for software developers. I discussed creating and strengthening your brand step by step: branding all your accounts and defining your motto, sharing and promoting yourself and maintaining a technical blog. I sorted these activities by difficulty into a kind of personal branding ladder, which will vary for different people depending on their personality. This is the final article covering advanced, that are more difficult and more time consuming branding activities.

Go Out!
Edge Conference at Google New YorkPersonal branding is - well - personal. So you need to meet people in person, interact with them. Find your local user groups or technology related meetups and attend regularly. Join the discussion and talk about the topics you are interested in. You do not need to present anything formally. Regular listeners who ask questions now and then are vital for the existence of any community. There is no way you can fail here. As long as you are authentic, people who share your enthusiasm will want to meet you and discuss your topics. You are interesting to like-minded developers, you just need to allow them to find you.

Present at a User Group
After attending the user group meetings regularly, it is time to take the next step and present something yourself. It it true that some people would rather face death than talk in front of a crowd, but the usual audience at community meetings is forgiving. Remember, most people in the audience are like you and already know you in person. They gave similar beginner talks or know that they should. First time speakers cannot be expected to give flawless talks, and that is the beauty of user groups, full of natural human beings, delivering refreshing and idiosyncratic presentations. Some communities are so successful in encouraging their members to talk, that they continuously breed world class speakers.

Nevertheless I am not saying that your talk does not need rigorous preparation and practice up front. There are several basic things that you can screw up in presentations in general, like giving a wall of text or death by power point. Do your research and read some articles on preparing content, creating slides, presentation techniques and such. There is also much content available on technical presentations in particular. In fact there is only one rule you need to keep in mind: Your presentation is not about you. It is not about you becoming a rock star speaker, it is about serving the audience. For example if you want the audience to read your code samples, make them easy to understand and write them in a large enough font. If the font is too small you are actually conveying the message that you do not care if people read it.

I gave my first presentation at the local Java user group five years ago. It worked out well and today, many presentations later, I still like to talk to smaller groups because these presentations often become conversations and large crowds make me nervous. If you have an extrovert character and like talking to people, giving regular presentations could be less cumbersome than maintaining a technical blog. Here you might change the order of steps. Anyway you need to do both!

Organise a User Group
Organising a local community is hard work. Meetings have to be scheduled, speakers contacted and so on. Usually the group leaders' work is not seen but vital for a thriving community. Step up and help the organisers, your help will be appreciated. Or maybe there is no local community for your favourite topic, then it is high time to create one. Creating new communities is easy using tools like Meetup or social media platforms. For example Aaron Cruz is a "community factory". Be created a new meetup for a topic that clearly has been missing, Clojure Vienna and organised a few meetups, which were a great success. Then he transferred the ownership of the group to the most enthusiastic member and went on to create another meetup.

Your group does not have to be local, there are good examples of virtual communities as well. For example Jonny Andersson runs a remote reading group, a small group of people sharing the interest to learn from books. Another, quite different example is the vJUG, the on-line Java User Group, which brings well known speakers on-line every month.

As I said above, being a community leader can be a lot of work. For example Peter Brachwitz of Clojure Vienna told me that he prepares a presentation if nobody else volunteers to do so. Now that is great leader spirit! Despite the effort, running an user group is a highly rewarding activity. You will be able to watch great presentations (if you organise them ;-) and have "exclusive" access to speakers and other community leaders. For example in the Java world there is an International JUG Leader's conference once a year.

Cydcor ConferencePresent at a Conference
Presenting at user groups is often informal, sometimes becoming a discussion rather than a polished presentation. The larger the audience gets, the more formal and professional your presentation needs to be. When submitting a talk to a large and well known conferences like Devoxx, you are competing with many other speakers to get accepted. Also your future talk needs much more practice. When facing 80 or 100 people for the first time, who are all looking at you in eager anticipation, your brain is likely to shut down, unless you are naturally gifted. At least mine did, and I did not even talk to really large crowds till now.

So your presentation needs more preparations, several dry runs, maybe even showing it to a colleague for feedback. This is much work, which keeps me from doing it too often, if at all. And I do not believe in talks with little or no preparation. Even if you do not mind making a fool of yourself, you are doing your audience a disservice. You are wasting their precious (and limited) learning time, when instead they could listen to great talks in parallel tracks.

At international events you might meet new people and grow your network beyond your local area. While this is already true for all attendees, the "magic" speaker badge lets you stand out. Other speakers will talk to you and regular attendees will stalk you to ask questions ;-) And speaking at international conferences can make you famous, really world-famous if you work hard. Working hard means attending at least one conference each month, all around the globe, besides your regular work. This is really tough, as veteran speakers like Dan North or Thomas Sundberg have assured me.

Organise a Conference
Did you spot the pattern? Find some event, attend, contribute and finally organise one yourself. What is true for local events is even more true for conferences. Again you can start small with local one-day conferences embedded in larger communities like Eclipse DemoCamp, Google DevFest or Code Retreat. Your event is likely relying on the infrastructure and help of a well running user group, because you cannot do everything by yourself. For example when we started Eclipse DemoCamp in Vienna five years ago, we did so with the help of the Java User Group Austria.

A much better example is GeeCON, my favourite Java conference which I attend every year. I believe its story is the following: Some guys of the Polish JUG met and complained about the lack of a great conference in Poland. They decided to create one. Already the first version of GeeCON was a huge success and over the years the conference became one of most awesome events I have ever attended. But GeeCON is also a perfect example of the hard work needed to run such an event. If the organisers are not in a hurry, e.g. to buy more wireless routers or fix some other problem, they are walking around the venue slowly, with tired, dark circled eyes. Lukasz Stachowiak, member of the GeeCON organisers, once told me that preparations for the next version of GeeCON start on the very next day after the previous one has ended. I am sure this is also true for Devoxx and all community-driven conferences.

Against Banned BooksWrite a Technical Book
Finally we reach the top of the food chain. Writing a book is probably the most time consuming activity. Tomek Kaczanowski told me that it took him two years to get his Practical Unit Testing book delivered. But the time was well spent. His book is getting more and more popular, which is nice in case of the little revenue each book gives, but much more important is the widespread reception of his book.

As I did not write any books myself, I can only refer to articles about doing so. For example Jack Shirazi's discussion if writing a technical book is worthwhile covers income vs. non-direct income vs. time spent of writing a book. It says "People are impressed by authors. If you have had a book published in a certain area, even if that book did not do particularly well, people are impressed." Since 2007 when he wrote the article, things have changed in publishing if you decide to self publish, as Tomek did. Jurgen Appelo recommends to read at least three books about self publishing before starting with it. And if you think about writing a book you should read Rand's excellent explanation how to write a book first.

It seems that writing a book is sort of making addicted. Many authors I know did write more than one book or at least think about writing another one, even if it would make their wives unhappy (again). Tomek decided to give his third book away for free. Bad Tests, Good Tests is a short book, nevertheless it took him some time to write it. Sure, giving the book away for free removes its direct income aspect, but increases its non-direct income, as more people will get it. Anyway, free or not, it is a great book and you should read it!

Thanks to all the people I used in this blog post as examples for successful branding activities, especially as I did not ask for their permission to do so.