15 October 2017

Introducing Code Smells into Code

Sgt. Sniff-a-lotCode Smells
Code smells are hints that show you potential problems in your code. They are heuristics: Like in real life, if something smells, look at it, think about it, and change it if necessary. In the classic book Refactoring - Improving the Design of Existing Code, Martin Fowler describes 21 code smells like Long Method, Primitive Obsession, Switch Statements, Feature Envy and other anti-patterns that indicate deeper problems in your code.

The Brutal Refactoring Game
Adrian Bolboaca came up with the Brutal Refactoring Coding Game. He explained the history of the game and the game itself on his blog. I attended his workshop at the XP conference 2013 and experienced the game first hand.

In the game participants are asked to write the cleanest code possible. If the facilitator spots any code smell, participants must stop and immediately remove it. Adding functionality is forbidden until the facilitator agrees that the smell has been removed. In his workshop, Adi gave us a numbered list of smells and gave cards with the appropriate number to pairs where he saw a code smell. And he was mercilessly flagging the smallest problems in our code. ;-)

Code Smells Used in the Game
Adi chose these code and test smells for his game:
  1. Lack of tests
  2. Name not from domain
  3. Name not expressing intent
  4. Unnecessary if
  5. Unnecessary else
  6. Duplication of constant
  7. Method does more than one thing
  8. Primitive obsession
  9. Feature envy
  10. Method too long (has more than six lines)
  11. Too many parameters (more than three parameters)
  12. Test is not unitary
  13. Test setup too complex
  14. Test has an unclear Act
  15. Test has more than one assert
  16. Test has no assert
  17. Test has too many paths
Adi told me that he chose these smells because he saw them most often in his clients' code bases. His list definitely misses duplication, deeply nested conditionals and a some more. A more complete list might contain 30 items, making it more difficult and potentially frustrating for participants. (Maybe I will come up with the Moar Brutal Refactoring Game in the future...)

Observations during the Brutal Refactoring Game
This article is not about the Brutal Refactoring Game, but about code smells introduced into code. The game allows observation how and when code smells are introduced (because the whole point is to spot and remove them). As part of my refactoring training I facilitated the game more than ten times. Each time took 3 to 5 hours and had six to eight participants. The teams were average teams with several senior developers and an occasional junior developer. People worked in pairs and implemented Tic-Tac-Toe. Most teams used Java, two teams used C.

Discussion of Introduced Code Smells
Here is the code smells statistic:

Code Smells Introduced
The chart shows the number of problems I flagged during the last ten games. The different colours of the bars show the different teams. Obviously not all smells are introduced equally often. The first smells appear 10 to 15 minutes into the exercise. One team using C had difficulties with the setup and was going forward very slow - they produced little code and very few smells.

The first smell I usually see is 1 - Lack of tests. Even people following the TDD cycle happen to create "more production code than is sufficient to pass the test." This happens in the beginning and also later during the game.

Naming is hard. Not surprisingly the most common smells (number two - Name not from domain - and number three - Name not expressing intent) are naming related. Naming things after the problem domain seems twice as hard as pure technical naming. Any non trivial method could be named process or execute, but that does not help understanding the code at all.

Primitive Obsession (number eight) is the most common single code smell I have seen during the game. It is introduced early in development when method signatures are created and APIs are designed. It occurs roughly as often as the naming related smells together. Most Tic-Tac-Toe implementations are (publicly) based on numbers, pairs of numbers, arrays of numbers or the like. Primitive Obsession is very dominant in many (Java) code bases. In my code reviews I am used to method argument lists like String, String, String, String, String, int, long, long etc. Instead of using all these primitive values, they should be wrapped and should not be visible at object boundaries. (I have written more about primitives in the past.) This is an object oriented design smell.

The third most often flagged code smell are long methods (number ten). This smell is introduced later, when logic is added to existing methods. I see this smell more in the second part of the game. Even when using TDD, this smell is introduced if the refactoring phase is skipped or taken lightly. Long methods are also very common in legacy code bases and difficult to understand or change. Everyone hates these 1000 lines long methods, still I find them in every (large) code base I look at.

Code Smells Categories
To conclude this analysis let us have a look at problem categories. I aggregated Adi's 17 code smells into four groups:
  1. Problems in test code
  2. Naming related smells
  3. Missing object orientation
  4. Complexity
Code Smell Categories
It seems that unit testing is the least problem - which it definitely not true. Most teams I work with have no automated (unit) tests for their production code. Maybe there were less testing issues during the game because the teams had learned about testing smells before. I practice refactoring with my teams after we have worked through all of unit testing.

Initially I was surprised to see missing object orientation high on the list. Now, after writing about it, I think it is also related to my "coaching/ learning plan". After refactoring I go for naming and finally object orientation. (Maybe the order of topics is wrong, but unit testing is easily sold to management and refactoring is asked for by developers often, making both topics ideal to start an improvement initiative.) I do not expect less naming problems, even after a few sessions on naming, because - as discussed before - naming is hard. I would expect the object orientation of the solutions to improve.

Samir Talwar wrote about his experience with the game. As facilitator he had a different focus, e.g. he was more strict about unnecessary if, treating it more like a No If constraint. He also saw different code smells being introduced. (I recommend reading his summary.) We both agree that naming is hard and causes many problems.

Comparison of Team Performance
While the participants were industry average - maybe even above - they were in need of improving. (Who is not?) The following bar chart shows the number of code smells introduced into the code by each team. (To compare the teams I removed the one with setup problems.) Some teams ran the exercise twice. Some of them improved, some did not.

Code Smell Categories by Team
On average, each team introduced 17 issues into their code base, right from the beginning of a small project, during a few hours of work. I am sure they tried hard because I was watching them, still this result is very disappointing. I am scared of the massive amount of code smells lurking in real world projects.

There is a noticeable difference between individual teams. Some teams created only half as many smells as other ones. Better teams introduced less code smells creating less technical debt.

Adi claims that you can have legacy code after 15 minutes. It is true. In a short time, the teams introduced many code smells into their code. The most common smells were bad names and Primitive Obsession. Different smells were introduced during different development activities. Some teams introduced less smells than others.

We need to focus on code smells. Noticing smells in our code is an important skill which can be trained. A good place to start practising are refactoring code katas like Emily Bache's Tennis Game and Yatzy. (Both exercises are available in many programming languages.) "Listening to code smells" improves our design. Finally I want to encourage you to watch out for primitive values on object boundaries as Primitive Obsession seems to be the most common problem in object oriented code.

Final disclaimer: The game is no scientific experiment throughout our industry. Only a few teams participated and the results are biased. Nevertheless I wanted to share the results.

23 September 2017

Verbs instead of Nouns

Verbs instead of Nouns is a basic Coderetreat activity. It was used right from the beginning of Coderetreat. I tried it the first time during the GDCR 2012. The goal was to focus on verbs instead of nouns (obviously ;-). By searching for verb names, we did not think about what a class represented or contained, rather what it did.

Constraints in General
A constraint, also known as an activity, is an artificial challenge during an exercise, e.g. code kata, coding dojo or Coderetreat. It is designed to help participants think about writing code differently than they would otherwise. Every activity has a specific learning goal in mind.

Constraints are the primary tool to focus a coding exercise. For example, to improve my Object Orientation, I will practise Jeff Bay's Object Calisthenics or even Brutal Coding Constraints. Some constraints are an exaggeration of a fundamental rule of clean code or object oriented design and might be applicable during day to day work. More extreme ones will still help you understand the underlying concepts.

Learning Goal
Verbs instead of Nouns is listed as stretch activity. Stretch activities are designed to push you out of your usual coding habits - your coding comfort zone - and broaden your horizon by showing you new ways how to do things. By design stretch activities might look awkward, ridiculous or even plain wrong.

The learning goal of Verbs instead of Nouns is to push you out of noun oriented thinking. Noun oriented thinking is a way of object orientation, where the nouns of the problem description become classes, and the verbs become methods. This is the classic definition of Object Oriented Analysis and Design. As with any technique, following it blindly is not healthy. According to Alan Kay Object Oriented Programming is about messaging and encapsulation. He wanted "to get rid of data". His objects are defined by the messages they accept. Object orientated programming becomes verb based, if we focus on behaviour.

In functional programming, verbs are natural. All activities are functions. For example Steve Yegge describes functional programming as verb based in his humorous critique of 2006's style Java. Verbs instead of Nouns is an object oriented constraint.

VerbInterpretation of the Constraint
Besides its name there is no information about this constraint available on the Coderetreat site. There was a discussion how to meet the constraint (which has been deleted to make space for the new GDCR organisation): Separate value objects from operations and build service objects for the operations, which would be named with a verb. Or do not consider what a class contains or represents, but what it does. This keeps the concerns separated and the classes small and simple.

Being a Value
Obviously not everything can be named with a verb. Values, at least primitive values, are things: 2, true, "Hello". The Oxford Dictionary explains value - the way we use it in code - as the numerical amount denoted by an algebraic term; a magnitude, quantity, or number. Now "Hello" is neither a quantity nor number, it is a constant term. The entry about Value Object on Wikipedia defines a value object as a small object that represents a simple entity whose equality is not based on identity: i.e. two value objects are equal when they have the same value, not necessarily being the same object.. The definition uses "having the same value"... I am not getting anywhere.

On the other hand, in the Lambda Calculus, even numbers are represented as functions. For example the number two can be represented by the higher order function n2(f,x) = f(f(x)), see Tom Stuart's Programming with Nothing. Being a function makes it verb based but which which verb would name n2(f,x)?

Suitable Exercises
For a stretch exercise, a suitable exercise is challenging. There is no point if everything goes smooth. We need an assignment that does not support the constraint. Everything that is functional in nature is not suitable, because functions are verb based. This rules out algorithmic exercises as algorithms are usually functional. We need a kata with some state - some "values" - and the need to mutate that. Let's try different problems.

Discussion of Game of Life
As I said, I did Verbs instead of Nouns first on the Game of Life. While Game of Life is a larger exercise, most of it can be implemented in a functional way, lending itself to the constraint. Here are some of its classes:

Classify has two implementations, ClassifyPopulation and ClassifyReproduction. Both classes check if a population is optimal for survival or not. There is one public method and its arguments are passed into the constructor. These classes are functors, function objects, the representation of functions in object oriented languages. The class name suits these class and the verb oriented thinking helped in extracting and evolving them.

LocateCell represents the position of the cells in the grid. It contains two integers x, y and an equals method to identify same positions. What is the verb of being a coordinate? A coordinate locates, as it discovers the exact place or position of something (Oxford Dictionary). Here Coordinate might be a more natural name.

I am unhappy with LookupLivingCells. It has two methods reproduce and isAlive. The verb Lookup only points to the second method. A proper class name should contain all functionality the class offers, so LookupAndTrackLivingCells is more appropriate. I do not like class names with And in them because they violate the Single Responsibility Principle. On the other hand - in an object oriented way - the class is fine as it encapsulates the collection of LocateCells and represents a Generation of cells.

Discussion of Trivia
Next I tried refactoring towards the constraint. Refactoring towards a constraint allows a more fine grained transition. Together with fellow craftsman Johan Martinsson we worked on the Trivia exercise and spent several hours extracting "verbs" from the legacy code base. (Many of the observations I describe later were made by Johan or found through discussion with him.) Let's look at some of the classes we created:

We extracted Ask. An noun oriented name might be Questions or QuestionsDeck. It is a closure over the list of questions and it does ask them.

MovePlayerOnBoard contains the board of the Trivia game. We felt being unable to escape our mental model of objects as state. On the other hand, the code for the class was chosen only by looking at the behaviour. It must be good. MovePlayerOnBoard has one public method but is not a functor because it contains mutable state, the positions of the players on the board.

Score is a similar reasonable class by object oriented standard. A player scores by answering correctly, or does not score by answering wrongly. Like MovePlayerOnBoard and AllowToPlay, it is a real object with internal, encapsulated state and various methods manipulating its state. These classes are far away from functors and functional programming.

Verbs are abstractions, too. There are "small verbs" like increasePurse, and higher level ones like moveAndAsk. Smaller verbs are easier to identify and to create or extract. Most of our verbs encapsulate primitives. If the code is primarily state, finding a suitable verb is hard. These verb names feel even more "wrong" than other verb oriented names. Maybe, when we only behaviour of a class is mutating the subject, we should show the subject in its name.

A method that does much is difficult to name with a single verb. In the refactoring exercise, we moved out logic to make the describing verb(s) simpler, clearer and "pure". During refactoring we had trouble finding concise verbs for convoluted legacy methods. I guess when creating verb based code from scratch, such methods would never exist. Naming classes as verbs helps to split logic into more classes containing different aspects of data.

Many verb oriented classes are functors, objects with a single method. Some are closing over state. There are classes with different aspects of the same verb, e.g. answerCorrectly and answerWrongly in a class Answer. Despite some weird names, the resulting design was always good. The constraint drives to nice, small, focused objects.

Usefulness as Exercise
The constraint is difficult. Especially when dealing with state, it is hard to find verb oriented names. It forces small, focused objects and discourages state oriented designs like Java Beans. Intermediate Object Oriented programmers will gain most of the constraint. They understand the basics of objects and usually create noun based classes. With more knowledge of object oriented design principles like SOLID, the constraint might have has less impact on the design.

Example Code

20 August 2017

Same procedure as every year

One advantage of freelance work is that I can take as many days off as I like. Since 2013 I am not visiting clients during summer, instead I am working on a different project. This year, my wife wants us to reach production, I am pushing hard, working 14 hours some days.

My Trustworthy Hammer Drill