Showing posts with label expert level. Show all posts
Showing posts with label expert level. Show all posts

4 July 2024

Encapsulation vs Business Rules

Business Men (licensed CC BY-NC-ND by Andreina Schoeberlein)No Naked Primitives is a Coderetreat constraint which trains our object orientation skills. No primitive values, e.g. booleans, numbers or strings, must be visible at object boundaries, i.e. public methods. Arrays and other containers like lists or hash-tables are primitives, too. I love this constraint, as it pushes people right out of their comfort zone. ;-) (I wrote about No Naked Primitives in combination with other constraints and included it in the expert level Brutal Coding Constraints.)

Value Objects
The usual designs to avoid naked primitives are Value Objects and First Class Collections. Value Objects, by design, expose the values they wrap with a getter because some other objects will want to use these values. What happens if I go extreme and do not allow any primitives at object boundaries? (Of course this is crazy, a clear case of Primitive Obsession Obsession. Still when Coderetreat facilitators get together to practice, things end up like that.) Let us take the Game of Life as an example. (If you do not know the Game of Life, read the description and implement it right away.) In the game, for evolving a generation, I need to count the living neighbours of each cell. The number of living neighbours is an integer and its value object in C# could look like
public class NeighbourCount {
    private int count;
    
    public NeighbourCount(int count) {
        this.count = count;
    }

    // ... code to manage the count

}
Now any code which depends on the data (i.e. the count) will have to be moved into the value object to be able to access the data. Following the rules of the game, if there are two or three living neighbours, a living cell lives on. The method Apply​Rules​OnLiving​Cell implements this rule.
public class NeighbourCount {

    // ...

    public GridSpace ApplyRulesOnLivingCell() {
        if (this.count == 2 || this.count == 3) {
            return new AliveCell();   
        }
        return new EmptySpace();
    }
}

public interface GridSpace {}
public class EmptySpace : GridSpace {}
public class AliveCell : GridSpace {}
Grouping the data (count) and the logic which is based on the data, uses or modifies it (Apply​Rules​OnLiving​Cell) together is a core principle of object orientation. Further all data is strongly encapsulated.

Polymorphism
The next method Apply​Rules​OnEmpty​Space is similar. The decision which of the two methods to call depends on the state of the cell, which is either alive or dead/non existing. This boolean state has to be encapsulated inside a class, e.g. class GridSpace. This class behaves differently for the values of the boolean state, which makes the boolean a simple type code. The object oriented way to work with type codes is to use polymorphism:
public interface GridSpace {
    public GridSpace ApplyRulesWith(NeighbourCount count);
}
public class AliveCell : GridSpace {
    public GridSpace ApplyRulesWith(NeighbourCount count) {
        return count.ApplyRulesOnLivingCell();
    }
}
public class EmptySpace : GridSpace {
    public GridSpace ApplyRulesWith(NeighbourCount count) {
        return count.ApplyRulesOnEmptySpace();
    }
}
The code looks weird and it is not my usual implementation of the game's rules. It has an issue: The rules of the game are distributed among three classes. This is Shotgun Surgery - when a single change is made to multiple classes simultaneously: If I need to change the rules, or even want to read and understand the logic of cell evolution, I have to go to three places.

Shot (licensed CC BY-NC-ND by Bart Maguire)Business Rules
On the other hand, a basic implementation of the rules using primitives (e.g. in Ruby because polyglot programming is cool),
def alive_in_next_generation(alive, living_neighbours)
  (alive and living_neighbours == 2) or 
  living_neighbours == 3
end
is one line of code and easy to understand. The game's rules - the business rules - are boolean expressions describing certain situations, which "the business" needs to act on. Typical examples of such situations are when an item is out of stock or a client qualifies for a discount. Business related conditions are called policies. (And there are predicates, which are boolean expressions, too. These have their origins in formal logic.) Boolean expression are functional in nature. So a functional design, i.e. functions operating on primitive data, could be more appropriate. Even in object oriented design there are use cases for objects containing only logic and no (mutable) data.

Conclusion
What is the point of my discussion? In the case of Game of Life, there is a tension between keeping data and its logic together versus keeping related logic together. This is particularly true for boolean expressions and code depending on them, as boolean values usually end up in conditionals. I like to keep "decisions" and the logic depending on them close together but I want to keep my business rules in one place even more. I am wondering if this is true for most design situations, besides Game of Life. Boolean logic is interesting because if allows variation in the automation. Code without any booleans is still useful, e.g. pure calculations or uniform transformations in a pipeline style of operations.

Taking it further?
While boolean is a primitive, it is different from other primitives. What happens if I do not allow any primitives besides boolean at object boundaries? The data of class NeighbourCount would still be encapsulated when I add relevant queries (in Python because I love programming languages):
class NeighbourCount:

  def __init__(self, count):
    self._count = count

  # ... code to manage the count

  def isTwoOrThree(self):
    return self._count == 2 or self._count == 3

  def isThree(self):
    return self._count == 3
Using these small methods, I get a concise implementation of the rules,
class Rules:
  def cellInNextGeneration(self, cell, count):
    if (cell.isAlive() and count.isTwoOrThree()) or count.isThree():
      return AliveCell()
    return EmptySpace()
Is this better? I am not sure. At least the (business) rules of the Game of Life are in one place now. They could be replaced with different rules if needed, making the design extensible. At the same time different rules would most likely require different queries in NeighbourCount. For example in Hex Life, I need a weighted sum of first and second tier living neighbours to decide the state of the next generation. This is not possible without adding new queries to NeighbourCount. The Open Closed Principle is not satisfied. (Then maybe Hex Life is too much of a change for any design to "survive".) My rules logic keeps calling into the encapsulated value object repeatedly, which looks much like Feature Envy. I feel like I am going in circles here ;-)

10 March 2024

Programming with Nothing

I like extreme coding constraints. A constraint, also known as an activity, is a challenge during a kata, coding dojo or code retreat designed to help participants think about writing code differently than they would otherwise. Every constraint has a specific learning goal in mind, for example Verbs instead of Nouns. After playing with basic constraints for a long time now, I need more challenging tasks. Combining existing constraints makes things harder: For example Object Callisthenics or my very own Brutal Coding Constraints are way harder than their parts applied individually.

Void (licensed CC BY-NC-ND by Jyotsna Sonawane)Missing Feature Group of Constraints
There is a another group of extreme constraints which I call the Programming With Nothing constraints. They are a subgroup of the Only Use <placeholder> constraints. All of these belong to the Missing Feature group. The well known No If and No Naked Primitives constraints are good examples of Missing Features because we take away a single element that we are so very much used to. Only Use <placeholder> constraints force you to use new constructs instead of something else. For example, Alexandru Bolboaca, the pioneer of Coderetreat in Europe, once mentioned the following constraints to me: Only Bit Operations replaces all arithmetic operations, like plus or multiply, with bit operations and Only Regular Expressions asks you to use Regular Expressions as much as possible. You can get pretty far with Regular Expressions in exercises like Balanced Brackets, Coin Change, Snake or Word Wrap. (Look for the Bonus Round at the bottom of the Word Wrap page.)

Programming With Nothing
But let us get back to Programming With Nothing. The first one of this group, which I came across ten years ago, was presented by Tom Stuart in his 2011 Ru3y Manor talk Programming With Nothing. Tom is taking functional programming to the extreme, only allowing the declaration of lambda expressions and calling them. The exact rules he is following are:
  • Create functions with one argument.
  • Call functions and return a result.
  • Assign functions to names (abbreviate them as constants).
Basically he is using the Lambda Calculus and this constraint is also referred to as Lambda Calculus. His talk is using Ruby, using only Proc.new, no booleans, numbers or strings, no assignments, control flow constructs or standard library. Clearly he is programming with nothing. (Here is the recording of the talk, his slides and the code.) Over the years I have seen similar presentations, even using Java.

The Fizz Buzz Kata
The goal is to implement the Fizz Buzz kata. While Fizz Buzz is very simple, it needs looping integer numbers up to 100, conditionals on integer comparison, integer division and strings. It is very small but not simple. Some people even use it during job interviews - which is controversial. The whole Fizz Buzz is:
for (i = 1; i <= 100; i++) {
  if (i % 3*5 == 0) 
    print("FizzBuzz");
  else if (i % 3 == 0) 
    print("Fizz");
  else if (i % 5 == 0) 
    print("Buzz");
  else 
    print(i);
}
And this is quite a lot if all you have is a lambda. I maintain a starting point for TypeScript, to be used in my workshops. This kind of exercise is fun, at least for me ;-). If you follow the assignment, i.e. work on numbers, then booleans, then pairs etc., you can use Git branches to jump to the next milestone - or take a sneak peak how it could be done.

Nothing Happened (licensed CC BY-SA by Henry Burrows)Extreme Object-Orientation
In 2015 I watched John Cinnamond's Extreme Object-Oriented Ruby, which is like Tom Stuart's Programming with Nothing. This version only allowed you to define objects which contain other objects and call the nested object's methods or return them. In his starter repository he described how to simulate booleans, numbers and so on.

Nothing but NAND
Then I tried to write Fizz Buzz only using NAND. This is Programming With Nothing the hardware way. How so? A NAND gate is a logic gate which produces an output which is false only if all its inputs are true; thus its output is complement to that of an AND says Wikipedia. More importantly, the NAND gate is significant because any Boolean function can be implemented by using a combination of NAND gates. This property is called functional completeness.. Because of its functional completeness it should be possible to create arbitrary programs. I started out with a Bit class which had its nand() function implemented and all other code was built on top of this. Numbers, i.e. arrays of bits,
class Numbers {

  static final Byt ZERO = new Byt(OFF, OFF, OFF, OFF, OFF, OFF, OFF, OFF);

  // ...

  static final Byt FIFTEEN = new Byt(ON, ON, ON, ON, OFF, OFF, OFF, OFF);
  static final Byt HUNDRED = new Byt(OFF, OFF, ON, OFF, OFF, ON, ON, OFF);
}
and bitwise logic,
class Logic {

  static Bit eq(Bit a, Bit b) {
    return not(xor(a, b));
  }

  // ...

  static Byt and(Byt a, Byt b) {
    return not(nand(a, b));
  }

  // ...

  static Byt ifThenElse(Bit b, Byt theThen, Byt theElse) {
    Byt condition = Byt.from(b);
    return or(and(condition, theThen), 
              and(not(condition), theElse));
  }
}
were straight forward. Arithmetic was cumbersome due to possible over- and underflows.
class Arithmetic {

  static BitOverflow inc(Bit b) {
    return new BitOverflow(not(b), b);
  }

  static BitOverflow add(Bit a, Bit b) {
    return new BitOverflow(xor(a, b), and(a, b));
  }

  // ...

  static Byt inc(Byt a) {
    BitOverflow r0 = add(a.b0, Bit.ON);
    BitOverflow r1 = add(a.b1, r0.overflow);
    BitOverflow r2 = add(a.b2, r1.overflow);
    BitOverflow r3 = add(a.b3, r2.overflow);
    BitOverflow r4 = add(a.b4, r3.overflow);
    BitOverflow r5 = add(a.b5, r4.overflow);
    BitOverflow r6 = add(a.b6, r5.overflow);
    BitOverflow r7 = add(a.b7, r6.overflow);
    return new Byt(r0.b,r1.b,r2.b,r3.b,r4.b,r5.b,r6.b,r7.b);
  }
}
For loops I added a sequence of bits which worked as the Instruction Pointer. Using the IP and the existing arithmetic operations I implemented goto which I used to jump back during loops. The final code did not look much different than your regular structural code, using functions and mutable data. The exact list of things I used was:
  • Data structures for a single bit, a byte (8 bits) and a series of bytes i.e. memory.
  • Bit nand(Bit other) as the only logic provided.
  • Getting and setting the values of the data structures.
  • Defining functions with multiple statements to create and modify data and call other functions.
  • A map to associate statements with memory addresses indexed by the IP. Was this cheating?
I had played with assembly in the past, which helped me to build my program from NANDs alone. It is a great learning exercise to understand computers' logical components and CPUs. There is even an educational game based on the idea of NAND.

What is Next?
I cannot remember how I ended up there, but next I tried to write Fizz Buzz using a Touring Machine. But this is a story for another time...

25 August 2022

Practice Speed

Several years ago I co-facilitated the Global Day of Coderetreat with Houssam Fakih. It was a nice Coderetreat. During the introduction Houssam presented the concept of practice and three levels of competence. The first level is that you are able to do a certain thing from time to time. The second level is that you are able to do it all the time. The third level is, and that was new to me, that you perform the thing consistently well and fast at the same time. In this short video there are three guys throwing balls at a boot at a fair. The guy on the right is on level one. He is able to throw the ball into the basket most of the time. The guy in the middle is competent and always hits the target. But the guy on the left is on another level. Besides hitting the basket consistently, he is using both hands alternating and is two to three times faster.

The Walk to Save Great Danes (licensed CC BY-NC-ND by Warchild)Deliberate Practice
In Coding Dojos and Coderetreats we practice all kind of coding techniques like Test Driven Development, software design and pair programming, but we rarely practice speed. The Coding Dojo mindset is explicit about going slow. There are a few exercises which have a timing aspect, e.g. Elephant Carpaccio or Baby Steps. Some people try to master the Baby Steps constraint by typing faster and using more shortcuts. While these are good things to practice by themselves, they bypass the exercise. Both exercises focus on taking smaller steps. The speed of typing is not the bottleneck.

Side Note: Speed of Typing
If the speed of typing is not the bottleneck in writing software, why do we focus on shortcuts and touch typing? The idea is, while coding, to stay in the flow and work on a higher level of abstraction. For example, when I want to move a method, and I can do it with a few keystrokes, I keep the current thoughts in my mind. But if I have to navigate, modify blocks of text, change method invocations, and so on, I think in more low level concepts like text editing or file navigation, and this breaks my focus.

Hackathon
One of my recent Coding Dojos became a Hackathon. A misunderstanding with the sponsor, who cared for food during the event, caused this. The sponsor wanted to run some challenge to make people think outside of the box, and make them try something new and innovative. They had prepared an assignment in Hackathon style. A Hackathon is an event that brings together experts and creates a collaborative environment for solving a certain problem. I avoid Hackathon because the format is competitive and people try to go as fast as possible to create some prototype or try to prove some idea or deliver some working code. Going fast above all else, e.g. skipping preliminary design or tests due time pressure is the startup trap. Obviously I dislike this way of working. (I see value in Hackathon as a tool for innovation and maybe team building.)

First I was unsure how to proceed. On one hand I wanted to please the sponsor, paying real money for the crafters community is a real contribution and separates the "talking" from the "doing" companies. On the other hand I wanted to stay true to the Coding Dojo experience. I thought of Houssam and remembered that going fast is a skill, one we need and rarely practice. In fact it is a skill we developers are often asked to apply, e.g. when the deadline is coming up or marketing has some new urgent idea. Like the guy on the left in Houssam's video, working fast and clean at the same time is hard. I made it a mix: I ran a Coding Dojo where the challenge was to finish some basic code in a given time frame. All Coding Dojo rules applied, the goal of the evening was to learn, to collaborate and to be nice. Even under pressure I reminded the participants to write tests because "your best code does not help you if it is broken". I encouraged them to work in small batches as "the most clever algorithm loses if it is unfinished". As facilitator I focused on helping people to create small, working increments. It worked well and after 90 minutes most teams had a working solution and three of them won a price.

Speed (licensed CC BY by Gabriel)How to Practice Speed
Considering my own, personal practice, I go in the same direction. I am practising a lot on old and new exercises, in various programming languages, some I have much experience and some I just learned. Working these exercises, adding hard or even brutal constraints is boring. I even tried contradicting constraints like Tell Don't Ask and Immutability. To try something new I started working against the clock. Maybe I can be the guy on the left.

Roman Numerals
My colleagues tell me that I am very fast in perceiving, reading and understanding code, as well as typing with shortcuts. When working against the clock my first exercise was Roman Numerals. I chose a small exercise because I expected to retry it a lot. I set a timer to five minutes and worked on the kata. When the timer rang I stopped and analysed my progress. Of course I failed to finish but I kept trying. Some rarely used editing shortcuts would have helped, and I learned some more of them. I did use TDD and three to four tests were the most I was able to get, which was enough. The test for Roman I created the method arabicToRoman, II added a recursive call, VI introduced a lookup for the literal by its value and 388 introduced all the remaining literals. When using Ruby for the same exercise I thought I would be able to beat my Java time, because Ruby is more expressive and there is less code to write. Unfortunately I am not as familiar with Ruby as I am with Java and the missing code completion slowed me down. Currently I need five minutes for a fully working Arabic to Roman Numerals conversion including subtractive forms. I am able to create good names and small methods in the given time, but I am unable to create more elaborate structures for holding the Roman literals and their values. Java is just too clunky. In short: the code is good, but the design is bad.

What Next
Then I tried the Coin Changer kata and after several tries I was able to beat four minutes. Coin Changer and Roman Numerals are much alike and Coin Changer is simpler as it has no literals, the coin value is also the returned value. I knew both katas well, and the exercise was in fact only about my typing supported by tests to avoid mistakes. Maybe I should try a larger assignment. Running through many repetitions would be tedious. Maybe I should try unknown exercises. At least they would be unknown for the first run. I need a large number of little TDD exercises of similar size and complexity. Time to do some searching.

6 May 2020

Learning yet another Programming Language

In 2000 Andrew Hunt and David Thomas wrote their influential book The Pragmatic Programmer, which is listed as second single most influential book every programmer should read. (I listed it in my book recommendations both 2012 and 2006.) Chapter one, tip eight Invest Regularly in Your Knowledge Portfolio says: Learn at least one new language every year. Different languages solve the same problems in different ways. By learning several different approaches, you can help broaden your thinking and avoid getting stuck in a rut.

Filled Tool Box (licensed CC BY-NC by hmboo Electrician and Adventurer)I started out as Java developer. Since than I have studied XSLT, Ruby, Scala, Forth, JavaScript, Scheme, Python, TypeScript, Go, C# and C. Out of these I even ran trainings for developers teaching them Python and TypeScript. In addition I had a glimpse of Visual Basic, Dart, Clojure, R, PHP, NATURAL, PowerShell, Kotlin and have relearned Assembly. I still need to learn Smalltalk, F#, maybe Haskell, J and of course Prolog.

I like programming languages and my learning approach is driven by curiosity and fun. Today I will describe my "standard", step by step way to get into a new language. I will assume you are an experienced developer, able to code in Java or C# who understands basic programming principles. I believe this approach is not suitable for programming newbies. I never use all these steps and sometimes change their order - like starting with the last one. Feel free to reorder or skip any steps not adding knowledge or fun.

1) Get an overview of core language features.
I start reading Wikipedia about the programming language I want to dive into. I am looking for the core features, used concepts and paradigms in the language. Code examples of these features provide a first idea of the language's syntax. The idea is not to know everything, just to be able to write some code. Writing code is the fun part, not reading. The more languages you know the easier and faster this step is. When learning Go one hour on Wikipedia during commute was enough. On the other hand for TypeScript I spent several hours reading the language reference (Handbook). For C# I skipped this step as I had seen most of the language features while facilitating Mob Programming sessions. You are done with this step when you have some idea what the language can do.

2) Figure out the usual setup and get it working.
Most programming languages come with their own ecosystem of runtimes, documentation, dependency and packaging mechanisms, testing frameworks, editors and other tools. When starting with a new language I try to use its typical tooling. Or at least I look for a decent IDE plugin to keep some level of comfort and productivity. Often this is painful and full of compromise. E.g. when working with Scheme I should have used Emacs but started with a basic editor and used VS Code in the end. For Go I needed 2 to 3 hours to set up the command line tools and VS Code integration. For C it took me 2 hours to compile and run a sample test alone - due to incompatible architecture binaries, sigh. I am still staying away from make tools due to the additional complexity. Eventually I will have to work through them if I want to use high level IDEs like Eclipse or CLion. If your interest in the language is purely educational, a simple editor might be enough to get started, like I used for Forth. You are done when you are able to edit, compile, test and run a Hello World application, e.g. in Windows Assembly.

3) Port small pieces of code (from a similar language if possible).
This helped me when learning C. I knew its basic features and had figured out how to compile and run a single file. And then I ported some small (refactoring) code katas. Porting code katas was easier than coding them because the solution and its code were already there and all I needed to deal with was syntax. If there is a similar language to start from, e.g. Java to C#, Java to PHP, C++ to C, even Java to C, then some of the syntax is proper right from the start. There was a lot of Google and StackOverflow involved and I managed to convert a small kata in around one hour. In the end I had ported several code katas to C. Emily Bache keeps inventing fun and interesting refactoring katas, which always need ports to other languages. For example I have ported her Parrot-Refactoring-Kata to TypeScript, Go and recently C. I have also contributed a Scheme version, but that was after I had learnt the language. You are done when you have contributed at least two ports of small refactoring katas.

4) Work through koans of the language.
Koans are an effective way to learn new programming languages. Programming language koans are a progressive sequence of little exercises, starting with basic things and building on each other to move to more advanced topics. The goal is to learn the language and core libraries. Usually the exercises contain failing test cases, where tiny pieces of code have to be filled in to make them pass. (I have used this idea to teach unit testing in Java and PHP as well as Python and C#. Porting the koans from JUnit to xUnit also helped me to get into C# - see the paragraph on porting small pieces of code.) Koans are awesome and I use them regularly. I worked through some Python koans, two third of Kotlin koans and all C Koans. The koans vary in size. While Kotlin contained 6 lessons, Python had 39 and JavaScript even 78. Working all these 78 exercises took me more than three full work days in total. You are done when you have completed (almost) all of the koans for the new language.

Ninety-Nine Problems
In case there are no koans for your language, look for "Ninety-Nine Problems". The idea is based on Werner Hett's P-99: Ninety-Nine Prolog Problems. These are little problems with different levels of difficulty. Sample solutions are available in Java, Scala, Haskell, Kotlin, F#, OCaml and probably others.

scratches - What is the connection between this image and item 6? (licensed CC BY-NC-ND by Sue)5) Watch some recorded talks and online presentations.
This is the most obvious step. I like to watch recorded presentations, during my commute. Sometimes some extra (passive) information helps me understand a language, or even this specific weird feature. Depending on time and interest this can be a few talks, or whole months of commute.

6) TDD some code katas from scratch.
Now is the time to write some code from scratch. I recommend starting with simple exercises. There is no point in getting frustrated right at the start. Manage the difficulty of your exercises: Use simple katas like FizzBuzz, Prime Factors, Roman Numerals or Word Wrap to get started. For example I used FizzBuzz to practice some XSLT and Prime Factors when revisiting old languages I used to know many years ago - like BASIC. I reuse these katas - and know their solutions - I just want to try them in a new language. Later I move to more complicated assignments like Bowling, Minesweeper (when I revisited Assembly after 20 years), Bank OCR (when I studied Scheme) or Conway's Game of Life. You are done when you have implemented a few code katas from scratch including tests.

7) Write more code, e.g. tackle a larger code base.
There is no other way of getting deeper into a programming language than using it. This calls for a larger side project. It can be a complex code kata, a little video game like Pong or Tron (Nibbles) - including graphics of course - or whatever comes to your mind. As I am fond of Scheme, I have build several Scheme interpreters in new languages. You could even implement your own unit testing framework, which is also recommended by Kent Beck as an exercise to get into a new language. (And I did that for Pascal, Assembly and Scheme.) Depending on the time invested into the previous steps, a side project includes more or less trial and error. If there is too much hassle, I stop and go back to previous steps to learn more about the basics. There is no point in being stuck. For example when studying Go, I went for the side project after 3 hours of researching the language, which was too early. So I spent some more time on theory and then continued working on my idea. My usual learning side projects take around 15 to 20 hours - or that is the time when I lose interest. You are done when you have worked on a larger code base for at least 15 to 20 hours.

8) At last get *all* the details.
Now that I am familiar with the basics of the language and its ecosystem, it is time to dive deeper. After some month of experimenting and hands-on practice - steps 2, 3, 4, 6 and 7 all involve writing code - I am drawn back to theory. I like to balance my learning between theory, experiments and practice. To conclude learning a new language I might study a classic book about that language. It should cover the language and its features completely and I read it from cover to cover. At that time I am already familiar with many parts of the language, reading progress is fast. I am interested in the all the details, the bits and pieces I did not encounter during my experiments. Language specifications are usually dry and perfectly suited for this step as well as classic titles like the "Pickaxe book", K&R and SICP (although SICP is so much more than a Scheme book...) You are done when you read a classic book on the new language from cover to cover.

Conclusion
I like programming languages and learning new ones is adventurous and fun. I try to learn a new language every year. Not all learning goes deep. Not all languages stick. Unless you are working on real projects in all of these languages at the same time, it is natural to forget some details. And that is perfectly fine. It is all about incorporating new paradigms and widening your perspective. So keep learning!

19 August 2019

Y U NO TDD

Y U No TDDDuring this year's GeeCON the crew organised an Open Space evening. (An Open Space is a self-organising meeting where the agenda is created by the people attending.) I participated and ran a session on the question why we are not doing Test Driven Development. (Y U No Do TDD?) I am running TDD trainings from time to time and wanted to get more insight where people are stuck with TDD.

Context
As I said, an Open Space is self organising, and only people interested in TDD attended my session. This is a typical problem of communities of practice - only people interested in the topic attend - which results in us living in a bubble. For example long time TDD practitioner Thomas Sundberg and Shirish Padalkar, lead consultant at ThoughtWorks, participated in the discussion. Depending on the background of each individual participant, my original question was understood as:
  • Why are you not doing TDD on production work at all?
  • Why are you not doing TDD most of the time?
  • Why are you not doing TDD all the time?
I collected the reasons not to do TDD during the session which I want to share here. Text inside quotation marks, e.g. "hi" quotes exactly what people said. While the previous three questions are slightly different, the reasons seem to be similar. I grouped the answers. I did not want to contradict or debunk these answers and have to hold back not to do so ;-)

Prototyping
I am "experimenting with something", the "expected outcome is unclear" and "it's only a prototype". Obviously these are valid reasons as Spikes are outside of TDD. These answers usually coming up quickly makes me wonder if they are kind of excuses sometimes. Experimenting with new libraries and APIs is covered further down, so what are we experimenting with? I worked with many developers who would agree that the expected outcome of their current ticket was unclear - because they did not take the time to analyse the story and understand the solution they were supposed to build? Additionally most of our prototypes go to production after all, don't they ;-)

Time Pressure
Another reason - given by some of my clients too - is their need to go fast: "I need to go very fast", there is "no time for that" and I "believe to be faster without it". While they might be wrong in the long term I understand the effects of pressure. One person made it more explicit, while he has no strong deadline, he said "I have a huge backlog, I am stressed". Indeed when I am extremely stressed, I find it hard to maintain a structured approach, especially if a lot of task switching is involved. Besides the needed skill to apply TDD under high load, much discipline is required to endure pressure. In such situations Strong Opinions and Dogma might help.

Missing the Bigger Picture
I am just "writing a script for myself". Maybe there is no need for automated tests when writing a one time script for myself. TDD has a testing aspect - and it has many other aspects like designing software, fast feedback and working incrementally. TDD is not only about testing. Some people miss that or have only partial understanding of the benefits or do not care for these benefits at the moment. The opposite reason is "because I know how the class will look like". Yes TDD is about software design as I said before, and I would like my class to work, too. Some people only want the fast feedback, e.g. using REPL based development and "looking at UI is faster".

Missing Priority on Testing
When starting with TDD, the testing aspect is most visible. After all we have to write a reasonable test first. For teams and organisations with low or missing priority on testing, people are "looking down on testing" and I got answers like "testing is a culture thing", "testing is not a first class activity" and "I am not asked to create a test by my project manager". Indeed it is hard to keep following TDD if it is looked down upon and if there is no time for quality work.

Avoiding Context Switches
There is a certain amount of context switching involved in TDD. Similar to Edward de Bono's Six Thinking Hats, we have different states which we have to be mindful of and which call for different actions. George Dinwiddie created a TDD Hat to show that. Maybe this switching is "not natural for some people". "I don't want to interrupt creative design with verification" and "I prefer staying in building hat and not change to testing hat". Similar one participant said that it is "easy to write code, harder to write tests, so I do it afterwards". I understand and there is certainly an urge to jump into the code and get hacking. I rarely feel that urge and I enjoy pair programming using the Ping Pong style because it enforces the separation of states without any (inner) discussion.

Missing TDD Skills
This is obviously the largest area and there is nothing wrong with not knowing how to apply Test Driven Development: Honest people just say "I can't do it". Many are aware of this problem and seem to be disappointed with existing material and/or look for more material to study TDD: "It is not taught at universities", "there are no good books" and "I am missing real examples". I know from my own experience that TDD is not easy to learn and some people are "scared for life after a bad experience" with it. Now the best way to learn TDD is to have someone show you while pairing with you. Even if there is no pair programming in your workplace, you can still experience it during a Coding Dojo or Coderetreat. Short of that, I recommend Kent's Beck Test Driven Development by Example, which is a short and excellent introduction.

New Language or Library
When discussing TDD and unit testing with a client, he said I "don't know the target technology" and "React is a new technology for us". I had to laugh. To me this sounded like "I got a car and know how to drive forward, but am not able to drive backwards." On the other hand I live at the dead end of a road and I see drivers working really hard to avoid driving backwards. Are they not able to do it? So maybe stopping halfway in the game (of skill acquisition) is natural after all. When working with some new language or working with an unknown API, I specially rely on tests to support me, these are Learning Tests.

It's too hard to test
I agree some things are harder to test than others. "Android is hard to test", "Vaadin is hard to test" and "some libraries are hard to test". (I have not worked with Android or Vaadin, I quote people.) We might need to know more about design to decouple things. This is definitely true for legacy code, as "existing code is usually hard to test". Some people see the root cause, like in "I don't know how to manage boundaries". In such situations we need (to know) more tooling. We definitely "need more tooling to test the UI" as UI is traditionally considered hard to test from a TDD perspective. Still, Steve Freeman and Nat Pryce, authors of Growing Object-Oriented Software Guided by Tests, always start their TDD (outer) loop with an UI test. GOOS is a great book and I recommend reading it if you want to go deeper into TDD.

It's too simple to test
If there are things which are too hard to test, there must also be things which are too simple to test, right? It is "useless to test, it is so simple" and it "makes no sense to test it". Maybe a better description is that it is "unclear what is important to test". From a TDD perspective no such things exist and I guess these reasons arise from the test after process, when looking at each public method and thinking how to test it. Further excessive test isolation, see Solitary vs. Sociable Unit Tests, will cause that.

Barriers to TDD adoption
Here is Matt Wynne's summary of Barriers to TDD adoption from a session during Lean Agile Scotland 2016. I recommend checking out the Twitter thread as Matt added detail discussions on temptation of fast reward, permission and safety to learn, "the egotist" and other reasons not covered by me.

Barriers to TDD adoption #lascot16 (C) Matt Wynne
What about test-induced design damage?
Maybe the only real reason not to do TDD is to keep the design integrity of your system. This idea was started back in 2014 by David Heinemeier Hansson, also known as DHH, and led to the whole Is TDD Dead? debate. DHH said that when using TDD code sometimes suffers tremendous design damage to achieve two testing goals: Faster tests and easy-to-mock unit tested controllers and that the design integrity of the system is far more important than being able to test it any particular layer. It is ironic that this never comes up during any group discussion or team interview. Probably because it is an expert level reason. If you followed the debate, DHH knew TDD, he used it for some time and liked it. And then, only then, did he know when not to apply it.

30 October 2017

Managing the Difficulty of Coding Exercises

There are different scenarios when we might want to change the difficulty of coding exercises. This depends on our skill and the topic we want to practise. If an exercise is too easy we get bored. There is still value in repeating the very same exercise, e.g. internalising certain patterns or improving keyboard navigation, but boredom does not help learning. Here is an unsorted list of options to increase (and decrease) the difficulty of coding exercises:

Most DifficultMaking it Harder: Constraints
A constraint, or activity, is an artificial challenge during an exercise. I have discussed some of them in the past. Some constraints like No If, Cyclomatic Complexity One or Only Void Methods are easy to follow but make it hard to write your usual code. To have more challenge chose constraints that work against the assignment, e.g. use an algorithmic challenge together with Only Void Methods. Algorithms are often functional in nature but void methods are no functions. Win!

To make things more interesting, constraints can be combined. For example, Object and Functional Calisthenics are constraints that combine several rules. When creating combined constraints, it is important to make sure the constraints work together. There is no point in forcing a functional style with No Void Methods and an object oriented style with Only Void Methods at the same time. When Martin Klose and I combined the Brutal Coding Constraints we spent around 20 hours experimenting and fine tuning them. By the way, these Brutal Coding Constraints are probably one of the most challenging.

When the list of constraints gets long, it is easy to make a mistake and forget to follow one or another. In these situations you need a reviewer, e.g. Coding Dojo facilitator, pair programming partner or static code analysis tool, who checks for violations of constraints.

Harder: Changing Requirements
Another way to spice up an exercise is to introduce requirement changes. This is particularly useful for groups, e.g. Coding Dojos, when participants do not know which requirement is going to change. For the usual Coderetreat exercise Game of Life, several interesting changes have been proposed, e.g. Hex Life, Vampire Cells and the toughest constraint (Wormholes) by Adrian Bolboaca.

I witnessed Martin Klose taking this to the next level: In his exercise Wind of Change, he puts on a tie (because he is the product owner now) and keeps changing the requirements every few minutes. This is a lot of fun and adds some time pressure as well.

Requirement changes is useful to verify a design, usually used in double sessions on software design during Coderetreats. When you are on your own, as soon as you finished the exercise, you think of changes to the requirements and how they would affect your current design.

Harder: Algorithmic Challenges
Algorithmic challenges vary from easy to impossible. Project Euler even has a difficulty rating on each exercise. Often algorithmic challenges are based on mathematics, which makes them not useful for people with less academic background. Also, as soon as you found a solution, the exercises get boring. Using additional constraints can make them fun again, but that would be different exercises then.

I have seen senior developers being more interested in algorithms than XP practises like Pair Programming or TDD. Algorithms are a perfect way to "lure" them into attending Coding Dojos. After a few dojos, people understand the value of practise and will agree to do basic katas with focus on TDD.

If you need a challenge, go for an algorithmic kata and chose a difficult exercise like Potter, Searching or one from Project Euler above number 20.

Harder: Try to be Faster
I do not like to apply time pressure during exercises, because people get sloppy when under pressure. On the other hand, this is what needs to be trained to not get sloppy. Houssam Fakih explains this with a short video (where three people throw basket balls. One is a beginner and fails from time to time, one is experienced and wins repeatedly and one, a "master", is doing the same, but much faster.) Houssam's suggestion is to do the same, but try to be faster. I did that once because I wanted to squeeze an one hour life refactoring demo into a 45 minutes presentation slot. It was hard work, exactly what I wanted.

BalanceWarning
When using constraints and other techniques I describe above, it is easy to go over the top. The exercises become too difficult and working on it is frustrating and eventually we stop doing it. While this might be OK for yourself, it must not happen when working with a group. Exercises like Brutal Coding Constraints are very difficult and not - I repeat - not suitable for a general audience. People tend to overestimate their skill and get frustrated easily.

When facilitating a Coding Dojo, I want to stay in control of the difficulty of the exercise for all participants. I aim for easier, simpler exercises and keep the difficult ones for myself. In rare cases, when I meet very skilled people, I assign them individual constraints, because I know them and I am confident that they will handle. I also make sure everyone understands that it is difficult what they want to do.

Making it Easier: Simpler Assignments
Start with a simple problem. There is always a smaller assignment, The smallest kata I know is FizzBuzz, it is just a single function. There is nothing wrong with FizzBuzz and its friends. I do it from time to time when I explore a new language or try different constraints (or when it is very late and I feel tired). Some function katas like Prime Factors are small too, but algorithmic in nature, so stay away from them. These katas are called FizzBuzz or Function Katas.

Easier: Use Well Known Problems
Solving a programming assignment includes many steps: e.g. understanding the problem, finding a solution, implementing the solution, testing it, etc. The assignment is easier if we get rid of some of these steps. If we use a well known problem, e.g. a ticket machine or a game everyone knows, we already know what is expected.

Easier: Clarify/Understand the Problem
Often the problem with an exercise is that people do not understand the problem. We are eager to get into the code, but we need to understand the assignment first: Take time to analyse the problem you want to solve. Google it. If it is a game like Tic-Tac-Toe, Minesweeper or Pac-Man, find an online version and play for a while. Draw some sketches or diagrams of what needs to be done. Create an list of initial acceptance criteria. To find them, you have to think about the problem. In Coding Dojos I ask people to spend the first ten minutes on creating a test list. This forces them into thinking about the problem.

If you practise with a partner, which I highly recommend, try Adi's pair programming game Solution Seeker. Solution Seeker makes you find at least three different solutions to your problem before you are allowed to implement one of them. This forces you to think hard about the problem and different options to solve it.

As a facilitator, make sure you fully understand the problem so you can answer any question about it. Give more explanations and discuss the problem from different angles. Provide posters or handouts of the problem for participants for later reference.

EasierEasier: Repeat the Same Exercise
Repeating the exactly same exercise is considered boring, but it helps. You will understand the assignment better after working on it once. After implementing it several times, maybe even in different programming languages, more and more aspects of the implementation are known and you can go deeper. (This why we run Game of Life in a Coderetreat six times. We do not want to fight with a new problem each session.) This is especially true for hard problems or if you are not satisfied with your process or final solution.

Easier: Focus on One Thing
After repeating the same exercise one or more times, the problem is sufficiently known and you can shift your focus to something else. Using a well known problem is also a way to focus on one thing, in this case you do not focus on solving the problem. There are exercises that isolate different aspects of development: For example, if I want to focus on finding test cases and designing unit tests, I go for the Gilded Rose. If I want to practice refactoring, I do Tennis or Yatzy. Both code bases contain ugly code which is more or less fully covered with tests, making it safe to refactor. There are exercises isolating other things, like incremental development, emergent design, SOLID principles, etc.

Koans belong into this category. Koans are series of little exercises, starting with basic things and building on each other to move to more advanced topics. They are useful to learn programming languages. They contain a list of failing test cases, where tiny pieces of code have to be filled in to make them pass. The idea is not only applicable to programming language constructs. For example I have created Unit Testing Koans to teach xUnit assertions and life cycle to junior developers.

All these exercises require prepared code. For example Gilded Rose is available in 26 languages, including lesser common ones like ABAP and PL/SQL. Trivia even contains COBOL and VB6 - which is very suitable for a legacy code exercise. Obviously prepared code limits the number of languages which can be used. If you want to practice in a new language like Elixir, Elm or Swift, you might need to port the code base first. Although, if the new language is trending, chances are high that someone already ported it.

Easier: Prepared (Helper) Code
Prepared code is useful in many situations, especially outside the core of the practise. Even code snippets or cheat sheets help. For example when I run the Data Munging exercise with focus on functional programming in Java, I show participants code snippets how to read the text file. File IO is not related to the exercise and I want them to spend time working with Lambda expressions and Stream.

Prepared code allows us to focus on one thing, but we need to understand the code first. Unfortunately this adds extra complexity. Unless you want to practice working with unreadable code, prepared code must be simple and super clean. Try to make it more expressive, maybe even verbose, than your usual code and use very descriptive names. Describe the code in the assignment. If there are more methods or classes, visualise their relations. For example in my Test Double exercise, I added a simple diagram of the prepared classes and their collaborators.

Easier: Guide Step by Step
Alex Bolboaca once told me that as facilitator of an exercise it is most important to manage participants' frustration. When I notice that most of the participants are unable to move forward, I take control of the group and guide them step by step. I am not giving them answers, but moderate the necessary process. Maybe we need to discuss the problem before hand on a white board. Or we discuss potential solutions up front. To get an initial test list, I keep asking how we will verify our product until we have a reasonable number of test cases. Sometimes I switch to Mob Programming where the whole group works on the assignment together and I am able to support them best (a.k.a. micro manage).

Conclusion
There are many options to make coding exercises easier or more difficult. I recommend starting easy. There is no point in hurting yourself or others. ;-)

Thanks to Kacper Kuczek and Damian Lukasik for discussing this with me.

29 August 2015

Introducing Brutal Coding Constraints

Last year I teamed up with Martin Klose to run a workshop at the Agile Testing Days. We knew that some really experienced developers would be there and aimed for an expert level workshop. We wanted a really difficult session, something that was hard, maybe even impossible to do. So we came up with the idea of Brutal Coding Constraints.

A constraint is an artificial challenge during an exercise designed to help participants think about writing code differently than they would otherwise. Some constraints are an exaggeration of fundamental rules of object oriented design and are applicable during your day to day work. The more extreme ones might still help you understand the underlying concepts of object orientation.

playmobil executionerBrutal Coding Constraints
The Brutal Constraints constraint is a composite constraint like Object Calisthenics, a combination of several constraints, some of them already difficult enough on their own. One particular combination that we like is
  1. No Conditionals, i.e. no if, unless or ?: operator. while can be used as conditional too and is not allowed as well.
  2. No Loops, i.e. no for, while, do or repeat until or whatever repetition constructs your language offers. Together No Conditionals and No Loops are sometimes called Cyclomatic Complexity One.
  3. TTDaiymi (TDD as if you Meant it), a very strict interpretation of the practice of TDD. This is optional, a "bonus" constraint for experienced developers. If you never heard about it, just ignore this one.
  4. No Naked Primitives, i.e. wrapping all "primitive" values, e.g. booleans, numbers or strings. Also general purpose containers like List, Map or Set are considered primitive. In extension all generic types of your language are primitive because they are not from your domain. A generic date (e.g. java.util.Date) is not from your domain even if you use dates, because it either does not define all methods you need or it defines other methods you do not need.
  5. No void, i.e. all functions must return something, methods with no return value are forbidden.
  6. Immutable, i.e. all data and data-structures must be immutable.
All six constraints are regular Code Retreat Activities so I will skip their further discussion.

Violating the Rules
When practising these Brutal Constraints with some kata, Martin and I were not able to find an implementation that would satisfy all constraints right away. We usually allowed violations in the beginning and refactored towards the constraints after the green phase. Sometimes we would leave a violation in for a few red-green-refactor cycles. It helped us to go through the list in each refactoring step to make sure we did not forget any constraint. It happened that we had put a condition somewhere in the code and forget about it - we are just that used to using conditionals and loop constructs. Because participants often ask for it, here is a list when to be strict about the rules. It is allowed to violate constraints:
  • temporary until you fix them during the next refactoring step;
  • temporary until you fix them after triangulating a solution, probably during a larger refactoring step after several cycles, e.g. after the third test;
  • if an used framework requires it, e.g. using a Runnable needs a void run() method;
  • if the testing frameworks requires it, e.g. JUnit test methods are void methods and @Parameterized tests need List<Object[]> which is a primitive container.
In general it seems easier to refactor later but that defeats the purpose of the whole exercise. We ordered the six constraints by some kind of priority or difficulty. If you have to violate a constraint, try to follow the ones higher up in the list at least.

difficultiesDifficulty
As I said before, Martin and I aimed for a really difficult exercise and I think we offered the only expert level session at ATD2014. And yes, this exercise was hard. The difficulty of each constraint was multiplied by their combination. For example I wrote about combining TDDaiymi, No Naked Primitives and No Conditionals last year. The attendees of the workshop agreed, "it was really difficult" up to "WTF" ;-). Brutal Coding Constraints are definitely too difficult for programming beginners, who even struggle with the concept of immutability.

An exercise like the Brutal Constraints can get frustrating easily. We told the participants that we made the session impossible on purpose, so they would not feel bad when getting stuck. When creating the workshop we got stuck ourselves several times, so we knew what to expect. During the session we paid close attention to the participants' mood and were prepared to offer hints on how to proceed without violating constraints. All participants worked hard and enjoyed the exercise.

Why practise like that?
When we prepared for the workshop we experimented with these constraints several times both in Java and JavaScript. It was difficult and interesting at the same time. I discussed some of our findings already. Also the ATD2014 participants liked the exercise. In the feedback round several people said that the constraints forced them to "think outside of the box" and that they liked the opportunity to "deviate from usual way" how they create software.

What about Functional Programming?
The Brutal Constraints focus on Object Oriented Programming. No Naked Primitives is the main driving force to create more types. On the other hand, constraints 1, 2, 5 and 6 might not challenge in Functional Programming. Instead of explicit conditionals some languages provide an Option or Maybe type and filter operations remove unwanted elements from containers. Most loops are unnecessary because containers provide map, foreach or similar operations. Also a recursive function call is not a loop. Pure functions and immutability are base concepts of functional programming anyway. I would like to see a solution following the above constraints in Clojure or Haskel. I am unsure how No Naked Primitives translates into Functional Programming.

Tic Tac ToeThe Assignment
In general the actual assignment, i.e. the problem that participants are asked to solve, does not matter for a kata but we wanted a problem that did not support the constraints. (Evil grin ;-) Such a problem would have a linear or higher dimensional structure with a need for looping (which is not allowed) and business rules, which are conditionals (which are not allowed either). We started with the classic Game of Life but it took us too long to reach a point where the constraints forced us to think harder. So we switched to a smaller problem, finding the winner of a game of Tic-tac-toe, which worked well for us.

Hints for Facilitators
If you plan to host your own Brutal Constraints exercise, your first priority is to help the participants to meet the constraints. It is easy to miss an if or a void method. We recommend printing the list of constraints and TDDaiymi rules as handout for each pair in the workshop. Second you need to pay attention to people's mood, as I described above.

For a short workshop it would help to force participants into situations conflicting with the Brutal Constraints as early as possible. One way to do this is to start with a prepared code base that already contains the first loop or conditional which needs to be removed. But then the list of supported programming languages for the workshop is small, ruling out less popular ones. Another way is to ask participants to follow a list of predefined test cases. While this allows any language, it impedes the creativity of solution finding. We still have to find a good list of test cases though.

Moar Brutality
But why should we stop here? We can make the exercise even more difficult. A suitable constraint to add is No Duplication, i.e. being very aggressive about duplication in the code. Unfortunately detecting duplication is less straight forward than following constraints 1, 2, 4, 5 and 6 which just deny certain reserved keywords or library classes. Another option is to add Baby Steps to force smaller working steps and Baby Steps has been combined with TDDaiymi already. When practising the constraints we committed every five to six minutes without problems.

Credits
Thanks to Martin Klose for creating the Brutal Coding Constraints with me. Pair facilitation is just so much more rewarding than solo work.