3 May 2014

The Life of (an) Ant

I already mentioned that Daniel impressed me by implementing Conway's Game of Life in MSBuild with MSBTest. Being a Java developer for most of my career, Apache Ant was the proper choice to build the cellular automaton using a build tool. I have worked with Ant before, so I was confident I would be able to pull it off.

Bring Out Your DeadIdiomatic Ant
I wanted a challenge and decided to avoid Ant extensions whenever possible. For example there are if and for tasks in Ant-Contrib which enable an imperative, procedural coding style. Obviously embedding any kind of script was also considered cheating.

An Ant's Universe
Wikipedia says that the universe of the Game of Life is an infinite two-dimensional orthogonal grid of square cells. How could I represent such a grid using basic Ant data structures? Which types does Ant support at all? Ant offers immutable properties and property sets, files, directories and paths and supports text file filtering using regular expressions. Which data structure could I use for the universe? How about directories? The universe is a two-dimensional grid, containing rows and columns, which might be defined by the names of nested folders. A single row in the grid is a folder which contains the folders of all its columns, e.g.
universe/
    row_folder_1/
      column_folder_1/
      column_folder_2/
      ...
    row_folder_2/
      column_folder_1/
      column_folder_2/
      ...
    row_folder_3/
    ...
That looks promising. But how to find the previous and following columns given a current column (folder) name? There are no arithmetic functions available in Ant, so there is no way to increase or decrease a number by one. But it is possible to append a character to a property creating a new one, allowing to reach the cells (folders) right and down of the current one, e.g.
universe/
    row_folder_x/
      column_folder_x/
      column_folder_xx/
      ...
    row_folder_xx/
      column_folder_x/
      column_folder_xx/
      ...
    row_folder_xxx/
    ...
To access the left and upper neighbours of a cell the name has to be shortened. Ant is not able to shorten String properties but the basename task is able to strip the extension off a file or directory name. So the row and column folders are called '.y' and '.x'.
universe/
    .y/
      .x/
      .x.x/
      ...
    .y.y/
      .x/
      .x.x/
      ...
    .y.y.y/
    ...
Testing Framework
To do it "right", I needed a testing framework. Although there is Apache AntUnit, I used a simpler solution to test my build files. Inspired by my former colleague Andy Balaam's Test-Driven Ant macros I created the necessary assertion macros on the way. For example the following code asserts that a certain property is set:
<macrodef name="assert-propery-set">
    <attribute name="propertyname" />
    <sequential>
        <fail message="property '@{propertyname}' not set."
              unless="@{propertyname}" />
    </sequential>
</macrodef>
Tell me about your neighbours
Having the data structure and assertions in place, it was time for the first test. I started with tests for counting the neighbours of a given cell, expecting its implementation in target count_neighbours of cell-build.xml.
<import file="lib/asserts.xml" />
<import file="cell-build.xml" />

<target name="test-count-neighbours-in-middle-when-no-neighbours">
    <make-tmp-dir />
    <antcall target="create_3x3_grid" />
    <touch file="${test.temp_dir}/.y.y/.x.x/cell" />

    <antcall target="assert-number-of-neighbours">
        <param name="p_cell_location" value="${test.temp_dir}/.y.y/.x.x" />
        <param name="p_expected_neighbour_count" value="0" />
    </antcall>
</target>

<target name="create_3x3_grid" if="test.temp_dir">
    <mkdir dir="${test.temp_dir}/.y/.x" />
    <mkdir dir="${test.temp_dir}/.y/.x.x" />
    <mkdir dir="${test.temp_dir}/.y/.x.x.x" />
    <mkdir dir="${test.temp_dir}/.y.y/.x" />
    <mkdir dir="${test.temp_dir}/.y.y/.x.x" />
    <mkdir dir="${test.temp_dir}/.y.y/.x.x.x" />
    <mkdir dir="${test.temp_dir}/.y.y.y/.x" />
    <mkdir dir="${test.temp_dir}/.y.y.y/.x.x" />
    <mkdir dir="${test.temp_dir}/.y.y.y/.x.x.x" />
</target>

<target name="assert-number-of-neighbours" depends="count_neighbours">
    <assert-propery-equals propertyname="cell.neighbour_count"
                           expected="${p_expected_neighbour_count}" />
</target>
First I fulfilled the test with a fake solution.
<target name="count_neighbours">
    <property name="cell.neighbour_count" value="0" />
</target>
After eight more tests,
<target name="all-test-targets" description="run all test targets">
    <antcall target="test-count-neighbours-in-middle-when-no-neighbours" />
    <antcall target="test-count-one-neighbour-left" />
    <antcall target="test-count-one-neighbour-right" />
    <antcall target="test-count-one-neighbour-up" />
    <antcall target="test-count-one-neighbour-down" />
    <antcall target="test-count-one-neighbour-up-left" />
    <antcall target="test-count-all-neighbours-around" />
    <antcall target="test-count-neighbours-in-left-up-when-no-neighbours" />
    <antcall target="test-count-neighbours-in-lower-right-when-no-neighbours" />
count_neighbours was finished.
<property name="cell.marker_alive" value="cell" />

<target name="count_neighbours" if="p_cell_location"
      description="sets property 'cell.neighbour_count' to the count of living
                   neighbours of a cell in the given location 'p_cell_location'.">

    <dirname property="cell.base_row" file="${p_cell_location}" />
    <dirname property="cell.base" file="${cell.base_row}" />

    <pref_self_next dim="y" folder="${cell.base_row}" /> <!-- (1) -->
    <pref_self_next dim="x" folder="${p_cell_location}" /> <!-- (2) -->

    <resourcecount property="cell.neighbour_count"> <!-- (3) -->
        <fileset dir="${cell.base}">
            <include name="${cell.prev_y}/${cell.prev_x}/${cell.marker_alive}" />
            <include name="${cell.prev_y}/${cell.self_x}/${cell.marker_alive}" />
            <include name="${cell.prev_y}/${cell.next_x}/${cell.marker_alive}" />

            <include name="${cell.self_y}/${cell.prev_x}/${cell.marker_alive}" />
            <include name="${cell.self_y}/${cell.next_x}/${cell.marker_alive}" />

            <include name="${cell.next_y}/${cell.prev_x}/${cell.marker_alive}" />
            <include name="${cell.next_y}/${cell.self_x}/${cell.marker_alive}" />
            <include name="${cell.next_y}/${cell.next_x}/${cell.marker_alive}" />
        </fileset>
    </resourcecount>
</target>
Line (1) defines the properties for the previous, current and following row, i.e. in the y dimension and line (2) does the same for columns, i.e. in the x dimension. Both sets of properties are similar, so I extracted a macro to remove duplication. (Remember: Red - Green - Refactor)
<macrodef name="pref_self_next"
      description="defines properties 'cell.self_X', 'cell.pref_X' and 'cell.next_X' for
                   given dimension/orientation 'dim' from center of current 'folder'.">

    <attribute name="dim" />
    <attribute name="folder" />

    <sequential>
        <basename property="cell.self_@{dim}" file="@{folder}" />
        <property name="cell.next_@{dim}" value="${cell.self_@{dim}}.@{dim}" />
        <basename property="cell.prev_@{dim}" file="@{folder}" suffix=".@{dim}" />
    </sequential>
</macrodef>
After determining the names of the previous and following neighbouring cells (folders), counting the live ones (files) was be done with Ant's resourcecount. Line (3) counts the number of files called 'cell' in the neighbouring folders.

To Be or Not To Be
With the number of living neighbours I was ready to implement the rules of the game right away. The target apply_rules uses this number to determine if a cell lives on or dies. With four tests,
<target name="all-test-targets" description="run all test targets">
    ...
    <antcall target="test-rules-with-two-neighbours-lives-on" />
    <antcall target="test-rules-with-one-neighbour-dies-on" />
    <antcall target="test-rules-empty-with-two-neighbours-stays-dead" />
    <antcall target="test-rules-empty-with-three-neighbours-comes-alive" />
apply_rules evolved to
<target name="apply_rules" depends="count_neighbours" if="p_cell_location"
      description="sets property 'cell.will_live' only if a cell in the given
                   location 'p_cell_location' will live on for the next generation.">

    <condition property="cell.will_live">
        <or>
          <and>
              <available file="${p_cell_location}/${cell.marker_alive}" />
              <equals arg1="${cell.neighbour_count}" arg2="2" />
          </and>
            <equals arg1="${cell.neighbour_count}" arg2="3" />
        </or>
    </condition>
</target>
Target apply_rules expects the property p_cell_location to point to a given cell (folder) in the universe. It first calls the target count_neighbours via depends and uses the cell.neighbour_count to determine if the given cell will be reborn, live on or die. This implements the rules of the game and ends this post. The next one will use apply_rules to run a full Game of Life on arbitrary universes. Stay tuned!

18 April 2014

How to divide learning time

As software delivery professionals we constantly need to learn and practice to keep our skills sharp. Last year I even spent three months pair programming to do exactly that. Possible activities required for learning and related to the sharing of knowledge are:
  • (a) studying, e.g. reading technical books or blog posts, watching (recorded) presentations as well as any form of consuming knowledge.
  • (b) practising, e.g. doing code katas, attending Coding Dojos or working on fun (but otherwise useless) projects.
  • (c) experimenting, which includes creating prototypes or trying out new frameworks or ideas.
  • (d) creating something real, e.g. hobby projects or maybe contributing to Open Source.
  • (e) documenting and sharing, e.g. creating how-to documents or writing blog posts.
  • (f) teaching, e.g. preparing presentations and talking at user group meetings.
Why are documenting, sharing and teaching considered learning activities?
Obviously sharing is no learning activity by itself but the process of analysing and describing things improves our understanding similar to Rubber Ducking. For example writing an article about some topic is a great way to bring one's thoughts into order and learn more about it. Documenting and sorting knowledge is also part of PIM (Personal Information Management) which is related to learning as well. Teaching is considered a learning activity by most people. Teaching is a great way to perfect the understanding of an idea. This is sharing on steroids as you are forced to dig deep into the topic to be able to explain it to others.

Piano LessonsHow should I divide my learning time into activities (a) to (f)?
With the start of 2014 I came back to (sort of) regular work and obviously my personal time for learning became limited again. I was not sure if I spent the available time in the most efficient way and looked for guidelines how to divide my learning time into these activities. All activities (a) to (f) seem vital, but likely do not have the same priorities. For example there are so many important books to read, I could spend all my time on reading alone, which does not feel right to me. Sören Kewenig pointed out that items (a) to (f) might be seen as the phases of a project: a -> [c,d,b] -> e -> f, which also indicates that all these activities are important and related.

This question is subjective ;-)
As usual when I have a question I ask StackOverflow. In this case I asked on Programmers. Unfortunately the question was closed in less than 19 minutes as being off-topic. (Many of my questions I ask on StackOverflow get closed but 19 minutes is my personal record so far ;-) While it is true that learning is a subjective process, there might be some research that gives clear advice how to optimise learning activities.

Comments suggest that practice is the key.
Nevertheless I was able to harvest at least some comments from my StackOverflow question. Robert Harvey wrote "if you're not writing code that illustrates the principles being taught, they won't stick. [...] I am saying that, without practice, the other stuff won't matter." I conclude that there needs to be a strong focus on (b) or (d) until the newly acquired material is incorporated into the daily work and applied every day. Then I asked my fellow craftsmen of the Softwerkskammer, the Software Craftsmanship Communities in Germany, Austria and Switzerland. Sören answered me that he used 50% of his free time for Coding Dojos (b) and 50% for working on Open Source (d). Jan Sauerwein spends his time on code katas (b) and preparing a lecture (f). Franziska said that she liked learning with others and therefore favoured (b), (c) and (f). Ulrich Merkel said that developing skills needs as much practice as possible - (b), (c) or (d), but probably he was talking about real work - together with discussion and feedback as corrective. So all answers agree that practice, either by real work or (b), (c) or (d) is important.

How I divided my learning time till now
Franziska asked me how I divided my learning time till now. Yes I should have checked my own time before asking ;-) So I started tracking my learning time. For several weeks I spent an average of 10 hours on learning activities.
  • (a) studying: 20%. I spent some time reading. I am participating in an online book club which forces me to keep up reading and allows for discussions afterwards. I read the book while commuting. Further I browse my feeds of blogs and news to get an overview about what's going on and to stay aware of new trends.
  • (a) studying: 30%. I watched presentations. This includes user group presentations as well as recorded ones.
  • (b) practising: 20%. Once a week I meet online with a friend to do remote pair programming. We schedule the appointment up to one month in advance, to make sure it even happens when workload is high - and it usually works.
  • (d) creating stuff is still part of my job, although recently I am not doing as much development as I used to do.
  • (e) documenting: 10%. I used some time to look up slides of seen presentations and to take notes about what I have learned.
  • (e) sharing: 20%. An average of two hours went into writing blog posts, which was not enough to publish a full article but I worked on fragments of posts which eventually will be published.
  • (f) teaching is part of my new profession and I prepare a presentation now and then. Sometimes I also present something at user group meetings.
My Learning Time % Till Now
This looks pretty balanced, there are no huge gaps. I am surprised how much I do read because I always feel like I should read more but then I am too tired most evenings. Maybe I should watch more demonstrations though. Also 20% practice might not be enough compared to the previous comments by Robert and Sören.

The Learning Pyramid
Not all learning activities are equally effective. For example I remember a vivid discussion about something much better than just reading about it. Edgar Dale summarises these findings as Learning Retention Rate which ultimately leads us to the Learning Pyramid. In the Learning Pyramid, the learning activities are ordered by how much they pay off.

Learning Retention Rate % by Edgar Dale
When looking at these rates I see that I need to add another activity to my list: discussion, which might happen during user group meetings, practice with others or teaching. So let me revisit my tracked learning time and distribute it according to the pyramid.
  • Lecture, 5% retention. Next to real lectures that I sometimes attend, I consider some user group meetings to be in this category. Sometimes presentations are of mixed quality and there is not much discussion. 15% of my learning time (a) goes here.
  • Reading, 10% retention. I spend 20% of my time reading (a).
  • Audio-Visual, 20% retention. To be fair, not all presentations I see are boring, so I consider some of them to be of this category. Here I spend another 20% of my time from (a).
  • Demonstration, 30% retention. It is difficult to assign one of these categories to a typical user group talk or conference presentation. Some presenters give high quality demonstration, but I see this rarely, maybe 5%.
  • Discussion Group, 50% retention. Especially in the reading group (a) there is much discussion. Maybe 10% of my time is of this category.
  • Practice by doing, 75% retention. With two hours of pair programming code katas (b) each week I score 20% here.
  • Teaching others, 90% retention. As I teach as part of my day job (f), I did not track the time. Also I am left with two hours documenting and blogging (e) which do not fit into the pyramid. To make things easier I will just award myself honorary 10% here.
When multiplying my learning time by the retention rate I end up with a weighted learning time, let's call it my "effective" learning time:
My Effective Learning Time %
  • Lecture 2%,
  • Reading 5%,
  • Audio-Visual 10%,
  • Demonstration 4%,
  • Discussion 14%,
  • Practice 40%,
  • Teaching 24%.

How I (maybe) should divide my learning time
It is time for a conclusion. I have spent some time analysing the topic - and this made some things clear to me - call it meta-learning. I do not know if the final distribution of my effective learning time is well balanced or not. On one hand the activities with smaller retention rate obviously do not pay off, so there is no point in doing more of them. On the other hand - maybe - there would be some benefit in spreading the learning across all activities. As the retention rates (5, 10, 20,...) look (sort of) exponential, the effective learning would also look exponential for equally distributed learning times.

Now I see that my feeling about not reading enough comes from the fact that I do not learn enough while reading, which is obvious when looking at my effective learning time for reading. Even if I would add another hour of reading each week, it would not make much difference. But I still believe that it is important to read technical books, because it is a good way to cover a topic in depth. So I will stick to my reading routine of one to two hours each week. I will continue watching presentations, because I usually watch them during cardio-workout. I will add demonstrations, but I am not sure where I could find these. Maybe I will look for user group meetings with live coding. I am doing a lot of practice already, and it is dominating my learning. I will have to think about this. Just half an hour moved from practice to teaching each week might improve my effective learning. So even if I give presentations as part of my day job, I will look for user groups willing to listen to me ;-)

As others already pointed out, different people have different learning styles. Learning styles might even change for the same person over time. I am very interested in your take on dividing learning time and how you think the few hours of personal learning could be spent best.

25 March 2014

Maven Site using Markdown

I like Markdown. It is a popular lightweight markup language, easy to read and easy to write. It is supported by many websites and text editors (and recently got standardised). I use it since many years and whenever I take notes, even in Notepad or on my mobile phone, I structure my text using at least I use * to list items and --- to separate paragraphs. Recently when I had to publish some documentation for a project, I was not surprised to find my existing notes in Markdown. So why not generate the final document from them?

Newley Marked Down Misspelled Sale Sign FailApache Maven can be used to build a project site. If you are a Java developer you probably have seen some of these sites already. But the auto generated Maven reports are not much use if you want to add real documentation. The recommended formats to create content are APT or Xdoc. While the APT format is similar to Markdown, I did not want to convert my documents and went off tweaking Doxia for Markdown. This took me extra two hours, but hey - work is supposed to be fun sometimes too, right?

Maven Doxia is the content generation framework used by the site plugin. It supports Markdown since Doxia 1.3. The site content is separated by format and Markdown files need to be inside a markdown folder and named *.md.
project
|-- pom.xml
`-- src
    |-- main
    |-- test
    `-- site
        |-- markdown
        |   `-- Readme.md
        `-- site.xml
To enable markdown processing in Maven site the doxia-module-markdown has to be added to your pom.xml.
<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-site-plugin</artifactId>
      <version>3.2</version>
      <dependencies>
        <!-- add optional Markdown processor -->
        <dependency>
            <groupId>org.apache.maven.doxia</groupId>
            <artifactId>doxia-module-markdown</artifactId>
            <version>1.4</version>
        </dependency>
      </dependencies>
    </plugin>
  </plugins>
</build>
During mvn site all Markdown files are translated to HTML and put into the target/site folder. To add them to the menu of the generated site, you have to add links to the decoration descriptor of Doxia, also known as site.xml.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/DECORATION/1.4.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/DECORATION/1.4.0
                        http://maven.apache.org/xsd/decoration-1.4.0.xsd">

    <body>
        <menu name="Documents">
            <item name="Readme" href="Readme.html" />
        </menu>

        <menu ref="reports" />
    </body>

</project>
Done. Success. Profit. ;-)