3 September 2014

Thoughts on Mastery

RakeDuring summer time I was on vacation in Tyrol which is an excellent place for hiking. A local magazine told a story about Josef Frauenschuh and his business of hand-crafted rakes. Josef is a true master craftsman: He is of old age, many years past official retirement. He creates large wooden rakes of highest quality using different kinds of wood for each part of the rake. His rakes are balanced and lightweight. He uses the best tools he can get, all of them are self-made and after 45 years he is still improving his tools and process. Obviously he does not accept any compromise. Once, when he needed more space to create the shafts, he simply made a hole in the wall of his workshop.

I liked the story a lot. It made me think about our craft and the concept of mastery.

I believe that one major aspect of mastery is age. It is not necessary by itself but rather is a side effect of the time needed to master a subject. Masters are aged because they have been practising their craft for 30 years or more and still try to improve. Following this definition it is obvious that there can not be many masters of software development, because our industry is still young. Only a few like Uncle Bob are working in the industry long enough, whereas most of us have less than ten years of experience. The so-called senior developers with five years of experience are just young journeyman, if at all.

Rastrello Di LegnoJosef is working hard. Although he could retire any time, he is working five days a week because the demand for his rakes is high. He is a master and his rakes are masterpieces, but does he get rich? I do not think so. Following the story I assume he has to put considerable effort into each rake, and sometimes the wood splinters and he has to start over. Being curious I compared the prices and his rakes sell for around 70 Euro, whereas eBay has some for ten to 20 Euro, although no wooden ones. So yes, the masterpiece is up to five times more expensive than a regular one.

Still it seems that Josef is not earning five times more per hour than other workers. Could it be that master craftsmen are relatively underpaid because they put in extra work to create magnificent things? Maybe his rakes are expensive to compensate for the small number he is able to produce. That does not compare well to our industry. We (developers) are greedy. We expect a good salary matching our experience and competence. The more experience we have, the more money we want, which seems only fair. The never ending demand for IT professionals spoils us and salaries rise during the first years of junior developers' careers. At least they rise up to a certain point, which might be 15 years of experience, when we become "too senior" for most employers. But this is another story.

I do not know Josef Frauenschuh but I believe him to be a true master craftsman. Instead of bragging about his achievements, he is modest and admits that sometimes he even has to listen to outsiders to improve his rakes. He does not need titles of seniority, nor is he proclaiming himself to be a master. I guess he considers himself still a simple carpenter. He is a great example. Some of his modesty would suite us software people well, and I will start with myself.

31 August 2014

In the meantime

Oops, summer is almost over and there are no new blog posts. But as you can see, I have been busy...

Demolishing The Brick WallThe picture does not show me but my wife, the whole family is working hard on the current project.

Dust In The AtticUnfortunately the building is not "soft" and the planned refactorings are difficult to implement.

31 May 2014

Game of Life Apache Ant

This post is about the second half of my quest to create Conway's Game of Life using a plain Apache Ant build script. In the first half I test-drove the code up to target apply_rules which determined if a given cell would be reborn, live on or die. This part is about applying the rules of the game to whole generations. Here are the test cases to identify the next generation.
<target name="all-test-targets" description="run all test targets">
    ...
    <antcall target="test-next-gen-living-with-no-neighbours-dies" />
    <antcall target="test-next-gen-empty-with-three-neighbours-comes-alive" />
</target>

<target name="test-next-gen-empty-with-three-neighbours-comes-alive">
    <make-tmp-dir />
    <antcall target="create_3x3_grid" />
    <touch file="${test.temp_dir}/.y/.x.x/cell" />
    <touch file="${test.temp_dir}/.y.y/.x/cell" />
    <touch file="${test.temp_dir}/.y.y/.x.x/cell" />

    <antcall target="next_generation">
        <param name="p_cell_location" value="${test.temp_dir}/.y/.x" />
    </antcall>
    <assert-file-exists file="${test.temp_dir}/.y/.x/next" />
</target>

...

<property name="cell.marker_will_live" value="next" />

<target name="next_generation" depends="apply_rules" if="cell.will_live"
      description="creates a 'next' file if a cell in the given location
                   'p_cell_location' will be in the next generation.">

    <touch file="${p_cell_location}/${cell.marker_will_live}" />
</target>
The target next_generation marks a cell which will live (on) by creating an empty marker file inside its folder.

Center of the Milky Way Galaxy (NASA, Chandra, 111009)The Universe is (maybe) infinite
In the previous part I defined the universe as a folder structure containing all cells.
universe/
    .y/
      .x/
      .x.x/
      ...
    .y.y/
      .x/
      .x.x/
      ...
    .y.y.y/
    ...
The universe is 2-dimensional and has a certain size. To support universes of arbitrary size I started testing for 2 times 2 grids,
<import file="lib/asserts.xml" />
<import file="universe-build.xml" />

<target name="test-create-empty-2x2-universe">
    <make-tmp-dir />

    <antcall target="create_universe">
        <param name="p_universe_location" value="${test.temp_dir}" />
        <param name="p_universe_size" value="2" />
    </antcall>

    <assert-file-exists file="${test.temp_dir}/universe/.y/.x" />
    <assert-file-exists file="${test.temp_dir}/universe/.y/.x.x" />
    <assert-file-exists file="${test.temp_dir}/universe/.y.y/.x" />
    <assert-file-exists file="${test.temp_dir}/universe/.y.y/.x.x" />
</target>
and then for 3 times 3, 4 times 4 and so on. The creation of each universe was straight forward, but there was a lot of duplication in the different creation targets. Creating an 1 by 1 universe was trivial and adding dimensions step by step seemed a good way to remove duplication.
<target name="create_universe_3" depends="create_universe_2">
    <antcall target="universe_add_dimension" />
</target>

<target name="create_universe_2" depends="create_universe_1">
    <antcall target="universe_add_dimension" />
</target>

<target name="create_universe_1" if="universe.location">
    <mkdir dir="${universe.location}/.y/.x" />
</target>
Increasing the grid's first dimension, the y- or row-dimension, of an existing grid is done by adding a new row folder .y.y.y. Adding the second, x- or column-dimension means adding another column .x.x.x to each row folder .y, .y.y and so on. In Ant this can be done using copy and regexpmapper adding another .x or, respectively .y to the matched group of .xs or .ys.
<macrodef name="universe_add_row"
      description="copies the row folders and renames them as additional row.">
    <sequential>
        <copy todir="${universe.location}" includeEmptyDirs="true" overwrite="false">
            <fileset dir="${universe.location}" includes="${universe.struct}" />
            <regexpmapper from="^(.*\.y)(/.*\.x)$$" to="\1.y/\2" handledirsep="yes" />
        </copy>
    </sequential>
</macrodef>

<macrodef name="universe_add_column"
      description="copies the column folders and renames them as additional column.">
    <sequential>
        <copy todir="${universe.location}" includeEmptyDirs="true" overwrite="false">
            <fileset dir="${universe.location}" includes="${universe.struct}" />
            <regexpmapper from="^(.*\.x)$$" to="\1.x" />
        </copy>
    </sequential>
</macrodef>

<target name="universe_add_dimension" if="universe.location"
      description="adds one dimension in column- and row-folders.">
    <universe_add_column />
    <universe_add_row />
</target>
That removed all duplicated mkdir commands in the creation targets. Still I was not happy with that solution because I needed to provide a target <target name="create_universe_N"> and all previous targets for each dimension N I wanted to support. I spent some time trying to resolve this but failed. Ant does not provide any kind of loop construct and does not allow for recursion when calling targets. (Using Ant-Contrib's extensions like For was out of the question because I wanted to write idiomatic Ant which is rather declarative than imperative.) So the current solution only supports grids up to a fixed dimension.

GenerationsGenerations
After creating universes of (almost) arbitrary size and seeding them I had to iterate all cells (folders) to update the whole universe at once. I did not need any new code, just a way to execute the cell-build.xml script inside each cell (folder) of the universe. After creating a (sort of integration) test for the desired functionality, I added a final target to the cell-build.xml that initialised the cell's location (p_cell_location) with the current build script's directory (${basedir}).
<target name="next_gen_for_basedir_cell"
      description="creates a 'next' file if the cell in the
                   basedir will be in the next generation.">

    <antcall target="next_generation">
        <param name="p_cell_location" value="${basedir}" />
    </antcall>
</target>
Then, after some experiments, I was able to use Ant's subant task to execute the logic for each cell inside each cell's folder.
<property name="universe.base_dir" value="universe" />
<property name="universe.struct" value="*/*" />
<property name="cell.build_file" value="cell-build.xml" />

<target name="next_generation" if="p_universe_location"
        description="applies the rules for next generation for each cell of
        the universe folder structure in location 'p_universe_location'.">

    <subant genericantfile="${cell.build_file}">
        <dirset dir="${p_universe_location}/${universe.base_dir}"
                includes="${universe.struct}" />
    </subant>
</target>
Including */* in the dirset selects only cells (the leaf directories) and no intermediate ones. I have never used subant before but I am still amazed by its power. It is a great tool to get rid of duplication or complicated series of targets. Even using an Ant extension with its loop would not produce such a concise way of iterating all cells.

The above code creates empty marker files (next) for all cells that will be alive in the next generation. All that is left is to delete the old generation (${cell.marker_alive}) and rename the new generation (${cell.marker_will_live}) to the current one.
<target name="evolve" depends="next_generation" if="p_universe_location"
      description="evolves the whole universe folder structure in location 'p_universe_location'.">

    <property name="universe.location" location="${p_universe_location}/${universe.base_dir}" />
    <delete dir="${universe.location}" includes="${universe.struct}/${cell.marker_alive}" />
    <copy todir="${universe.location}" includeEmptyDirs="false" overwrite="false">
        <fileset dir="${universe.location}" includes="${universe.struct}/${cell.marker_will_live}" />
        <regexpmapper from="^(.*)${cell.marker_will_live}$$" to="\1${cell.marker_alive}" />
    </copy>
    <delete dir="${universe.location}" includes="${universe.struct}/${cell.marker_will_live}" />
</target>
The final test is more an acceptance test than a unit test because it exercises all rules and functionality by evolving a whole universe which has been seeded a blinker.
<target name="test-evolve-updates-all-cells">
    <make-tmp-dir />

    <antcall target="create_universe">
        <param name="p_universe_location" value="${test.temp_dir}" />
        <param name="p_universe_size" value="3" />
    </antcall>

    <!-- seed blinker -->
    <universe_seed_cell location="${test.temp_dir}" row=".y.y" column=".x" />
    <universe_seed_cell location="${test.temp_dir}" row=".y.y" column=".x.x" />
    <universe_seed_cell location="${test.temp_dir}" row=".y.y" column=".x.x.x" />

    <!-- run GoL -->
    <antcall target="evolve">
        <param name="p_universe_location" value="${test.temp_dir}" />
    </antcall>

    <assert-file-does-not-exist file="${test.temp_dir}/universe/.y.y/.x/cell" />
    <assert-file-does-not-exist file="${test.temp_dir}/universe/.y.y/.x.x.x/cell" />

    <assert-file-exists file="${test.temp_dir}/universe/.y/.x.x/cell" />
    <assert-file-exists file="${test.temp_dir}/universe/.y.y/.x.x/cell" />
    <assert-file-exists file="${test.temp_dir}/universe/.y.y.y/.x.x/cell" />
</target>
It works! ;-) Here is proof in form of a little animation of the Ant Game of Life in action:Ant Game of Life Dir Listings