26 February 2022

Porting RLE8

I am going back in time (again) and playing with some of my old source code: In the days of MS-DOS, 16 bit operating systems and 640 kB of RAM, I created a whole bunch of MS-DOS utilities and tools, mostly written in Turbo Pascal. In the last 20 years I have only ported a few of them. From time to time I miss one of these old tools - but never miss it enough to invest the time to write it from scratch. Last year I came across p2c, a Pascal to C translator by Dave Gillespie. Oh such joy - and I used it to port my old tools. One tool I used - which I had not created myself - was RLE8 by Shaun Case, Public Domain 1991.

art Moderne (licensed CC BY-NC-ND by Nadine)RLE8
RLE stands for run-length encoding compression. It is a form of data compression in which repeated values are represented by a count and a single instance of the value. It is a very simple form of compression and in the nineties it sometimes reduced disk and memory space significantly. It was used in early versions of Windows BMP for bitmap compression named BI_RLE8: An RGB format that used run-length encoding (RLE) compression for bitmaps with 8 bits per pixel. The compression used a 2-byte format consisting of a count byte followed by a byte containing a colour index. There were versions for 4 and 8 bit image data, RLE4 and RLE8 respectively. "RLE compression was used in the stone-age" as one forum comment reads and today there not much interest in it. The only reference I found was for benchmarking highly optimised code.

Finding the Original Source
Finding the original source code was difficult. It is in the nature of the modern WWW that new pages appear and old ones disappear. Fortunately the RLE8.EXE printed the name of its author and its license: Shaun Case 1991 Public Domain. After some googling I found an article about it, which later turned out to be the contents of the Readme someone had reposted almost ten years later on GameDev. Eventually I found the Retro Computing Archive with its collection of CD-ROMs containing shareware and Public Domain software from the late 80's and 90's. RLE8_SC. Win! (The Retro Computing Archive is great, many of its ZIP files are unpacked and therefore crawled by Google which helped me find it.)

Porting from Turbo C
The code compiled without issues, but included a header I did not have, dir.h, a header from Borland's Turbo C. I guess this is the biggest issue when porting old code - calls to non-standard library functions. I missed fnsplit, a function which split file specifications into component parts. While I could have created the function myself - an excellent opportunity to practice my C - I searched more and luckily someone had created it already. Thank you Robert B. Stout. Robert granted license to use his source files to create royalty-free programs, which RLE8 is. After adding some more of Robert's code, and removing code which was not needed, RLE8 compiled and linked, even with my pedantic settings of gcc -std=c99 -pedantic -pedantic-errors -Wall -Wextra. I loved it. Witness the power of C, a 50 years old programming language, still moving forward.

Download original and modified sources together with binaries for DOS and Windows x86.

3 September 2021

Baby Steps Push Challenge

Earlier this year I met with fellow developers Roland Germ and Bastien David to discuss how we could make coding exercises more engaging and how participants would have more fun. The training formats we use, like Coding Dojo and Coderetreat, were designed for people to have fun (and engage in deliberate practice in order to improve their skills). Ideas like gamification came to mind and we worked on several constraint games to spice up our coding sessions. (A constraint is an artificial challenge during an exercise to make the exercise more interesting, challenging or fun. I like constraints, e.g. my recent Dice Namer Constraint. In this article I will describe another fun constraint: the Baby Steps Push Challenge.)

At the same time I was working with a client who was adopting Continuous Delivery. One of the issues the developers faced was that they were used to work in larger increments. We had worked through several exercises which encouraged smaller steps, e.g. Taking Baby Steps (revert the code if tests are failing after 2 minutes), Test Commit Revert (revert the code if tests are failing at any time) and even Limbo (revert the code if integration with other code fails). I was looking for more exercises to encourage small steps.

Let Slip the Tugs Of WarIdea of Push Championship
Following my discussion with Roland and Bastien I merged some of their ideas into a "Push Championship" - a challenge who would be able to deliver smaller increments.
  • For this exercise I would need some basic TDD or Refactoring exercise.
  • Pairs would work on the same task on their own using version control with branches.
  • While dealing with merge conflicts is part of trunk based development, there should be no artificial branching problems. Participants just try to make more commits.
  • All commits must be green which is tested by CI - actually it is much easier to count the number of successful build actions on each branch.
  • The facilitator ("referee") needs to grade commits afterwards to avoid cheating the game and awards bonus points for good commit messages.
  • Elements of gamification like coins for each commit/push would be nice.
I created some infrastructure to support the exercise. First there is a central server to count pushes and display the dashboard of positive and negative events during such a "championship": Push Counter Server. It has "Push" in its name but it is more general and can be used to count any form of positive (counting up) or negative (counting down) event. The server application is published on Docker Hub. You can run it with docker run -p4567:4567 codecop/push-counter and the dashboard will be available at http://localhost:4567. You can deploy it to Heroku, which I did. (scripts/heroku_deploy does the job - you need to change the Heroku application name.)

Counting Pushes
The server dashboard displays the current game statistics of all teams - that is the number of positive events - and plays a coin sound (if you allow your browser to do so) when someone scores a point. To notify the server I execute curl at the end of a GitHub build action, e.g. curl -X GET ${{secrets.PUSH_COUNTER_URL}}/​record/team-name?​build=green. Whenever someone pushes to the repository, GitHub runs its build action and the dashboard gets updated.

Distinguishing Pairs
The easiest way to distinguish participant pairs is to use branches. In a scenario where each pair works the same exercise, this works well. I figure out the branch name with some shell and Git commands:
export GIT_BRANCH_NAME=$(git symbolic-ref --short HEAD)
export GIT_BRANCH_NAME=$( echo "$GIT_BRANCH_NAME" | sed 's/ /%20/g' )
curl -X GET ${{ secrets.PUSH_COUNTER_URL }}/record/branch-$GIT_BRANCH_NAME?build=green
The Push Counter Sample Client contains samples with several ways to identify pushes:
  • Using the branch name as shown above. This is dynamic and can be used for pairs working on branches.
  • Hardcoding the branch name in the GitHub action also needs dedicated branches.
  • Reading a team name from a file, i.e. using a lookup file on each branch is also possible.
  • When working trunk based, the GitHub user who pushed can be used (i.e. $GITHUB_ACTOR). This works if pairs do not switch machines.
  • Using a team name lookup by the GitHub user works also when pairs switch machines. Setup of the user name to team name mapping is necessary before the challenge.
  • Using the GitHub user of the last commit is also possible.
  • See the GitHub workflow file.
Test Run
When I tested the setup, I used - you guessed it - my favourite Prime Factors kata. The final score on the dashboard was:
Prime Factors score Choice of Kata
Because of the competition, all participants should be able to finish coding, i.e. it should be possible to finish the exercise in one hour (as part of a workshop or learning hour). Refactoring exercises are a good choice because the given test suite makes sure only working changes/ pushes contribute to the total score. Unfortunately most refactoring exercises are larger than one hour. Then the goal of the refactoring needs to be more specific.

Emily Bache's Parrot Refactoring Kata is a suitable exercise: It is small. (I am able to finish it on my own in less than 20 minutes. In fact - with some practice - I can run through its steps in less than 10 minutes.) And it has many steps, especially if you aim for baby steps. I counted 48 individual changes you can make - and I even missed a few. The concrete task here is to fix the code smell of switch on type code and clean up the code using as many individual, working refactoring steps as possible. Each commit should be delivered (pushed). And then you can repeat the exercise and try to break the score.

Other suitable refactoring exercises might be Top Gear and Promotion Service from my own list of code katas.

Coding Dojo Run
Later I ran the exercise in our local Coding Dojo community. I showed some slides to motivate small increments and gave some hints how to make smaller steps. I prepared a branch of Parrot which reported to my Heroku dashboard. All went as planned and here is some of the feedback I got from participants in the retrospective:
  • The kata (parrot) was the right size, we had plenty of time to work carefully.
  • We had to perform the same change three times - for the three sub-types - and used different approaches for each of them and compared them in the end. That was a bonus.
  • We thought about "How small is small enough?"
  • It is always possible to go smaller - you can commit more often.
  • The constraint encouraged us to take much smaller steps than usually.
  • Very small steps require lots of communication.
  • Very small steps can become confusing due to the pause in between steps.
  • I liked the dashboard and its icons.
  • It would be nice to sit in the same room with the dashboard on a big screen.
  • This notion of scoring works well in teams which have a good relationship.
Your Turn
Now it is your turn to work the exercise. You do not need the server infrastructure if you are on your own. Just clone the parrot and see how many small steps you can make. Then try again and improve your score. Some participants were able to make 52 meaningful, tiny changes to the Parrot using Java. Try to break this high score. Everything above 40 is good and above 50 is very good. If you want to use the exercise for a Coding Dojo, fork the prepared branch of Parrot and set the repository secret PUSH_COUNTER_URL to your dashboard. Have fun!
Final score after review

14 June 2021

The One Thing

Last year, during second lockdown, I watched a message of my guro Jakob where he proposed reading some books in the time when you are not able to practice in the dojo. He specifically recommended two books, The One Thing: The Surprisingly Simple Truth Behind Extraordinary Results and Atomic Habits: An Easy and Proven Way to Build Good Habits and Break Bad Ones. I had seen both books in my Goodreads feed and had not paid attention. But when your master asks, you better follow ;-)

FocusThe Book
The book is about extraordinary results determined by how narrow you can make your focus. Gary Keller insists that it is possible to reach any goal if you focus on it. And by focus he means that you really focus i.e. dedicate four hours of each day to your one goal. It has to be one single goal. And then, by the thousands of hours you have put into your goal, you will see results. Yes obviously, when following this approach, you will see results. In this sense, the One Thing is extraordinary and boring at the same time. I highly recommend reading it.

First I planned to summarise the book for you, and others have done a better job doing that. The book's homepage is a start and you can find many summaries of the book online, like this one on Four Minute Books. While summaries are short and save time, you will not get the benefit of the book without reading it. A book is not only about its content. Here the book is cementing its message by many times the author tells you the same thing in different words, a.k.a. repetition. The total time it takes to read the book is also time you have to reflect on the material.

Maybe my summary above is a bit lame, so let's discuss the book's impact on my life. And it had a profound impact as you will see. The first and utmost thing I remember from reading the book is the permission to drop stuff from my agenda. Like all developers I know, my daily plan was way too full: work, write a blog post like this one, release an episode of the Coderetreat Facilitation podcast, run a public Coding Dojo for the community, follow up on this cool idea I read about, finish reading SICP and get back playing with Scheme, create more exercises for workshops, keep up to date with Twitter and other feeds, work out, spend time with family, fix this next thing in the flat my wife is complaining about, fight climate change and much more. How was I supposed to allocate 4 (FOUR!) hours each day for the thing I want to do? It was impossible. Obviously Gary was crazy. ;-) Or I had to drop some tasks from my agenda. The idea of focus and dropping tasks was not new to me. I remembered J.B. Rainsberger mentioning a "Not Doing" column in his task list in 2014. I myself have put whole technologies on my blacklist - which I would ignore from this time on, but I have never dropped tasks and ideas so aggressively. I had several TODO lists of low priority where tasks went to die without being honest to myself.

The First Month
I spent the first month following the One Thing on a meta level because I did not work on anything but worked on the process itself. First I increased my focus: I deleted most icons - usually things to do or bookmarks to look at - from my desktop. I removed TODO markers in documents and on my hard drive. Then I reduced my commitments, e.g. I dropped the schedule to blog and record, stopped visiting meetups, reduced the number of learning hours and decided on the maximum number of hours I would work for each client per week. And I was OK with that. Letting go became a mental state and I had been following the Minimalists and Marie Kondo already before I started.

Suddenly I had more time - I even had some unscheduled time to play video games which I almost never did. In hindsight this was not extraordinary at all. If you drop most things you are working on, you will have more time available. It is a matter of focus. Lockdown and home office helped as I saved more than two hours every day. While some colleagues complained that they worked two hours more, I used these two hours - giving me half of my required four hours already. And I used the first month - 80 hours total - to practice the One Thing process. I started each day with a few minutes of mediation to improve my focus. I read more books about focus, e.g. Leo Babauta's book on focus and simplicity, forming habits, e.g. Atomic Habits as mentioned above and learning. I studied these books, taking notes and collecting them. I recommend all of them. I clean up my "office" and spent some time decluttering it, storing stuff in drawers or trashing it to further improve my focus during work.

One example of my achievements since I started using the One Thing methods are the books I have been able to read this year. The Pragmatic Programmers once wrote that you should read at least four technical books a year and the last year I was able to follow their rule was 2016. Since then I have been struggling to read at least one technical book a year. Reading became hard, I had problems focusing and I usually fell asleep after reading a few pages. I decided to add some reading time every day to my four hours of focused work, reading on the topic of focus of course. So including the One Thing, I have read ten books in five months.

Spending four hours on focused work each day reduced the time I had available for other things. After some struggle, I had more clarity - and stopped my self-deception. For example, if I see an interesting article linked on Twitter I either read it immediately, or I forget about it. There is no point in adding it to my read-later list, because there is no read later list because there is no extra time for that. I would not read it anyway, my read-later list kept growing and growing. While I was dreaming of doing so much in the past, I have become a naysayer now. For example "this is a good idea. I might try it. (One second later.) No, not really this is not my One thing and I have no time. End of idea." This is refreshing and freeing. Clarity is one of my needs.

Crunch Time
One drawback is that I am always in crunch time. As my time is more limited, I have to prioritize harder and postpone many tasks I know I need to do. This makes me feel uneasy at times. And for the things I still want to do - not as my One thing, but from time to time, e.g. maintaining my blog - I have to live with tiny increments. For example, I was working on this blog post 15 minutes at a time for more than a month. It was a bit frustrating. Still writing and "Public Relations" are not my priority.

Not having time to attend meetups caused some FOMO (fear of missing out) and made me miss the people of the community I had not seen in a while. In general being unable to do things which I liked mad me sad. I have been attending every one of our local, bi-monthly Coderetreats since its inception January 2019. When I was unable to attend for the first time in more than two years, I felt very, very sad. I felt a loss, almost like mourning. Maybe this was the first time in my life when I really let something go. It is easy to let go things you are not attached to.

Crunch Time Summary
Time for a very short summary (as my latest 15 minutes increment is almost spent): Read the book. Find your One Thing. Focus on it. It is possible to allocate four hours each day to focus on your highest priority, though it might be hard to stick to it. I am half a year into the process and did not miss a single day. Keep pushing!