Strong Opinions

As Code Cop I meet a lot of developers and I have to tell some of them that their code is crap and that they need to drive down their technical debt. Others need to test more, maybe start doing TDD. There are a few who would not listen. Some of them are even proud of stubbornly not listening, they say that they have strong opinions about the topic. They do not seem interested in in another view on their code or design? Why is that? I have some ideas.

Dunning Kruger Effect
The Dunning Kruger Effect is a cognitive bias in which people of low ability have illusory superiority and mistakenly assess their cognitive ability as greater than it is. In other words, beginners believe themselves senior and think that they know better. There is no need to listen to an outsider.

I know this effect myself: When I learnt my first programming language(s), as soon as I could write a few lines of code, I felt invincible. (I thought that) I could do everything. I ruled. I knew how to do it and I knew I was right and that there was no other way to do it properly. Today, after writing code for 20 years in more than 25 languages, I do not feel like that any more. Sometimes I would like to go back to my state of mind of the late nineties and I try to look at new languages like a child - with a beginner's mind - but I know too much details. I know there will be nasty details, even for that newest and hottest languages of today.

Expert Beginner
An expert beginner has quickly reached (what looks like) expert status. He or she voluntarily ceases to improve because of a belief that expert status has been reached and thus further improvement is not possible. (I recommend reading the whole series of Erik Dietrich, How Developers Stop Learning: Rise of the Expert Beginner and further posts of his series.) Expert beginners make up defending arguments because they are the experts.

I have met some of these. I vividly remember two senior developers who had been with one of my clients for more than 16 years. They were opposing me on everything I said. Sure they had a superior knowledge of the application they had built and maintained for so many years, and they knew the domain they were working in pretty well. I am sure they were adding value to the product. They both were strong influencers. As soon as they would speak their mind, all other team members would just agree. But, but, but... Sigh. They were still using Ant, did not know Maven, did not write unit tests, did not care for clean code and so on - they made me very unhappy.

Quadrants of Knowledge
The quadrants of knowledge seem connected to my previous points. The four quadrants are

1. Known Knowns: What you know that you know
2. Unknown Knowns: What you do not know that you know
3. Known Unknowns: What you know that you do not know
4. Unknown Unknowns: What you do not know that you do not know

When we advance our career in software delivery, we learn: We collect concrete knowledge (quadrant 1), we gain experience which allows us to have gut feelings about some things (maybe quadrant 2) and we hear about things that we have no idea about (quadrant 3). There are gazillion things in software that I have no idea of: ANTLR, APL, AWS, Category Theory, Distributed Computing, Elm, Gradle, Guava, Haskell, HBase, J2EE, Kubernetes, Mongo DB, Networking, Node.JS, Prolog, Security - these are just the first few that come to my mind. It is easier to list things which I do not know in areas that I do know. I have been Java developer for 15 years and there are so many things in the Java ecosystem which I have no clue about. On the other hand there are not many things I can say about C#, because I only know it little.

So the more we learn, the more we know that we do not know, increasing our Known Unknowns (quadrant 3). The philosopher Confucius (551–479 BC) even said, Real knowledge is to know the extent of one's ignorance. Like the Dunning Kruger Effect, beginners have no idea about the vast size of Known Unknowns quadrant and do not expect surprises or potential improvements.

Strong Opinions, Weakly Held
The ideas of the Dunning Kruger Effect, Expert Beginners and Known Unknowns offer some explanation why some junior developers are not open for discussions. I have also met senior and expert level developers who are the same. When attending Software Crafting unconferences, e.g. SoCraTes, attendees are eager to learn and open for discussion. After all, that is the idea of unconferences, right? Still, many of them have strong opinions about almost everything, e.g. Spring Boot is cool. No, Spring Boot is hell. ;-) Different from beginners, all of them know the problems of strong opinions and claim that they only hold onto them weakly.

Weakly Held, what is that supposed to mean? Isaac Morehouse explains in These Four Words Will Help You ‘Hold Strong Opinions Weakly': You act as if they [your opinions] are true unless and until it is proven they are not. Maybe people like strong opinions because they give them the power of definiteness. Choices are much easier and life is more predictable with definiteness and absolutes. Holding opinions weakly also gives the power of openness. This sounds hard: Basing one's actions on some definite "truths" and re-evaluating these truths whenever they are challenged. Seems like a difficult balancing act, if not a contradiction. For sure that state of mind is not easy to get into. I still need to see someone with strong opinion - weakly held or not - to change his or her mind,

Rant
When I started this article, it was supposed to be a rant. As usual, writing down my thoughts helped me to structure the material in a better way. (I consider writing an act of learning.) So where is the rant? Here it is:

Person with strong opinion == arsehole who does not want to listen.

That is a bit harsh, I agree. Obviously beginners need to listen more and due to Known Unknowns I expect experienced and expert developers to be open to discussion on any topic at any time. The strong opinion itself is not the problem, but the resulting behaviour, especially if strong opinion is used as warning or justification of some sort. I witness strong opinion mainly as excuse to not discuss or challenge existing views.

What about me?
Of course I had strong opinions during my professional life. I did not earn the nick name of Code Cop for nothing. Someone even called me Code Nazi. And my opinions were definitely not weakly held. I like absolutes. (Now I see - as I just learned above - the power of definiteness.) I guess I still have some strong opinions, and they are not just opinions, they are dogma: a system of principles proclaimed as unquestionable.

I have changed and it seems that the Code Cop is getting soft. ;-) The extended experience and discussions have weakened my definiteness. There is always some special case, the almighty it depends. And since I am working with people, and try to help them, I am more compassionate. I understand their problems, the difficulty to get time for quality and the pain of legacy code. Sometimes I catch myself not recommending what I believe the technically best course of action because it will be a lot of unpleasant, hard work for the team. No, that is too soft. If they made a mess, they have to fix it. Time to fall back on dogma. (Sound of firing cannons in the background ;-) All the time these balancing acts...

Only modified files in Jenkins

For a custom Jenkins build I needed to know all changed files since the last green build. I searched a lot and found a solution as combination of several StackOverflow answers. It took me some experimenting to get it working: I installed the Groovy plugin, configured the Groovy language and created a script which executed as system Groovy script. Here is the complete step by step guide for Jenkins 2.63, SVN and Groovy 2.4.11.

Jenkins only provides the current revision in the environment variable $SVN_REVISION. Of course Jenkins knows the information about changed files of each build as it is shown in the build status page. I guess a plugin would be able to access the model, but that is too much work. Fortunately the Jenkins Groovy plugin allows scripts to run under the system context having access to hudson.model.Build and other classes.

Groovy Programming Language
The Groovy programming language is a dynamic language which runs on the JVM. It integrates smoothly with any Java program and is the first choice for scripting Java applications. While not strictly necessary I recommend downloading the SDK's zip and unpacking it on the host where you run Jenkins, usually into the folder where you keep your development tools. For testing and debugging I also install it on my local workstation (in the same location).

Groovy in Jenkins
Next comes the Jenkins Groovy plugin. Open Jenkins in the browser and navigate the menus:

• Manage Jenkins
• Manage Plugins
• select tab Available
• filter "groovy"
• select Groovy
• Install

(You will need Jenkins admin rights to do so.) Then tell Jenkins which Groovy to use and where to find it. To configure the Groovy language go to

• Manage Jenkins
• Global Tool Configuration
• go to section Groovy
• Add Groovy: Give it a name and set GROOVY_HOME to the folder you unpacked it, e.g. /tools/groovy-2.4.11.
• deselect Install automatically
• Save

Now Jenkins supports Groovy scripts.

Run a Groovy script in the build
Now let's use a Groovy script in the project. On the project page,

• Configure
• go to section Build
• Add build step
• select Execute system Groovy script
• paste Groovy code into the script console
• Save

Now when you trigger the build, the script will be executed.

Debugging the Script
Of course it does not work. How can I debug this? Can I print something to the console? Groovy's println "Hello" does not show up in the build log. Searching again, finally the gist by lyuboraykov shows how to print to the console in system scripts: Jenkins provides the build console as out variable,

out = getBinding().getVariables()['out']

which can be used like out.println "Hello". Much better, now I can debug. Let's wrap the out.println in a def log(msg) method for later.

Getting the changed files of the current build
StackOverflow answer by ChrLipp shows how to get the changed files of the current build:

def changedFilesIn(Build build) {
  build.getChangeSet().
    getItems().
    collect { logEntry -> logEntry.paths }.
    flatten().
    collect { path -> path.path }
}

This gets the change set hudson.scm.ChangeLogSet<LogEntry> from the build, gets the SubversionChangeLogSet.LogEntrys from it and collects all the paths in these entries - this is the list of all file paths of all changed items in all commits (LogEntrys). I guess when another SCM provider is used, another type of ChangeLogSet.LogEntry will be returned, but I did not test that. To better understand what is going on, I added explicit types in the final Groovy script, which will only work for Subversion projects.

Getting all builds since the last successful one
I want all changed files from all builds since the last green one because they might not have been processed in previous, failed builds. Again StackOverflow, answer by CaptRespect comes to the rescue:

def changedFileSinceLastSuccessfull(Build build) {
  if (build == null || build.result == Result.SUCCESS) {
    []
  } else {
    changedFilesIn(build) +
      changedFileSinceLastSuccessfull(build.getPreviousBuild())
  }
}

In case there is no previous build or it was successful the recursion stops, otherwise we collect changed files of this build and recurse into the past.

All Together
Let's put it all together,

def changedFiles() {
  def Build build = Thread.currentThread()?.executable
  changedFileSinceLastSuccessfull(build).
    unique().
    sort()
}

After collecting all duplicates are removed, as I do not care if a file was changed once or more times, and the list is sorted. In the end the list of changed files is saved as text changed_files.log into the workspace. (The complete jenkins_list_changed_files.groovy script is inside the zipped source.)

While developing the script, the Jenkins script console was very handy. As soon as the script worked, I created a the file jenkins_list_changed_files.groovy, put that under version control and changed the build definition step to use the script's file name. Next time the build ran, the script file would be executed, or at least so I thought.

Script Approvals
Unfortunately system Groovy script files do not work as expected because Jenkins runs them in a sandbox. Scripts need certain approvals, see StackOverflow answer by Maarten Kieft. To approve a script's access to sensitive fields or methods navigate to

• Manage Jenkins
• In-process Script Approval (This is the one but last item in the list.)
• Approve

The sandbox is very restrictive, the full jenkins_list_changed_files needs a lot of approvals:

field hudson.model.Executor executable
method groovy.lang.Binding getVariables
method hudson.model.AbstractBuild getChangeSet
method hudson.model.AbstractBuild getWorkspace
method hudson.model.Run getNumber
method hudson.model.Run getPreviousBuild
method hudson.model.Run getResult
method hudson.scm.SubversionChangeLogSet$LogEntry getPaths
method java.io.PrintStream println java.lang.String
new java.io.File java.lang.String
staticMethod java.lang.Thread currentThread
staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods flatten java.util.List
staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods println java.lang.Object java.lang.Object
staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods sort java.util.Collection
staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods withWriter java.io.File groovy.lang.Closure

Creating a new java.io.File might be a security risk, but even println is not allowed. Adding all these approvals is a boring process. The build breaks on each missing one until everything is well. As soon as you have all the approvals, you can copy Jenkins' scriptApproval.xml found in JENKINS_HOME (e.g. ~/.jenkins) and store it for later installations. The full scriptApproval.xml is inside the zipped source.

Conclusion
Jenkins' Groovy integration is very powerful. System scripts have access to Jenkins' internal model which allows them to query information about build, status, changed files etc. On the other hand, development and debugging is cumbersome and time consuming. IDE support helps a lot. Fortunately StackOverflow knows all the answers! ;-)

Three Rules of Coaching Engagement

Last year software crafter and Kotlin enthusiast Oleksii Fedorov asked me about technical coaching. He was going to work with a small team and focus on technical enablement. He would do so by pair and mob programming with them for some time and later support them remotely if needed. During that time there would be no pressure or deadline to deliver new features so they could focus on improving.

Oleksii is a like-minded peer. I met him on several occasions in the past. He is fun to work with and considers teaching an integral part of pair programming. I did not expect any problems. He kept pushing me and made me think about my principles. In the end I came up with three points I like to make clear right from the beginning, i.e. set the ground rules of the engagement.

1. Manage expectations
Managing expectations is crucial. I learnt that the hard way during my Pair Programming Tour. Whenever I run a session, e.g. pair or mob programming and often also prepared workshops, I start with a discussion to collect expectations. For longer contracts I recommend to start with a dedicated meeting to do so. Usually expectations are fuzzy, like "I want to learn something", so keep pushing with questions like "How will you know it was worth it? How will you measure that you learned something?" and so on. In the end I want the team to come up with ways to measure their learning and define their own success metrics.

2. Learning is their responsibility
Oleksii planned for two weeks. In two weeks people will not change (much). Such a coaching engagement does not last long enough for much impact, rather it is a start of a (maybe even life long) journey of learning and improving. This is true from day one and during the initial meeting I make it clear that I am not a teacher. I will not push knowledge onto them to be forgotten the next day. To get something from working together, people have to participate, to engage, to invest time and effort into the topic. They will only get so much from it what they have put into it themselves. Learning is their responsibility.

3. Treat them as strangers
After removing the responsibility to teach anything - because learning is their responsibility - I feel less stressed. As coach I am still responsible for creating a safe space, providing content, managing exercises and a lot of other things. If someone does not want to participate or feels like doing something else, I have to be OK with that. Even being paid for "teaching" should not change that. My advice here is to treat the team members as "strangers", like we treat new visitors of a Coding Dojo. As a facilitator I try to help, but I will not oppose or coerce.

Feedback
Oleksii started the job soon afterwards. I was sure everything would work out fine. When we met again he told me that my three rules had helped him. 1. He collected expectations in the beginning and managed them while moving forward. 2. People understood that they had to invest and some participants even showed up for sessions during their free time. 3. People where entering and leaving sessions all the time, still he stayed relaxed.