IDE Support for Scripts

Last month I wrote about dealing with modified files during build in Jenkins. The solution uses Groovy scripts and the Jenkins Groovy plugin which allows execution of scripts in Jenkins' system context. These System Scripts have access to Jenkins' internal model like build status, changed files and so on. The model includes several classes which need to be navigated.

In many situations little additional scripts make our lives as developers easier: For example the Groovy System Script to customise Jenkins when building a NATURAL code base mentioned above, or a Groovy Script Transformation to perform repeated code changes in a large Java project using WalkMod, or a custom build during a Maven build, using any language supported by the Apache Bean Scripting Framework, e.g. Groovy, Ruby, Python or others. Now if the scripts are getting complicated IDE support would be nice.

Getting support for the language in the IDE
Depending on the language, most IDEs need installation of a plugin or extension. I will briefly describe the steps to set up Groovy support into Eclipse because it is more elaborate and the organisation I initially wrote this documentation used Eclipse. Getting plugins for IntelliJ IDEA is straight forward.

• First obtain the version of Eclipse you are using. Customised distributions like the Spring Tool Suite, IBM/Rational products or NaturalONE follow different version schemes than Eclipse. Navigate to the Help menu, item About, button Installation Details, tab Features, line Eclipse Platform to see the real version. For example, NaturalONE 8.3.2 is based on Eclipse 4.4.
• The Groovy plugin for Eclipse is GrEclipse. For each major version of Eclipse there is a matching version of GrEclipse. The section Releases lists the Update Site for each release. Find the suitable version. (If you use a newer Eclipse, maybe you have to use a snapshot version instead of a release version.) Copy the URL of the Update Site.
• The section How to Install explains in detail how to continue and install GrEclipse.

After that Eclipse can talk Groovy.

Enabling support for the language in the project
Next we configure the project for the additional language. In Eclipse this is often possible through the (right click) context menu of the project, menu item Configure. I usually do not bother and edit Eclipse's .project file directly to add the needed natures and build commands. Adding Groovy to a non-Java project needs adding to .project,

<projectDescription>
  <!-- existing configuration -->
  <buildSpec>
    <!-- existing configuration -->
    <buildCommand>
      <name>org.eclipse.jdt.core.javabuilder</name>
      <arguments>
      </arguments>
    </buildCommand>
  </buildSpec>
  <natures>
    <!-- existing configuration -->
    <nature>org.eclipse.jdt.groovy.core.groovyNature</nature>
    <nature>org.eclipse.jdt.core.javanature</nature>
  </natures>
</projectDescription>

and .classpath

<classpath>
  <!-- existing configuration -->
  <classpathentry exported="true" kind="con" path="GROOVY_SUPPORT"/>
  <classpathentry exported="true" kind="con" path="GROOVY_DSL_SUPPORT"/>
</classpath>

Similar changes are needed for Ruby (using org.eclipse.​dltk.core.​scriptbuilder and org.eclipse.​dltk.ruby.​core.nature) and Python (org.python.​pydev.PyDevBuilder and org.python.​pydev.pythonNature). If I am not sure, I create a new project for the target language and merge the .projects manually. Sometimes other files like .classpath, .buildpath or .pydevproject have to be copied, too.

Enabling support for libraries
Adding a nature to .project gives general support like syntax highlighting and refactoring for the respective language but there is no completion of library classes. Especially when using new libraries, like Jenkins Core, using code completion helps exploring the new API because you get a list of all possible methods to call. The trick is to add the needed dependency to the project in a way that it is not accessible from or packaged together with the production code.

An Example: Adding Jenkins System Script support to a NaturalONE project
To add dependencies to the NaturalONE project earlier, I converted the Java/Groovy project to Maven (by setting another nature and builder) and added the needed classes as fake dependencies to the pom.xml.

<dependencies>
  <!-- other dependencies -->
  <dependency>
    <groupId>org.jenkins-ci.main</groupId>
    <artifactId>jenkins-core</artifactId>
    <version>2.58</version>
    <scope>test</scope> <!-- (1) -->
  </dependency>
  <dependency>
    <groupId>org.jenkins-ci.plugins</groupId>
    <artifactId>subversion</artifactId>
    <version>2.7.2</version>
    <scope>test</scope>
  </dependency>
</dependencies>

Groovy has optional typing and after adding the types to the definitions and arguments, Eclipse will pick up the type and we get code completion on available fields and methods:

import hudson.model.Build
def Build build = Thread.currentThread()?.executable
build._ // <-- code completion, yeah!

This made developing and understanding the script to find modified files much easier. The Jenkins dependencies in the pom.xml are never used, as the System Script runs inside Jenkins where the classes are provided. The dependencies are declared to "trick" Eclipse into analysing them and providing type information. The scope of the dependencies is set to test (see line (1) in code snippet above) so the dependencies are not packaged and cannot be called from the production code. This zipped repository contains the NATURAL project together with all the Eclipse configuration files.

Another Example: Adding WalkMod Transformation support to a Maven project
Another situation where I wish for script support in Eclipse is when writing WalkMod Script Transformations. WalkMod is an open source Java tool that can be used to apply code conventions automatically. Read this tutorial by Raquel Pau to see how WalkMod works. WalkMod allows for Groovy scripts to define code transformations which manipulate the AST of Java classes. Navigating the AST is difficult in the beginning.

After adding the Groovy nature as shown in the previous example, the relevant dependencies to get code completion for the AST are

<dependencies>
  <dependency>
    <!-- API -->
    <groupId>org.walkmod</groupId>
    <artifactId>walkmod-core</artifactId>
    <version>3.0.4</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <!-- AST -->
    <groupId>org.walkmod</groupId>
    <artifactId>javalang</artifactId>
    <version>4.8.8</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>com.squareup</groupId>
    <artifactId>javapoet</artifactId>
    <version>1.10.0</version>
    <scope>test</scope>
  </dependency>
</dependencies>

In case of Walkmod transformations, the interesting class is (org.walkmod.​javalang.ast.​CompilationUnit) and it is already in scope as variable node. To add type information we need to alias it:

import org.walkmod.javalang.ast.CompilationUnit
def CompilationUnit cu = node
cu._ // <-- code completion on available fields and methods

Conclusion
Supporting scripts in Groovy, Ruby or Python make our lives easier. With some additional extensions or plugins and some tweaks in the project configuration our IDEs will support the scripting language. Why should we accept code editing like in the early 1990s?

Work Harder

With this article I want to prompt you to work hard(er). What does working hard mean? When I looked for different translations of German Strengt Euch an, I found several ones, which are all suitable: Work hard. Keep it tight. Push yourselves. Put some heart into it. Put some muscle in it. Put your backs to it. Make your best effort. Just play it like you fucking mean it. Obviously hard work needs your strength. It also needs your heart - motivation and dedication. It might test your limits, both physical and psychical ones.

My favourite example of hard work was writing books in the Dark Ages. Monks were copying books by hand, adding drawings and decorations as they went. An extreme example is the Codex Gigas, which was handwritten by a single, anonymous monk. Because the scribe was a monk he may only have been able to work for about three hours a day, and this means that the manuscript including decoration probably took at least 20 years to finish, and could even have taken 30. Now that is a whole lifetime for collecting and handwriting a single book. Imagine the dedication.

Maybe less extreme is the life of master craftspeople. Several years ago I wrote about a master craftsman hand-crafting rakes. He could retire any time, still chose to improve his methods and work hard all day.

Benefits of hard work
Most of you will agree, that hard work pays off. Here is a little TED talk by Richard St. John, who explains why it is so: Hard work is the real secret to success. I vividly remember a professor of Mathematical Analysis during my studies: At the beginning of a lecture he wrote a theorem on the blackboard, followed by an easy proof. The proof looked good and we students understood what was going on. Then he said "Was man leicht kriegt ist nichts wert" ("What you get easily is worth nothing"), pointed out a mistake in the proof and erased it. He used the remaining of the hour to sketch a valid proof which was complicated and much harder to follow.

Hard work has more value
Theodore Roosevelt even said that "Nothing in the world is worth having or worth doing unless it means effort, pain, difficulty..." Maybe he was exaggerating, but a little pain never hurts. (Pun intended ;-) So is the motto in weightlifting training: No Pain, No Gain. Psychologist Katarina Veselko mentioned the topic in one of her talks. During the following conversation she explained: It not so much that we do not value things that are easy to get, it is more about the fact that when we need to invest a lot of energy, time, money, ... into achieving something, the result will be more valuable for us because we have invested a lot into it. It is some kind of Cognitive dissonance. She also believes that most things worth having are not easy to get; most things that are valuable to us are not achieved in a way that is always fun, fast and easy, but also includes obstacles and challenges.

Instant Gratification
The term Instant Gratification is often used to label the satisfactions gained by more impulsive behaviours: choosing now over tomorrow. And today it is harder to delay gratification than it used to be. We have been trained by technology and social media to expect results fast, without much effort. For example, there are more than 2 million hits on Google how to earn 5k extra, many of then offering "work from home, 30 minutes daily" up to "basically doing nothing". The same is true for losing weight without any diet and so on. If you want to read more on Instant Gratification and its problems, I recommend Courtney Ackerman's Definition with Examples.

Conclusion
It seems that working hard has become unpopular. Why work towards a long term goal when many things are handed over on a plate, or downloaded at the click of a button, or ours in twenty-four hours for just £9.99 extra? I do not think that life works like that. Why do I come up with this in my blog? I am writing this because I think we Craftspeople need to work harder. I am not talking about working more hours. In fact working overtime is against the idea of Craftsmanship because we need our free time to reflect, exchange and learn.

I am talking about pushing ourselves more: Following up on that refactoring you did not finish last week. Refactoring mercilessly, always pushing to keep the whole code base clean and consistent, even during architectural changes. I am talking about not accepting lesser standards because of low level, different or even obsolete technologies or environments. I am talking about overcoming pointless bureaucracy or managers who have no idea about code quality.


I know it is hard. Years of arguments with colleagues, fighting superiors, struggling to grow and working broken processes have taken their toll. Sometimes I feel old. But I have to defy them. Let us continue pushing, trying to work harder and make an effort!