19 December 2011

JavaClass 0.4 released

Much has happened since I created JavaClass almost three years ago. I started with version 0.0.1 because much was missing in there. Since then I managed a small release once a year. Version two added class names and references. Last year I added classpath abstractions and moved the project from Rubyforge to Google Code (and then in the future to Bitbucket and eventually to GitHub). This year I finished version 0.0.4. Now the Java class file parser supports everything except fields and methods. Adding them would be no big deal, I just had no need to do it till now. This fourth release feels more complete and should rather be named version 0.4 than 0.0.4. So the next version will be 0.5.

Coffee BeansNew in JavaClass 0.4
I added support for Java 5 Enums and Annotations to JavaClass::ClassFile::JavaClassHeader and write code to retrieve all interfaces implemented by a class. Further I created Maven- and Eclipse-aware classpath abstractions. If you point JavaClass to the folder of a whole Eclipse workspace, you get a JavaClass::Classpath::CompositeClasspath containing all projects found in that workspace. Finally I fixed some defects, e.g. a character to byte conversion problem found in Ruby 1.9 and a printf inconsistency between Windows and Linux platforms. See the history of JavaClass 0.4 for the complete list of changes.

RDOC documentation was improved and several examples have been added. In fact these examples are real world scripts that I use to analyse code bases. They are just formatted nicely and commented in very detail.

Find Unused Classes
For example let us look for unused classes. By "unused" I mean classes that have no incoming reference inside a code base. Such classes might still be referenced by Java Reflection, but more likely are just dead code. I have seen legacy code bases where more than 10% of all classes were unused and their deletion was a welcomed reduction of the maintenance burden. A list of classes that can (potentially) be deleted is a valuable asset for the next clean-up cycle. Still, each class on the list needs to be double checked because classes with main methods, Servlets and other framework classes might show up on the list but still be used.
require 'javaclass/dsl/mixin'
require 'javaclass/classpath/tracking_classpath'

location = 'C:\Eclipse\workspace'
package = 'com.biz.app'

# 1) create the (tracking) composite classpath of the given workspace.
cp = workspace(location)

# Create a filter to limit all operations to the classes of our application.
filter = Proc.new { |clazz| clazz.same_or_subpackage_of?(package) }

# 2) load all classes of the application, this can take several minutes.
classes = cp.values(&filter)

# 3) for all classes, mark all referenced types as accessed.
cp.reset_access
classes.map { |clazz| clazz.imported_types }.flatten.
  find_all(&filter).
  each { |classname| cp.mark_accessed(classname) }

# 4) also mark classes referenced from config files.
scan_config_for_3rd_party_class_names(location).
  find_all(&filter).
  each { |classname| cp.mark_accessed(classname) }

# 5) find non accessed classes.
unused_classes = classes.find_all { |clazz| cp.accessed(clazz) == 0 }
The above example uses EclipseClasspath and TrackingClasspath under the hood. TrackingClasspath adds functionality to record the number of usages of each class. The classes that are not used by any imports (step 3) or any hard coded configurations (step 4) are potentially unused.

Java Class Literals in Ruby
Inspired by date literals in Ruby I thought about Java class name literals in Ruby. This feels a bit crazy so it is not enabled by default. After including JavaClass::Dsl::JavaNameFactory you can write
clazz = java.lang.String    # => "java.lang.String"
Note the absence of any quotes around the right hand side of the above assignment. The returned class name is not just a plain String, but also a JavaQualifiedName with some extra methods to work on Java names, like
clazz.to_java_file          # => "java/lang/String.java"
clazz.full_name             # => java.lang.String"
clazz.package               # => "java.lang"
clazz.simple_name           # => "String"
Adding these rich strings caused me a lot of trouble, but that is another story.

15 November 2011

Visualising Architecture: Eclipse Plug-in Dependencies

In my job at the Blue Company I am working on a product family of large Eclipse RCP applications. Each application consists of a number of OSGi plug-ins which are the main building block of Eclipse RCP. There are dedicated as well as shared plug-ins. I am new to the project and struggle to understand which applications depend on which plug-ins. I would like to see a component diagram, showing the dependencies between modules, i.e. on OSGi level.

There is an Eclipse's Dependency Visualization tool inside the Eclipse Incubator project: Plug-in org.​eclipse.​pde.​visualization.​dependency_​1.0.0.20110531 which displays dependencies of plug-ins. This is exactly what I need. Here is the visualization of one of the products:

Plug-in Dependency Analysis
(The yellow and green bounding boxes were added manually and indicate parts of the architecture.) The tool's main use is to identify unresolved plug-ins, a common problem in large applications. While you find unresolved plug-ins by running the application, the tool helps even before the application is launched. For more information see this article on developerWorks about the use of the Dependency Visualization. The latest version of the plug-in, including its download for Eclipse 3.5.x, is available at the testdrivenguy's blog.

11 November 2011

My Buddy Check List

Job InterviewLast DemoCamp a fellow craftsman asked me how to find a proper employer to work for. How to find out if a team you are going to join is kicking ass? How to check if the future team mates are good people? (By our definition a good person cares about his or her work, doesn't rush, writes unit tests, keeps the code clean and does many other nice things. :-) So how to find out if you want to work with them? These are good questions.

Mindset
Applying for a job is symmetric which means it's a process working in both directions. Of course you try to prove yourself worthy of the new job. There are many resources on the internet that tell you how to write your CV and how to behave during interviews, so you would not screw up. There are even guidelines for managers which questions to ask including the famous FizzBuzz. All these guides are targeting your value as employee and how to present it properly.

On the other hand your future employer has to show to you that he is worthy, too. The interviewer is his first representative and must be prepared well and take the interview serious. Later he or she might tell you about the company's generous compensation scheme or the team or the development process to impress you. In the end of the interview you are usually asked if you have any questions. This is the time to ask and so I do. During the past years I have compiled a list of questions that I ask during interviews. Today I want to share some of them.

Process
I always start by asking the Joel Test. Although it failed me in the past, I still believe it gives a good overview of the used development process. Then I ask the interviewer to define quality. Quality is perceived different by different people and the manager's idea of quality has considerable influence. Usually this leads to an interesting discussion if the manager is really interested in you.

Team Spirit, December 2006Team "Theory"
The team I'm going to work with is important to me and I start asking about it right away. "How large is it? How is it structured? When was it formed?" A team needs time to form and I like to join high performance teams. So is it a real team where each team member found its place or is it just a group of individuals sharing the same room? Starting a gig with setting up a new team is risky if you lack authority to influence future hiring decisions. The team might not end as you would like it to be.

Team Spirit
Next I try to figure out the team spirit. "When did the last person leave the team and why? Why is there an open position? Are there any contractors on the team?" A high percentage of contractors is a bad sign. First contractors do not fully participate in teams as their engagement feels more temporarily. Second most contractors I know either have no idea about coding or are extremely prolific but then their code looks like hell. (But I'm sure this is a coincidence.)

How Does it Feel?
If all things are going well you might be invited to meet the team, which in some cases has the final word in the hiring decision. And even if not, a tour of the office never hurts. I always try to meet my future team mates and spend at least half of a day with them. How does it feel spending time with those people? Are they in a good mood or is there a dark cloud of fatalism hanging above them? Are they friendly and open minded? How do they communicate? You will spend a lot of time with them, so you better like them.

Anonymous at ROFLCon What Do You Know?
During this time I "interview" my future colleagues. Well, it's not a real interview, more a chat to figure out what makes them special. My questions just guide me during these conversations. And I would also tell them things about myself. Most likely they know better than me if I fit into the open position and the team or not. To get started I'm looking for someone who knows more than me. In our technical domain it's very easy to know more in some area than most developers. "So what's your background? How long have you been developing software? What were your most exciting projects? Was it cool stuff? Were you excited about them?" (Did you care?)

As these people are developers, I get more technical and ask them about their favourite Java framework or library. I like language fundamentals so Apache Commons fit me perfectly, but that's just me. Everyone has his or her favourite toys. Think about them, discuss them. Are they esoteric, mainstream, fundamental, boring? On the other hand the similar question, which is the first library to add to a new project, has only one right answer. It's not Apache Commons and it's certainly not Spring or Hibernate, but it's JUnit. Even if you do not develop test driven, you will need it sooner or later. If we happen to talk about design I always ask "What's your favourite design pattern?" I hope it's not singleton, because singletons are evil, all other answers usually lead to a good discussion about object orientation. After talking for some about what he knows and what I know I sum it up with my last question of this section, "What will I learn from you?"

Do You Care?
Next I'm looking for individuals that love to code, that care for their craft and are burning with enthusiasm. Asking about hobbies and how one spends his or her personal time doesn't sound related at first, but it is. The real question is if he or she does code on personal time, maybe is even an open source contributor. For example, some time ago I met an older guy and he didn't seem very enthusiastic. The whole team had impressed me so far and I was sure that he would just be a nine to five employee. I started asking him about his duties on the project but quickly moved to the area of personal time. It turned out he was into wireless network topology and played around with wireless infrastructure at home. He wasn't coding but still fooling around with technology. I was impressed.

Hard Work Can HurtFurther I want to know "Which blogs do you read?" If you are interested in your craft you have to stay in touch, play around with new stuff and read books. The pragmatic programmers recommend reading four technical books each year. So "What was the last technical book you read?" Talking of books, "Who is Donald E. Knuth?"

What About Quality?
Ultimately I'm looking for software craftsmen and I wouldn't be genuine without diving into code quality and clean code. Everybody with just a faint interest in code quality, object orientation or self improvement knows Robert C. Martin, so I always ask if they know Uncle Bob. Finally I try to determine if I will hate this person every time he or she commits some changes to the repository: "What is most important about code? What is the worst defect for code? What is clean code for you?"

Warning
Let's finish with a word of caution: These questions help me figuring out if a future job is good for me but it's not foolproof. It happened that everything looked great and still the real job experience was awful. Also be aware that these questions result from my personal experiences. They need not to be good for you. Don't blame me if you end up in a group of coders from hell.

22 October 2011

Freeze Custom Ruby Strings When Used as Keys in Hash

Last week I spent quite some time chasing a single issue in my JavaClass Ruby gem. It really annoyed me and I could not find anything useful even using Google. I had to dig deep. Read what happened: I began with some kind of rich string, quite similar to the following class:
class RichString < String
  def initialize(string)
    super(string)
    @data = string[0..0] # some manipulation here
  end
  def data
    @data
  end
end

word = RichString.new('word')
puts word               # => word
puts word.data          # => w
That was not special and worked as expected.

Lost ... !!Then I happened to use instances of RichString as keys in a hash. Why shouldn't I? They were still normal Strings and their data should be ignored when used in the hash.
map = {}
map[word] = :anything

word_key = map.keys[0]
puts word_key           # => word
puts word_key.data      # => nil
The last line warned me "instance variable @data not initialized". Oops, my little @data went missing indicated by the bold nil in the last line. First I did not know what was causing the problems. I was baffled as all tests were green and had a good coverage. I spent some time digging and rewriting a lot of functionality until I found that Hash#keys() caused the trouble when given my RichStrings as hash keys.
puts word == word_key   # => true
puts word.object_id == word_key.object_id  # => false
Aha, Hash changed the keys. It's reasonable to prohibit key changes, so a String passed as a key will be duplicated and frozen. (RTFM always helps ;-) But how did it do that? It did not call dup() on the RichString. As Hash is natively implemented, I ended up in the C source hash.c.
/*
*  call-seq:
*     hsh[key] = value        => value
*     hsh.store(key, value)   => value
*/

VALUE
rb_hash_aset(hash, key, val)
  VALUE hash, key, val;
{
  rb_hash_modify(hash);
  if (TYPE(key) != T_STRING || st_lookup(RHASH(hash)->tbl, key, 0)) {
    st_insert(RHASH(hash)->tbl, key, val);
  }
  else {
    st_add_direct(RHASH(hash)->tbl, rb_str_new4(key), val);
  }
  return val;
}
So when the key is a String and not already included in the hash, then rb_str_new4 is called. (I just love descriptive names ;-) Furthermore string.c revealed some fiddling with the original key.
VALUE
rb_str_new4(orig)
  VALUE orig;
{
  VALUE klass, str;

  if (OBJ_FROZEN(orig)) return orig;
  klass = rb_obj_class(orig);
  if (FL_TEST(orig, ELTS_SHARED) &&
      (str = RSTRING(orig)->aux.shared) &&
      klass == RBASIC(str)->klass) {
    long ofs;
    ofs = RSTRING(str)->len - RSTRING(orig)->len;
    if ((ofs > 0) || (!OBJ_TAINTED(str) && OBJ_TAINTED(orig))) {
      str = str_new3(klass, str);
      RSTRING(str)->ptr += ofs;
      RSTRING(str)->len -= ofs;
    }
  }
  else if (FL_TEST(orig, STR_ASSOC)) {
    str = str_new(klass, RSTRING(orig)->ptr, RSTRING(orig)->len);
  }
  else {
    str = str_new4(klass, orig);
  }
  OBJ_INFECT(str, orig);
  OBJ_FREEZE(str);
  return str;
}
Frozen StringI didn't quite understand what was going on in rb_str_new4(), but it was sufficient to read a few lines: If the original string was frozen, then it was used directly. I verified that.
map = {}
map[word.freeze] = :anything

word_key = map.keys[0]
puts word_key           # => word
puts word_key.data      # => w
Excellent, finally my @data showed up as expected. Fixing the problem added some complexity dealing with frozen values, but it worked.

Freeze your custom Ruby strings when you use them as keys in a hash (and want to retrieve them with Hash#keys())

19 October 2011

Awesome Book Marks

Since the beginning of code-cop.org I have put strong emphasis on my personal branding. Till now I have created various t-shirts, business cards and buttons. I use these buttons to award conference speakers who delivered good presentations and to thank contributors who helped during Hackergarten. But now my mother-in-law excelled all of them. See my new hand embroidered book marks:

Awesome code-cop.org Book Marks
These are some awesome masterpieces! Thank you Lidia.

13 September 2011

Getting Started with Hackergarten

Hackergarten is a computer programming contributor group. Read more about Hackergarten here. To join the group either find an existing Hackergarten near you, or start your own. This is a step by step list of what I did when I started Hackergarten Vienna.

The Simple Way to Start a Hackergarten
  1. Contact a few people that might be interested.
  2. Find some Open Source committers and the smallest tickets/ideas to implement from their projects. Usually committers are short on personal time anyway and will love to find someone who is willing to help.
  3. Ask them to create a list of possible things to do in advance and publish it. Maybe have a theme for a night.
  4. Determine the preconditions (what is needed to code...) and publish them, so people are able to instal before.
  5. Find a place with wireless network. Most likely this is the office of a small company, e.g. Canoo or Sphinx. You might also find some pub with a separate room/wireless for no charge.
  6. Negotiate a proper date and time. At least for the first time. Then having a fixed time, e.g. first Tuesday each month is probably the best.
  7. Meet.
  8. Discuss the agenda for max. 15 minutes, do not discuss too long.
  9. Have food and drink ready so people do not have to leave for it.
Read the Hackergarten FAQ for further details and what to avoid.

What to Do
There have been all kinds of contributions made during a Hackergarten. Here are some general ideas what to do during a Hackergarten:
  • Fixing an outstanding ticket (bug) in project X, submitting a patch.
  • Writing Javadocs/doc pages for project Y.
  • Building a plugin for project Z.
  • Making screen casts showing how you can integrate W with Q.
  • Writing a new feature for project P.
  • Create a kata cast.
Also everything applies from Ways to Contribute to Open Source without Being a Programming Genius or a Rock Star.

12 August 2011

Word Wrap Kata Variants

It's time for some exercise - time for a code kata. I like the simple ones because they don't take much time and still provide a certain amount of training. After having done the Prime Factors Kata more than 20 times using Java, Ruby, C#, Turbo Pascal, BASIC and even Forth, I feel like trying a new one for a change. I choose the Word Wrap Kata, also by Uncle Bob. Unlike Prime Factors, Word Wrap seems to be less popular, there are only a few experiences with it published, e.g. Word Wrap using Python.

Recursive
The kata's task is to write a function that, like a word processor, breaks the line by replacing the last space in a line with a newline. The most straight forward solution to this is IMHO the recursive one. I only need to consider the first blank or forced break and call myself with the remaining text.
public String wrap(String line, int maxLineLen) {
   if (line.length() <= maxLineLen) {
      return line;
   }
   int indexOfBlank = line.lastIndexOf(BLANK, maxLineLen);
   int split;
   int offset;
   if (indexOfBlank > -1) {
      split = indexOfBlank;
      offset = 1;
   } else {
      split = maxLineLen;
      offset = 0;
   }
   return line.substring(0, split) + NEWLINE +
      wrap(line.substring(split + offset), maxLineLen);
}
Usually kata is not about the final solution, but about the process to get there. Still I want to compare different solutions, so let's analyse this one. If the line needs to be split n times (into n+1 shorter lines) then this solution creates 3*n String objects and additional n StringBuilders for string concatenation. ... 4*n objects are created.

Exercise Time at Karate DojoTail Recursive
To be tail recursive a function's last statement must be the recursive call.
public String wrap(String line, int maxLineLen) {
   StringBuilder accumulator = new StringBuilder();
   wrap(line, maxLineLen, accumulator);
   return accumulator.toString();
}
private void wrap(String remainingLine, int maxLineLen, StringBuilder accumulator) {
   if (remainingLine.length() <= maxLineLen) {
      accumulator.append(remainingLine);
      return;
   }
   int indexOfBlank = remainingLine.lastIndexOf(BLANK, maxLineLen);
   int split;
   int offset;
   if (indexOfBlank > -1) {
      split = indexOfBlank;
      offset = 1;
   } else {
      split = maxLineLen;
      offset = 0;
   }
   accumulator.append(remainingLine.substring(0, split));
   accumulator.append(NEWLINE);
   wrap(remainingLine.substring(split + offset), maxLineLen, accumulator);
}
This solution creates the new String in the very end. It needs 2 Strings per new line and only one StringBuilder and a final String to return it. ... 2*n+2 objects are created.

Loop
A tail recursive function can be rewritten to reuse the stack frame transforming it into a plain loop. As Java does not support that optimisation, I do it by hand.
public String wrap(String line, int maxLineLen) {
   StringBuilder accumulator = new StringBuilder();
   String remainingLine = line;
   while (remainingLine.length() > maxLineLen) {
      int indexOfBlank = remainingLine.lastIndexOf(BLANK, maxLineLen);
      int split;
      int offset;
      if (indexOfBlank > -1) {
         split = indexOfBlank;
         offset = 1;
      } else {
         split = maxLineLen;
         offset = 0;
      }
      accumulator.append(remainingLine.substring(0, split));
      accumulator.append(NEWLINE);
      remainingLine = remainingLine.substring(split + offset);
   }
   accumulator.append(remainingLine);
   return accumulator.toString();
}
The loopy :-) solution creates the same number of objects as the tail recursive one, but has reduced call overhead. ... Still 2*n+2 objects are created.

Optimised Loop
Let's optimise away the splitting of the remaining line because it gets split again in the next call, so all these Strings are only temporarily used.
public String wrap(String line, int maxLineLen) {
   StringBuilder accumulator = new StringBuilder();
   int pos = 0;
   while (pos + maxLineLen < line.length()) {
      int indexOfBlank = line.lastIndexOf(BLANK, pos + maxLineLen);
      int split;
      int offset;
      if (indexOfBlank > pos - 1) {
         split = indexOfBlank;
         offset = 1;
      } else {
         split = pos + maxLineLen;
         offset = 0;
      }
      accumulator.append(line.substring(pos, split));
      accumulator.append(NEWLINE);
      pos = split + offset;
   }
   accumulator.append(line.substring(pos));
   return accumulator.toString();
}
Now only one String is created per new line, one for the last remaining part and one after the final concatenation. ... n+3 objects are created.

wrappedUsing a Buffer
If I could get rid of half of the String splitting, why not drop the other half too?
public String wrap(String line, int maxLineLen) {
   StringBuilder accumulator = new StringBuilder();
   accumulator.append(line);
   int pos = 0;
   int inserted = 0;
   while (pos + maxLineLen < line.length()) {
      int indexOfBlank = line.lastIndexOf(BLANK, pos + maxLineLen);
      if (indexOfBlank > pos - 1) {
         accumulator.setCharAt(inserted + indexOfBlank, NEWLINE);
         pos = indexOfBlank + 1;
      } else {
         accumulator.insert(inserted + pos + maxLineLen, NEWLINE);
         pos = pos + maxLineLen;
         inserted++;
      }
   }
   return accumulator.toString();
}
Only one StringBuilder and one String are created. ... 2 objects are created. This is definitely the most garbage collector friendly solution because it creates only one temporary object, the StringBuilder.

Copying Characters
If there are blanks in line then all characters get copied once in append() and all solutions behave similar. (The method substring() does not copy characters, it just creates a new String with different pointers in the character array of the original String.) But if there are no blanks in line then the last solution copies all characters after the split point around one or more times. Copying large numbers of characters might be slower than allocating an object, especially as the JVM/Hotspot is optimised for short lived, small objects.

Resizing the StringBuilder
For all solutions using an explicit StringBuilder another optimisation is to avoid automatic resizing. The size of accumulator must be large enough to contain all additional newlines. If a line has a blank then it's replaced, so no new character is added. Only when a line contains no blank, then a newline is inserted. This can happen up to lineLen / maxLineLen times. Flooring (rounding down in integer division) is the right thing because after the last part, e.g. remainingLine, nothing is added.
...
   StringBuilder accumulator =
      new StringBuilder(calcMaxSize(line.length(), maxLineLen));
...
private int calcMaxSize(int lineLen, int maxLineLen) {
   int maxCharsAdded = lineLen / maxLineLen;
   return lineLen + maxCharsAdded;
}
Note that all these optimisations are highly theoretical. In a typical web or database application it doesn't matter at all if a few temporary objects are created or not. Remember that "Early optimisation is the root of all evil" (Donald Knuth). I would stick with the first solution because it's the shortest and easy to understand.

Bonus Round
public String wrap(String line, int maxLineLen) {
   return line.replaceAll("([^ ]{" + maxLineLen + "})" + // 1
      "(?=[^ ])" +                                       // 2
      "|" +                                              // 3
      "(.{1," + maxLineLen + "})" +                      // 4
      " ",                                               // 5
      "$1$2" + NEWLINE);
}
This solution is even shorter, just one line using a Regular Expression. It replaces the split points with a newline or adds one. The pattern matches areas of line which (1) contain exact maxLineLen characters that are not blanks, (2) which must be followed by a character that's not a blank (and which is not consumed) (3) or it matches areas of (4) one up to maxLineLen characters (5) which are followed by a blank. The match is replaced with the first (1) and second (3) groups together with a newline. The single blank (5) is not a member of any group and is dropped.

(Get the source.)

8 August 2011

Maven Plugin Harness Woes

Last year I figured out how to use the Maven Plugin Harness and started using it. I added it to my Maven plugin projects. Recently I started using DEV@cloud. DEV@cloud is a new service which contains Jenkins and Maven source repositories. CloudBees, the company behind DEV@cloud, offers a free subscription with reduced capabilities which are more than enough for small projects. I set up all my projects there in no time, but had serious problems with the integration tests.

Using a local Maven repository other than ~/.m2/repository
Repository (unnamed)You don't have to use the default repository location. It's possible to define your own in the user's settings.xml or even in the global settings. But I guess most people just use the default. On the other hand in an environment like DEV@cloud, all the different builds from different users must be separated. So CloudBees decided that each Jenkins job has its own Maven repository inside the job's workspace. That is good because the repository is deleted together with the project.

Problem
The Testing Harness embeds Maven, i.e. forks a new Maven instance. It fails to relay the modified settings to this new process. During the execution of the integration test a new local repository is created and the original local one is used as a remote one (called "local-as-remote"). But without any hints, Maven uses ~/.m2/repository. So the true local repository is not found and all needed artefacts are downloaded again. This takes a lot of time (and wastes bandwidth). Dependencies that exist only in the local repository, e.g. snapshots of dependent projects, are not found and the integration test fails.

Solution
RepositoryTool.findLocalRepositoryDirectoy() uses an instance of MavenSettingsBuilder to get the settings. Its only implementing class is DefaultMavenSettingsBuilder and it tries to determine the repository location from the value of the system property maven.repo.local. Then it reads the user settings and in the end it uses ~/.m2/repository. The solution is to set the maven.repo.local system property whenever the local repository is not under ~/.m2/repository. Add -Dmaven.repo.local=.repository into the field for Goals and Options of the Jenkins job configuration.

Using additional dependencies while building integration test projects
indirectionAfter the plugin under test is built and installed into the new local repository the Maven Plugin Harness runs Maven against the integration test projects inside the src/test/resources/it folder. The approach which I described last year forks a Maven with pom, properties and goals defined by the JUnit test method.

Problem
The integration tests know the location of the new local repository (because it is set explicitly) and are able to access the plugin under test. But they know nothing about the local-as-remote repository. They can only access all artefacts which have been "downloaded" from the local-as-remote repository during the build of the plugin under test. So the problem is similar to the previous problem but occurs only when an integration test project needs additional artefacts. For example a global ruleset Maven module might consists of XML ruleset configuration files. The test module depends on the Checkstyle plugin and executes it using the newly build rulesets. So the object under test (the rules XML) is tested indirectly through the invocation of Checkstyle but the ruleset module itself does not depend on Checkstyle.

Solution
All POMs used during integration test have to be "mangled", not just the POM of the plugin under test. The method manglePomForTestModule(pom) is defined in the ProjectTool but it's protected and not accessible. So I copied it to AbstractPluginITCase and applied it to the integration test POMs.

Using settings other than ~/.m2/repository/settings.xml
Cluster ConfigurationIf you need artefacts from repositories other than Maven Central you usually add them to your settings.xml. Then you refer to them in the Jenkins job configuration. Behind the scenes Jenkins calls Maven with the parameter -s custom_settings.xml.

Problem
Similar to the repository location, the custom settings' path is not propagated to the embedded Maven and it uses the default settings. This causes no problems if all needed artefacts are either in the local-as-remote repository or can be downloaded from Maven Central. For example a Global Ruleset might contain some Macker Architecture Rules. The snapshot of the Macker Maven Plugin is deployed by another build job into the CloudBees snapshot repository. The test module depends on this Macker plugin and runs it using the newly built rulesets.

Solution
AbstractPluginITCase calls BuildTool's createBasicInvocationRequest() to get an InvocationRequest and subsequently executes this request. Using any system property the InvocationRequest can be customised:
if (System.getProperty(ALT_USER_SETTINGS_XML_LOCATION) != null) {
   File settings =
      new File(System.getProperty(ALT_USER_SETTINGS_XML_LOCATION));
   if (settings.exists()) {
      request.setUserSettingsFile(settings);
   }
}
Then the value of the used system property is added into the field for Jenkins' Goals and Options: -s custom_settings.xml -Dorg.apache.maven.user-settings=custom_settings.xml.

Alas I'm not a Maven expert and it took me quite some time to solve these problems. They are not specific to CloudBees but they result from using non default settings. Other plugins that fork Maven have similar problems.

6 August 2011

Diaries of a New Employment

SkyscraperI have got a new job. My last employer, sort of a startup, went bankrupt. That's particularly sad, because they offered "Research Days". Similar to Google Friday, you were allowed to work on a random topic one day in a month. One day is not much, still it's so much more than other companies offer. The new one is big. After some nasty times at big companies, I'm not sure if big is good for me, but still I have to try.

(Sort of a) Diary
Since first of July I've been working there. Some friends asked me how am I doing. So here is my diary. It's biased because I'm too strict. To be fair - everything looks good. I'm not dying from excitement, but it's ok. I have to get used to it.
  • Day 1 - Shocked. It's my first day and I'm already totally shocked. A senior team member commits production code containing System.out.println because he doesn't bother to remove them. Later another team member doesn't know Eclipse's "Compare With/Each Other" function. I feel like running away.

  • Day 4 - Déjà vu. I'm the new guy in the team and the new guy is supposed to update the development setup documentation.

  • Day 6 - First Blood. Out of curiosity I take a brief look at the existing code base: There are more than 7000 classes with about 600000 non commenting source statements. There is all the classic legacy stuff you would expect to be hidden in such a large code base, e.g. methods containing up to 700 lines, many empty catch blocks and much more. But my favourite findings are the 60 Java source files which have been commented out completely. (I didn't know the Java compiler allowed empty files without any class declaration.)

  • Day 8 - Deadlock. I'm supposed to add information about myself to the internal directory. To edit my entry, I need to fill in a valid phone extension, which I don't have yet. So I try to get one but the phone extension registry would not give me one because my directory entry is incomplete.

  • Day 11. For two hours I'm unable to start my e-mail client. I'm trying all kind of workarounds but it just wouldn't work. I'm getting desperate. In the end it turns out that there is an internal application to clean up and restart the mail client which finally solves my problem.

  • DiaryDay 12. Today I saw the first person wearing shorts in the office. I was already getting stressed by being the only one wearing t-shirts and shorts among many people wearing suit.

  • Day 14 - Hope. We are preparing a list of refactorings that would improve maintainability of the code. I stay quiet because I'm the new one and don't have all the information. I'm glad when another team member proposes what I wanted to say. There is hope.

  • Day 15. The project manager wants to give me responsibilty for the continuous integration build. I cringe because setting up and maintaining the build is cumbersome and boring. Fortunately my manager objects that "there are other important things for Peter to do". +1 for saving me.

  • Day 18 - Public Relations. I know it's a waste of time, but I'm stubborn and ask the person responsible for PR/communications about the mode and support for publishing and presenting. I never got an answer.

  • Day 20 - Coder from Hell. A senior colleague does not care for compiler warnings because "he knows what he is doing". Well he should know better, especially as we are tasked to clean up some legacy mess that exists because of people like him. Anyway he doesn't give a shit about clean code. I haven't met such a kind of a developer before.

  • Day 22 - Print a Page. I need to print a page. This is usually not a problem. I install the printer driver, print the page, go to the machine and find that the device is out of order. Then a colleague tells me that it's broken all the time. Where is the next printer? So I find the printer on another floor, install, print, go there and find that the room containing the printer is locked. Ok, I repeat the whole schema with the printer of another floor, go there - but it's not "there". I can't find this bloody printer. I repeat with the printer on yet another floor, go there - and finally I'm able to collect my printout.

  • Day 28 - Long Line. I just found a single line of code containing 2881 characters. Yes, a single line. That's by far the longest line I have ever seen and it's even way beyond any long lines recorded by fellow craftsmen.

31 July 2011

Finally a Proper Keyboard

At last GeeCON I met Hamlet D'Arcy and he distributed keyboard stickers with IntelliJ IDEA's keymap. I had them lying around for some time because I did not want to put them on my primary keyboard (because it does not have any labels). Last week I cleaned my old Silicon Graphics keyboard and boosted it with these stickers. Doesn't it look great?

Silicon Graphics Keyboard with IntelliJ IDEA Keybinding
I had to customize the stickers to my German keyboard. The Shift and Enter keys are shorter and I had to cut their stickers. Then I thought about changing the keymap inside IDEA and to put the stickers on the keys where they would be on an English keyboard. But I got confused, and in the end I put the stickers on their according places. I only changed the binding for [ to ö and ] to ä because 8 and 9 where [ and ] are on, were already taken.

1 July 2011

My Favourite Topic

Last month I had the opportunity to take part in a job assessment of the Austrian Software Test Experts. The managing directors and I have been friends since long as would be expected from guys who say that "software test is cool" and a "fanatic about code quality". I was looking forward to meeting the whole staff of testing experts.

FavoriteThe CEO had asked me to prepare an introduction and a short presentation about some code quality related topic. Unfortunately I had a very busy time and wasn't able to prepare anything in-depth. The night before I just hacked together a little presentation by copying all kind of slides from some of my previous presentations. Stealing the joke from Uncle Bob's talk Craftsmanship and Policy (2:30), I titled it My Favourite Topic, which is of course - me. ;-)

Feedback
I didn't take the presentation too seriously however the CEO said it was hilarious, almost comedy styled. People who had known me liked it a lot. But it turned out that I did a really poor job. Half of the audience didn't get the joke and therefore did not know what I was talking about. They got the humorous, self-deprecating introduction wrong and were discussing if I was disrespectful or just unprofessional. They hauled me over the coals.

16 June 2011

Headhunter Fail

Recently I was called by a headhunter. I don't mind being contacted by them if they do their research properly and have interesting things to say. But it wasn't the case this time...

Head huntingThe Call
I don't know why these people (headhunters) always need to call. I personally would prefer an email. I will read it when it suits me. But being soft-skilled, they have to talk (i.e. call). Talk if you have to, but please research and find the proper phone number! This poor fool called the company's main number and my boss was the one to pick it up. This was quite an embarrassment.

She (the headhunter) told me about a position I might be interested in. Well, I don't have time to talk, but send me some details, would you. You know my email address? Yes, of course you do, in contrary to my mobile phone number. So what's the point in calling me anyway? Is there a checkbox on your form to make sure the candidate is able to cope with embarrassing phone calls? But I'm repeating myself. Just don't call me.

The E-mail
Finally I had some hard information in my mailbox.
  • Who she was.
  • Company she was working for. (I had not known it before.)
  • Web page of the headhunting company. (I checked it out and it did not impress me at all.)
  • and the offer.
The Job Offer
And the offer really made me laugh. Here is a rough translation (Google translate rules ;-):
Our client is a reputable company in the high-tech
industry and international leader in its niche.
Wow! It's a reputable and internationally known company. Who cares, but what's the niche? What is it doing? Is it producing stuff or just selling things? How big is it?
Your tasks:
* Analysis and development of existing systems
* Creation of software applications
* Testing of applications
* Implementation
* Fixing bugs in existing applications
Aha, my task is to develop software. That's expected from an offer for a Java software developer. Not much to see here. I would like to know what kind of applications, how the testing is done etc.
Your profile:
* Some kind of technical education
* Experience in software development in Java
* Experience with XML
* Understanding of development processes
* Dynamic personality
* Fluent in English
So one should have experience in developing software with Java. At least here is some information: The company which is hiring is using Java and XML (somehow). They have some kind of development processes. Or they want to have one. But what kind of process, e.g. waterfall or agile? Or do they just want people to be aware of documentation and testing. Is there any Java developer out from school that would not match this profile?

I'm not interested at all
The offer was poor and boring, I didn't even bother to answer it. But still I'm asking myself how the headhunter-researcher could think that this a position I might be interested in.

6 February 2011

UltraEdit and Gforth

Forth
Forth is a computer programming language. Among other things Forth is stack-based, which makes it unique. (Well it's not entirely unique, just quite different from the usual languages like Java or C#.) It's not as popular as other programming systems and most likely you don't know it. I didn't know it either, but the interview with Charles H. Moore, its creator, sparked my interest. So I chose Forth as my new programming language for the last year.

UltraEdit Forth code completionSyntax Highlighting
I use UltraEdit and love it. I created a wordfile for ANS compliant Forth. Adding a wordfile for syntax highlighting is simple. UltraEdit versions 14.20 and earlier use a different approach, but the wordfile is still compatible. As soon as UltraEdit knows about the structure of a Forth program, it's able to provide code completion (shown on the screenshot on the right) and function lists showing all defined functions (see another screenshot).

Limitations
  • The wordfile contains only core words. (Adding words is easy, just add them to the proper section of words in the wordfile.)
  • // is only a comment, if there is a blank or a newline before it, but the later is not possible to define in wordfiles. So line comments starting in the first column are not highlighted properly.
  • UltraEdit supports only symmetric strings, e.g. character sequences between " and ". Forth sequences between S" or C" and " are somehow like strings, but in fact S" and C" are parsing words. These words are highlighted, but not recognised as the beginning of a string. So known words inside strings will be highlighted as well.
  • There is no way to define the pattern for matching numbers and UltraEdit uses its internal magic. This causes some minor glitches in number formatting, e.g. with negative numbers.
Tool Integration
Last year I attended the lecture Stackbased Languages and played around with Gforth that was recommended there. There is a good Gforth integration in Emacs, but I want to stick with UltraEdit. Using its tool configuration I defined a menu item together with a keyboard shortcut for invoking Gforth:

UltraEdit tool configuration for Gforth
Create a new menu item with name gforth, set the command line to gforth "%n%e" -e bye in the working directory %p and make sure the output is captured. Another tool worth defining is the Gforth ANS Report with command line
C:\gforth\gforth.exe C:\gforth\ans-report.fs "%n%e" -e "print-ans-report bye"
Acknowledgement
Experimenting with Forth was part of several System One Research Days. Thank you System One for your support :-)

(Download the UltraEdit wordfile here.)

21 January 2011

For You Scala Enthusiasts

On my way to a Jelly at sector 5 I walked past several streets which I had never seen before and stumbled upon a Scalagasse (Scalaalley or Scalalane):

Scalagasse in Wien MargaretenI didn't know that Martin had already whole streets dedicated to his golden egg :-)

(In fact the alley is named after the Viennese priest Johann Scala (1816–1888), who was a member of the district council.)

13 January 2011

The Lie

<fiction>
Let's start with a little story about an enthusiastic software developer called Bruno. The story begins with an interesting job offer: A well known company is searching for an experienced developers to enlarge an existing team due to an increased business demand. The company does software since more than 40 years and scores reasonable at the Joel Test.

The Interview
The company is looking for top developers, a difficult coding exam during the interview is making sure to weed out the less skilled ones. The exam contains of a full range of Java traps: Beginner mistakes, try-finally idioms, generics and even some Java Puzzlers.

Bruno is one of the best. He finishes the exam in no time and scores well. Two small mistakes he makes just manage to curb his enthusiasm. The interview goes on with questions about common Java frameworks, ORMs etc. For the first time since long he feels challenged and invigorated by the exam's questions and the interview. He asks his usual questions any serious developer should ask about a future workplace, e.g.: How complete is the build? How does the code look like? Are there many singletons? (Because he just hates singletons. They are pure evil.) He gets some reasonable answers and accepts the offer in the end. The world seems like a happy place.

Broken DreamsA Happy Place?
What look's like the beginning of a happy story is in fact its end. As soon as he starts working for the company he discovers that the interviewers had been lying about some things. Maybe they didn't know (but ThreadLocals are still singletons, aren't they). Maybe they were in some state of wishful thinking, confusing planned things with reality. Maybe there were other reasons nevertheless Bruno is disappointed.

Even worse Bruno's work does not reflect the skill level suggested by the interview: Java 4, JDBC statement driven persistence, no challenges, no problems to solve. All he has to do is to make already deeply nested ifs a bit deeper to find a place for more business logic. Bigger changes are discouraged due to the fear of breaking anything. (This might be the case for most modern business applications based on legacy code. Their business logic winds itself in endless, cascading ifs through all layers of the code. Or when did you need to come up with a more complex algorithm than iteration the last time?)

Bruno doesn't want to quit. Running away just doesn't just feel right. He has high expectations. But as the relation with his boss gets worse he gives up and quits. Everybody including himself agree that he wasn't the proper guy for the job in the first place. Hmm. How could that happen?
</fiction>

The Essence of the Lie
It has been two years since I first realised "the lie" as what it is. There were times when it really bothered me, but most likely it's just another thing to consider when looking for a job. It's well known that there are at least two kinds of software people. Some call them developers vs. programmers, smart vs. non smart, craftsmen vs. hackers, experts vs. journey men, professionals vs. duct tape programmers, coders vs. "code monkeys", etc. The essence of "the lie" is that some companies looking for the first kind are in fact in need of the second.

This is not a rant against these second kind of programmers. There's nothing wrong with them. (Note: There is a negative undercurrent in duct tape programmer or code monkey but that's most likely due to the arrogance of us craftsmen. So I will stick with the term plain programmer.) For certain problems in certain environments the plain programmer is simply the better choice.

Anatomy of a Plain Programmer
Now I'll try to list some virtues of a plain programmer :-)
  • There is no need for a plain programmer to know his or her IDE well, he just has to know the debugger, because that's the tool he will use most of the time to poke in the legacy code.
  • And he doesn't need to know any advanced language features and God forbid he must not use them. (His peers will be confused, so he should better stay with the stuff everybody knows.) So for example there is no need to know concurrency. (Note: I'm not joking, I witnessed my boss questioning the usefulness of an internal training about Java concurrency features.)
  • It's desirable that he doesn't think too much about his work, better cascade another if and change existing code as little as possible then mess with the underlying design. (Refactorings and design changes almost always spoil dead lines, esp. when they were ridiculous in the first place.)
  • He has no ambitions and does not feel the need to change already messed up processes.
  • And last but not least he has to have a high capacity for suffering. He will need it.
Stop Sign In Front Of My HouseIn my experience it's more about the environment enforcing or at least encouraging these behaviours than someone just wanting to be like that. Less aspiring people with a high tolerance to discouragement just happen to endure it longer.

When Hiring
So next time you need a programmer make sure you know what kind you need. Don't lie to yourself. Don't give in to wishful thinking. It might not be comfortable for you but your project/organisation might be a mess. If you are in such a situation that you need a plain programmer, hiring an expert developer is just a mistake. You don't need a rock star programmer to churn out another business rule. Don't ruin another craftsman's life by sending him into a battle he can't win.