Showing posts with label quality. Show all posts
Showing posts with label quality. Show all posts

11 September 2019

Human Needs vs. Bad Code

We human beings have basic needs. There might be a finite, limited number of fundamental needs. We then choose different strategies to meet these needs. Techniques like Nonviolent Communication (NVC) are based on identifying and meeting shared needs. For example, my goal of professional training and coaching is not only knowledge transfer - i.e. teaching people useful tricks. I also want to stretch them, make them think "outside the box" and change their behaviour: e.g. focus in self-improvement, take control of their career and ultimately assume their social responsibility. When I thought about my work as Code Cop, what I liked about it and what inspired me, I arrived at my own needs: learning and growth - I want my clients to learn and grow. Further clarity, consistency and integrity which is what I need when working with software.

My Needs (Wordle)
Needs are central to our work, our own needs as much as the needs of our colleagues and clients. Earlier this year I attended an unconference, meeting some friends in Grenoble. We discussed how to align technical coaching work, e.g. Technical Agile Coaching, with business goals. We paired up and worked on different approaches. I was not surprised when one team started the whole alignment with discussing basic needs of all people involved. It was very interesting and outside of the scope of what I want to write about today.

Missing Code Quality
Today I want to write about possible reasons for missing code quality. I discussed human factors before and held the strong opinion that developers creating bad code were unskilled, lazy and weak. When considering needs and strategies to meet them, the issues is getting more vague. Which needs could be fulfilled by a developer creating some quick and dirty code, duplicating some method or adding some library just to play around with it? When I discussed this with my friend Aki Salmi, Software Crafter and Communication Trainer, he quickly came up with a bunch of needs like acceptance, appreciation, cooperation, consistency, inclusion, respect/self-respect, stability, trust, integrity, autonomy, choice, challenge, competence, contribution, creativity, discovery, effectiveness and purpose. This is a huge list of needs to get started. As an exercise to the reader, try to figure out how the listed needs could be met.

Needs Met When Creating Bad Code (in German)
Three Examples
Now let's motivate some of the listed needs in detail. For example, someone might bring in some technology which is not necessary for the project and introduces additional, unnecessary complexity. Met needs might be creativity (I want to try something new), learning (I want to learn it), joy (I enjoy playing with it), safety (I will add it to my CV - Resume-driven development), autonomy and choice (I choose myself), consistency and integrity (I have always been using it), safety (I already know how to use it) and so on. What about someone who never argues for quality related changes, clean-ups, more time or reduced scope? Needs met might be appreciation (My boss is happy), ease (I do not argue with anybody), security and protection (I keep my job) etc. And for typical fire fighting, quick-and-dirty developers: competency (I can do it), efficacy (I am fast), effectiveness (I can make it work). These are just a few ideas I got while discussing the topic during a workshop.

Needs Met When Creating Good Code (in German)
Opposite
Now let us look at the opposite side. For quality code, I value consistency - which is obviously a good thing. It is also consistency which keeps some people from adapting to new ways of working, as in "this is the way I've always worked" - obviously a bad thing. If I keep my code base clean, I am sure that I can work on it later, which makes me feel safe. The same need for safety might keep someone from trying something new, because it is scary, or it might make people thrash their code because their boss is requesting too much in too short time.

Conclusion - If Any
Needs are everywhere. They are universal, cannot be denied nor argued. We use some strategies to fulfil needs when we mess up the code. There are many needs involved, maybe that is why there is so much bad code written. On the other hand, we use strategies to fulfil needs when keeping our code clean. It scares me that opposite behaviour might be driven by same needs. How can we condemn these duct-tape and legacy coders when they are just driven by their needs as we are?

27 February 2017

Time for Quality

Since one of my very first presentations in 2009 I have been asked how to make time for testing and code quality related activities. Keeping the code of high quality is difficult if your manager or product owner is only interested in deadlines and the number of hours spent on creating the software. People told me that in their organisation there is neither time nor budget for code quality and that their boss does not consider it necessary. While this is the perfect subject for an angry rant, this time I want to list some options instead.

Context
First let me set the context for my answers:
  1. The demand for software developers and IT professionals in general is ever increasing. We are a highly privileged part of the entire workforce. While finding a decent job is always a (subjective but nevertheless real) problem, finding any work in software just to feed the family is not an issue. There is always another job.

  2. Testing is a mindset - you have to want it. There are thousand excuses why not to write a test or delay that particular cleanup. Some people pretend not to be allowed to create quality code, while they are just lazy. I would rather hear their honest views. I am suspicious of colleagues, who claim an interest in quality code but keep delivering crap on a daily basis.

  3. More often than the lack of interest is the lack of knowledge. For example many developers I meet are not sure how to write good automated tests, because they have written only a few in their entire career. Because they are unsure - they do not know how to start - they claim it to be time consuming and drop the idea. If I do something for the first time I will not do it well. Until I master the new skill I will be slow. These developers need training, more specifically they need practise. Every developer needs to be familiar with clean code, refactoring unit testing, and much more. If you feel not sure enough to apply these core skills in your daily work, ask for training, attend programming workshops at conferences or get a programming coach. Coding Dojos and Coderetreats are a great place to start.
Actions
So you want to write unit tests and are able to do it. But your boss has no interest in it or there is no team culture to do it. What could you do?

An argumentFirst read my favourite article of Joel Spolsky, Getting Things Done When You're Only a Grunt, which covers some organisational issues. Like Joel, I recommend staying true to yourself - just do it. This is hard in the beginning when you are alone, but others might follow. (It takes a lot of energy and several years to transform a whole team from within.) When complaints about your work start coming in you have several options:
  1. Do not try to defend yourself and avoid arguing. This is the way you work, end of discussion. Again, you need to be fluent and sure to be able to do that. Especially if you are a new employee you have some privileges. The team might accept your new style. If you are applying testing and clean code successfully others might follow.

  2. Argue for code quality. Sure you need some extra time to write these tests, but then you will not have to go back and fix the code again and again which results in less context switches, less defects and less hot-fix releases. I have not met any manager who would not understand that. But sometimes we (IT) do not use the right vocabulary to be understood by managers. When you talk to your boss, it is better to talk about cost, risk and financial benefit than technical issues. Using the management speak is a skill. Start by collecting hard facts about code quality related activities. Research the estimated cost and later benefit for your particular case and compare it to the risk and later cost of skipping it. The Technical Debt metaphor helps here. Create a simple Powerpoint (yes, blasphemy ;-) with these facts. End with the red traffic light or the estimated downward trend of the team's velocity. For example instead of "the code is ugly" you could say that "because of its inconsistent structure you are more likely to introduce mistakes when changing it." Charts displaying the absence of structure, e.g. class diagrams with edges all over the place, make the missing structure obvious for non-technical people.

  3. Silence, pleaseHide the activities. Push for code quality while you create the code. If you write the test before (TDD anyone?) or immediately after the implementation, writing unit tests is not visible as separate activity. The same is true for refactoring. Do not wait for Friday afternoon to clean up the code you wrote all week, because you might not have time for that. If you improve the code after each green test, there is no bad code even if you are forced to stop early. Further add the time for testing and cleanup to your estimates, but do not talk about it. I know some developers who estimate higher than others. Their estimates are accepted because their solutions are better and have less defects.

  4. Make excuses. Even technical managers do not know the exact details about the code you work with. Maybe there were some issues not anticipated during planning. There are always some. Communicate their (exaggerated) impact and argue that you needed some time to deal with them while in reality you had to restructure some legacy class to add to it.

  5. Finally, you can always find another job. Today you have more options than ever due to remote work.
There are several related questions on Stack Exchange which give more options, e.g. what do you do when your boss doesn't care about code quality or how can I convince management to deal with technical debt.

I wish you all the best for your quest for more code quality in your daily work!

9 June 2016

Oracle Code QA

As Code Cop I want all my code to be clean so I keep my sanity when maintaining it. Some basic pillars that support internal code quality regardless of programming language are Coding Conventions, automated (unit) tests, Static Code Analysis and Continuous Integration. I discuss all of them in my Code Quality Assurance lecture (and its latest slides are here). A good development process covers all these and more.

Recently a colleague inherited a bunch of Oracle PL/SQL code and asked me for help. Being used to Java and many tools that help us keeping the code in shape, e.g. JUnit, Checkstyle, PMD, Jenkins, he wanted the same for his database code. While some programming language ecosystems are traditionally strong in supporting the things I mentioned earlier, some other languages seem to lack behind. Clearly there are fewer options for less used languages. But that must not stop us from applying the same rigour to our code. Let's get started!

Everyone is Marcin Grzejszczak todayDatabase Naming Conventions
First we need coding conventions because consistency is important. Unlike Java where most projects follow the Oracle conventions, there is no such thing for Oracle databases. Instead there are several, sometimes contradicting proposals and you have to put together your own set of rules. Here are some reasonable ones for schema objects:PL/SQL Coding Conventions
The Procedural Language/Structured Query Language (PL/SQL) was introduced by Oracle in 1992. It is a compiled, procedural and structured language. By these attributes it is similar to modern languages like Java or C#, and all the general advice for naming, formatting, commenting, function scope and code size apply. Even object oriented concepts like Encapsulation or Coupling are meaningful (to a certain degree). See my presentation on Clean PL/SQL for more details. Again there are no official conventions from Oracle.
  • Steven Feuerstein's Naming Conventions and Coding Standards contain a list of naming conventions for PL/SQL variables together with some guidelines and a discussion of rejected conventions. If you do not know Steven, he is probably the authority on PL/SQL programming and knows what he is talking about. He also outlines a way to check the conventions, which I really like.
  • Philip Greenspun's SQL Style contains a few rules on formatting SQL statements for better readability.
  • Trivadis' PL/SQL and SQL Coding Guidelines are a complete set of standards regarding naming, formatting, language usage and control structures. It is a very comprehensive document of almost 60 pages and looks really impressive.
How to Choose Your Own Conventions
As there is no standard, you need to roll your own. To get started I recommend reading all the resources above (and even google for some more) and get an idea what could and should be defined. Then you look at your existing database objects and source code. Usually developers follow some conventions and some percentage of the code uses similar patterns in formatting or naming. If one of the used conventions is in the limits of the different proposals above - and you like it - then start with it. (Starting from something that is already there reduces your options and the resulting conventions are less optimal, but on the other hand you have a bigger change to get the code into a consistent state, because some part of the code follows the rules. If there are no existing patterns in your code, if you are starting from scratch or if all you see is crap, you still need to define different conventions.)

Quality ControlStart with a small set of rules in the beginning. There should be some naming schemes, table aliases and formatting rules. If you define too many rules at once, there will be too many violations in the existing code and people will argue that adhering to the conventions is too much work. Later, when everybody got used to the rules, it is time to add more of them. You will find more specific rules during the lifetime of a project, e.g. by identifying bug patterns to be avoided in the future. Conventions need to grow. Unless you are beginning a new project and want to start with a full set of conventions, the Trivadis conventions mentioned above might be too comprehensive to start with. But they are an excellent example how a full blown conventions document looks like.

Reviewing your code, reading the provided resources and collecting the basic rules that apply and that you like should not take you more than a few hours. It is more important to start with the first version of coding conventions sooner than to start with a complete set later.

Unit Test
If you are used to JUnit, RSpec or Jasmine you will be disappointed. There is not much support for unit testing in PL/SQL.
  • Usually - if at all - developers create stored procedures that call other procedures and check the results programmatically. If these test procedures follow a common convention, e.g. raising an exception on test failure, it is possible to automate calling them from the command line or build server.
  • Another option is utPLSQL, a basic unit testing framework created by Steven Feuerstein. It works as expected, but lacks the comfort of modern unit testing frameworks. I used it to test my PL/SQL port of Gilded Rose. (Gilded Rose is a testing kata where you need to create a lot of tests. It is an excellent exercise to get a first impression of a unit testing framework.)
  • Oracle SQL Developer has some support for automated testing. Unlike utPLSQL it is driven through the user interface. Tests are created and executed through the UI of SQL Developer. A Test case is a set of input values - usually rows in one or more tables - and a call to a stored procedure. Then the updated values are compared against a set of expected values. To see this in action, check out Jeff Smith's introduction to Unit Testing Your PL/SQL with Oracle SQL Developer. It is easy to create first tests, but test definition lacks the power of a general purpose programming language. Further I do not like that test definitions are "hidden" in some SQL Developer specific tables, the Unit Test Repository. However if you are a heavy user of SQL Developer, it might still be reasonable to use it for testing, too.
  • DbFit is an extension to FitNesse, a standalone, acceptance testing framework. DbFit tests are written using tables, making some test scenarios more readable than xUnit-style tests.
  • Steven has more recommendations for unit testing PL/SQL, including Toad's Code Tester for Oracle. If you licenced the Toad Suite, it is worth checking out its testing functionality.
Unit testing is mandatory, but no single approach or framework looks superior. I believe the best approach is to evaluate the different options, using the tools you already have. Maybe create some tests for the Gilded Rose in each of the testing frameworks to see what works for you and what not.

Static Code Analysis
magnifyA vital part of code quality assurance is static code or program analysis. The source or object code is analysed without actually executing it to highlight possible coding errors. I love static analysis but have never used tools for PL/SQL myself. Steven agrees with me that we should use Lint Checkers for PL/SQL - and of course he is right. He lists some tools that add warnings besides the checks provided by Oracle.
  • A free tool is PMD for PL/SQL. PMD is my favourite code checker for Java and it works great. There are only a few rules for PL/SQL but more can be added easily. (See how I added custom rules to PMD in the past.) PMD is definitely a tool I would use first.
  • Trivadis PL/SQL Cop looks very promising. I am not sure about its licence, but it seems to be free. Rules must be checked automatically each day, e.g. in the nightly build, so tools must work from the command line. Again I do not know if PL/SQL Cop works like that. The next step would be to experiment with it and see if it can be run from the command line.
  • Another great tool is the Sonar Source PL/SQL Plugin. The plugin adds PL/SQL support to SonarQube. SonarQube is a free, open platform to manage code quality. It is widely used in the Java community. The plugin is commercial, but if you need to manage a lot of PL/SQL code I would recommend buying it nevertheless.
  • There are several other commercial tools available, e.g. ClearSQL by CONQUEST, which I did not check.
For static code analysis I follow the rule that more is better. I recommend to start with a basic tool, e.g. PMD, and keep adding tools and rules over time. In existing projects you need time to fix the violations, e.g. WHEN OTHERS THEN NULL, and starting with too many rules in the beginning creates a lot of work.

Putting It All Together
After you established conventions, added automated tests and configured some static analysis tools, it is time to put it all together and shorten the feedback loop. While you could run tests and checks manually from time to time, it would be more helpful to do so every night, or even better on each check-in/push. (Checking your code on each check-in requires you to put your DDLs and package sources under version control. While this adds some extra steps to your development workflow, I highly recommend doing so.) A tool like Jenkins or another Continuous Integration server could be used to create an empty database instance, execute your DDLs and compile all your packages. Starting with an empty database instance is important to avoid works on my machine problems. Then Jenkins should run all tests to verify that the code works as expected. The final step is to analyse your code for violations of coding convention and potential problems. Many people add more steps like generating documentation or packaging deployment bundles suitable to be deployed by the operations team's DBA.

Just Do It!
Terence Parr recommends to automate anything that you might screw up and he is right. Creating working software is hard enough, we should not bother with manual tasks, rather automate them. Further automated checks keep the quality of our software high, resulting in faster maintenance and less bugs. This leaves us more time for the interesting parts of software development - solving problem and creating solutions.

7 July 2013

Where is the Quality?

I am professional Java developer since almost 14 years. I have been coding for education, fun and profit for 27 years and three months now. I bought my first book on how to program BASIC for the Commodore 64 in 1986, even before I got the computer itself. I have been enthusiastic about all this since then and never stopped pushing forward. But I am depressed. Maybe I get depressed easily, as once after a particular bad gig I had to take an antidepressant for more than a year. Fortunately no burn-out this time - I am just fed up. Time for a rant ;-)

AngerThe current state of the IT industry makes me unhappy. That is nothing new and maybe my last employer is a particular bad example, but I do not think so. I have been with some companies during my career and talked to developers working on different projects, and usually I hear the same story: Deadline first, crazy rush, get the shit out of the door, repeat. I am the Code Cop, I try to keep things together, clean them up, whip them into shape. But I am wasting my time. My work has no meaning. I am just one but "they" are legion. Recently I got really angry when being confronted with bad code. How do you dare to deliver such crap to my production code base. This is all a big joke.

Some developers have no idea about object orientation, encapsulation or how to design a class at all. The senior developers know their business and applications well, still some of them are unable to create a reasonable design or model classes with a single responsibility. This does not only apply to off-shoring, but to local teams as well. Managers and architects are talking about all aspects of the development cycle but the actual creation of the core asset, the executable code. Development of code is seen as a commodity, unimportant work, just fill in the gaps.

Dark Future
Last month I was invited to a small panel discussion. Together with a test automation expert and an user interface specialist, we were discussing with the audience about the future of software development. The opening question was for our advice for students of higher technical education. Where the testing and user interface people predict a bright future, I have a darker vision. I would not advise anybody who is passionate to start working in today's mainstream IT industry. Software is growing more and more complex and changing it takes longer and longer. At the same time the cost pressure increases. Current trends like Water-Scrum-Fall try to fix these problems with a process. The real issue is the bad quality of our code. We have accumulated a massive amount of technical debt. The same is true for testing. Yes, we plan for unit tests during development, but then need to finish the requirement first and skip the tests. In the end we get some UI test automation, delivered by an outsourced testing team. Hello software testing ice-cream cone.

Dead EndEscape?
I get many mails from head hunters looking for Java developers. But their work is as broken as ours. I am sure, if they would just read one post of my blog they would not send me information about an open position for a junior developer who is supposed to add buttons to a web application. Some head hunters are even specialized in IT personnel but seem to be as (un-)skilled in their work as most of us. From my experience they just do not care, else they would not send me letters with a wrong name in the salutation. Anyway, all advertisements for job vacancies I have seen so far were flat and did not say anything about what to expect. I am not excited.

I believe that the Software Craftsmanship movement was born to address these issues. But Austria is a small country and I am not aware that there are companies like 8th Light around. Still I know of many passionate developers and would like to hear about companies, where all members of the development team are craftsmen. It will definitely be a small company, at least a small development team hiding somewhere. So please stop hiding and come outside, I am looking for you! (I really hope you exist ;-)

15 December 2010

Code Quality Assurance v2

Last week I gave my presentation on code quality assurance, for the second time this year, and it looks like it will become an integral part of the lecture. That would be great. Maybe one or another soon-to-be developer will get interested by the ways of the craftsmanship and not become a Duct Tape programmer.

(Code) Monkey and some Duct Tape...I was quite nervous in the beginning of the presentation as I had never spoken in front of so few people :-) Honestly, giving it the second time helped me a lot and I was much more relaxed and didn't hide behind my laptop most of the time. There were few students, but they asked clever questions in the end. Only one student complained about too much mathematics needed to solve the prime factors kata. Well it's not that complicated, but maybe I will try the word wrap kata next time.

Dear Students
There is a list of references at the end of the slides, only a few but you should read them. Go ahead, read them now! I will wait here. If you are desperate you might listen to the recording of last year's quality assurance presentation. It's missing the demos, but my explanations should give you a general idea of what's going on. Good luck and don't succumb to the dark side!

9 March 2010

Code Quality Assurance

Last week I had the opportunity to give an one hour presentation on code quality assurance as part of the lecture on software testing to students of the Fachhochschule Technikum Wien. By "code quality assurance" I meant principles and techniques used by software developers to test their software and keep it free of bugs.

I believe that the most important ingredient of code quality is the mind-set of the developer. So I started with some slides about the Zero-Defect Mindset and Software Craftsmanship. Then I did a live demo performing the Prime Factors Code Kata to show the basics of unit testing, Test-Driven Development and regression testing. This was the main part of the presentation.

Break  FreeAfter that I explained the principles of code coverage, continuous integration, static code analysis and code reviews to the students. I mixed the theory (slides) with hands-on examples on the newly created Java code using EclEmma, Hudson, PMD and ReviewClipse.

Doing the demo was fun and the whole presentation was a success. For the demo I tried to stick to Scott Hanselman's Tips for a Successful Technical Presentation, esp. font size (Lucida Console, 16pt). Here is my "BigFonty" checklist:
  • Create a new, clean user profile for presentation only.
  • Set icons to large and number of colours to maximum.
  • Remove all icons from the desktop and choose a plain desktop background. I like to minimise all windows if I get lost between them.
  • Disable any screen saver and turn off energy saving. Otherwise they will definitely activate at the most annoying moment.
  • Set the command shell font to Lucida Console 16 point, bold, green on black. Have the default shell point to your main demo directory.
  • Clean up the browser, remove unnecessary tool bars and symbols. Unfortunately, at least in Windows, new users always have tons of crap on the desktop and in the browser.
  • Set the default browser page to empty or your main demo web-site.
  • Set the font size in your browser to very large and enable override of font sizes in styles. This is done in some accessibility sub-menu.
  • Use the browser in full screen mode (F11). You need all the space available for the large text.
  • Set the main font in your IDE to Lucida Console 16. In Eclipse it's enough to change the Text Font (in the Basic category in the sub-menu Colours and Fonts in Appearance).
  • Turn on line numbering in the IDE for quick reference of single lines.
  • Maximise the IDE and use a full screen source window whenever possible. In Eclipse just press Ctrl-M to maximise a view.
  • Start all applications like IDE or any server before the presentation. They may take some time.
(Download slides or source of Prime Factors Code Kata.)

Update 20 April 2010

Student Feedback

Today I got the feedback evaluation from FH Technikum Wien. Several students mentioned my presentation as exciting and full of practical experience. :-) One called my presentation idiosyncratic - I don't mind, it definitely was. It's only weak point was that students were not able to study using the slides alone. Next time I will prepare some handouts with more information.Radio Podcast Days

Update 8 May 2010

German Podcast of Presentation

I finally managed to post-process the (German) audio stream of the talk and combine it with the slides. Watch the quality assurance podcast (in German). It's still missing the demos, but my explanations should give you a general idea what's going on.

30 September 2009

Slacking Off

... or why automatic checks are necessary.

Human Factors
I must confess, I'm a slacker. For example I've been writing this post for three months and still haven't finished. I skip my workouts again and again. More important things just pop up all the time. Concepts like interesting or important are subjective and priorities are likely to change between individuals and over time. So everybody has his or her sweet spot of slacking. It's impossible (and probably also unwise) to work hard on all aspects of life. When everything runs smoothly, people get sloppy. (Again that might be something good for boring, repetitive tasks - except when a surgeon performs his 1.000th appendix extraction.) When things work out great, we might even get delusions of grandeur and bathe in the glow of our own greatness. Everybody does it, you do it, I do it. Only Chuck Norris does not.

Hmm. I'm mixing different behaviours here: slacking, sloppiness, laziness, lack in motivation, doing things half-hearted, leaving things unfinished. I use all these words synonymously. I Know that's not entirely correct. (Probably that's the reason I can't get this post into proper shape. I've already rewritten it five times. I know that I must not ship shit but I'm getting tired. So I will have to live with it. I'm sloppy myself ;-)

SlothThere are several causes for these factors, e.g. lack of interest (I don't care), boredom (I do it the hundredth time), distraction (I'm not able to concentrate on it - I just love cubicle spaces.), lack of background information (why do I do this crap), fear of wasted effort (I might not need it later) and time pressure (I have no time to do it proper).

Oh My!
What implications do these factors have for code quality? (By code quality I mean the internal code quality, maintained by the developer day after day.) Consider a product 'A'. Features have been added to it for the last five years. The natural laziness of all developers has taken its toll. The code is a mess. Maintenance costs go up. Suddenly code quality gets important. Suddenly management is interested in coding conventions and development processes. Suddenly people are aware of a need for an architecture. Suddenly people want to stop slackerism. But when the product is in trouble it's too late. Not really too late, as software is soft and can be changed all the time, it's just much more expensive. All these things are not new. It's well known that software erodes over time. Slacking developers may just be one of its causes.

Check What?
After this lengthy introduction I prepared my point: The need for automatic checks. Checks are good for you. (Like daily sit-ups.) Do them. Even better, set them up so you don't have to do them yourself. (Somebody does all the sit-ups for you. Every day. Isn't it great ;-) Remember: if it's not checked, it's not there. Paper is patient, automatic checks are not. Really, make your checks and reviews automatic. It's important, like your daily vitamins.

Automated testing is only one aspect of checking your code, albeit the most popular one. The test infected community already knows that if it's not tested, it's broken.. So next to testing you need to check other aspects of your code, like coding conventions. Usually these include whitespace policy, formatting, naming and other design idioms. Coding conventions cover a much broader area than most people think. They are not only about naming. They are also about higher level boiler plate code, e.g. how to handle transactions, how to access the database, how to log, how to handle exceptions, etc. These things are project specific and depend on the overall architecture.

Slacker Vandalism? End Work Check It!
All projects have some sort of coding conventions. But are they complete? Are they documented? Do developers comply with them? Unlikely. They need to be documented and even more need to be checked automatically. Probably most of your rules are not checked. It's time to write them down and define some concrete checks for them. Most tools and even some IDEs ship with basic rules for simple things like whitespace, naming or common coding idioms. These are perfect for a start. Start small. Use a few rules. You can always add more later. The limit of what you can check depends entirely on your determination: design rules, layering, modularity, architecture, code coverage and documentation and much more.

The problem is that rule enforcement provokes opposition. People don't want to leave their cosy comfort zone. Discussing and agreeing on a new coding convention is not a problem. But adding a new rule to already checked coding convention might be a fight. You have to convince developers to accept it. You have to argue with management for time to remove rule violations in legacy code. You have to struggle through, especially when you're only a grunt. Small steps are crucial. Don't press on it too much. If there is opposition, offer to drop the new rule. Make it look like there is the option of not having it. This enables discussion. (Of course that's not an option and you are not really offering it, but people like to have options to discuss about.) As soon as some rules showed their value, developers will vote for them if you oppose them, be the devil's advocate.

Automatic!
So let's finish this rant about human nature. I'm a slacker. Most likely there are some more in our trade. We must accept that. we are lazy. We make mistakes. Sometimes we are weak. That's normal. We just have to be aware of it. So be paranoid. Don't trust anyone. Automate anything that you might screw up. (Robustness #2) Automatic checks are your safety net. They help you avoiding making the same mistake twice. If there is a bug in your code, create a unit test to ensure the bug stays fixed. If you have inconsistent formatting, add format checks to your daily build. If you notice wrong usage of a design idiom during a review, create a custom rule to enforce proper usage. If ... well you get the point.

All this leads to the 2nd Law of Code Quality - Automatic Checks to fight slackerism.

14 March 2009

Law of Code Quality: Consistency

Fall MixtureImagine you have some MDBs (Message Driven Beans). You want to get rid of them because they are still EJB2 and they suck. You want to use Spring's JMS capabilities instead. Sounds quite good. But after some time, due to some budget problem or pending dead-line, you stop converting the old stuff "because it does not add value" (and which is almost guaranteed to happen). So you end up having MDB plus Spring JMS mixed throughout the code and maintenance people have to know both of them. It's difficult enough to know one of them, but now you end up having both solutions messed up on the long run.

Copy & Paste
In brownfield development you always have some code before you start. The look, quality and design of this code is very important for further developments. Extending complex legacy stuff often involves a copy-and-paste style of coding. (If this is preferable is a story of its own right but will not be discussed here.) In the context of maintenance I consider copy and paste a good habit because this way the existing conventions and patterns are obeyed, even if they are not documented. Of course the positive effects of copy paste are only achieved if the "right" piece of the system is copied. Like reference solutions in generator development, aka templates (Link MDSD Generator), the copied piece has to be of highest quality, i.e. according to conventions and guidelines defined for the application.

Implicit Conventions and Uniformity
If the guidelines and conventions of some software are not written down properly, the only documentation is the code itself. Even if there is decent documentation, capturing all aspects of software development is at best very difficult or can't be done at all. There are always some implicit conventions that are only available in the code. The more an application is uniformly satisfying these conventions and designs, the more pieces of it serve as "safe" templates for further extensions and modifications. This helps new members on the project to find their way around. And good (maintenance) developers see these implicit conventions in surrounding style and patterns, adapt to them and work according to them. Uniform code makes it easier for them to adapt to the new code base and "get to speed".

Broken Window AboveBroken Windows
As the pragmatic programmers toughed us in rule 4, broken windows are causing problems in real as well as virtual life. If the conventions are visible and clear, who would dare to stand out in breaking them? On the other hand, if e.g. a piece of code is formatted in three different ways, there is no shame in introducing a new style. People tend to stick with the things they know, because it is faster and feels more secure. (And usually we think it to be superior to things we do not know.) This leads to patchworks in the code. This mixture gets maintained (read copied) from time and time and all the broken windows get spread throughout the code base, the differences live on and grow.

So I postulate the 1st Law of Code Quality - Code Consistency. This applies to source layout, naming and other coding conventions, typical code fragments (also called idioms, most of the time some boilerplate code), design concerns, layering, architecture, used libraries, technologies etc. Consistency in the code is the most important issue. Failing to have a consistent code base will cause all the troubles known from mixed designs, mixed technologies, making it difficult to maintain and get new people into the team.

Living with Changes
Of course we have to change things again and again. It's easy to have consistency in simple conventions, like formatting, just use Eclipse format on the whole source tree and put Checkstyle into your build. Short code idioms like getting a database connection are more difficult. These are rarely documented, but once you have them unified, a Ruby or Groovy script can do almost any syntactical change to your code using powerful regular expressions. More complex changes, e.g. replacing EJB 2 with something else, is more involved. However just don't make the mistake to leave the old stuff as it is. No excuses about small budget, needed retest, pending dead-lines and such! If you can't convert the old stuff, leave it as it is. If you are too "weak" to change it, you earned staying with it. Bringing in new technologies needs a strong plan, better a script, to convert everything existing to the new style. All remaining code has to be changed to use the new technologies as they are supposed to. Typical idioms and best practices of that technology should be obvious at the end. You don't want a Java program to be coded like old C, a few classes with lots of static, monolithic methods, static data etc. The same is true for any refactoring.

A Brighter Future?
In keeping your code base consistent you need help. Especially in the beginning we need someone who flags new inconsistencies, reminds us of conventions. Use static code analysis as soon as you have identified a consistency target. (That might be as simple as grep *.java.) The proper (consistent) way to do something should be enforced from day one. Unfortunately documenting it is not enough. For layout use tools like Checkstyle. Other conventions and boilerplate code can be checked with tools, that support custom rule definition, e.g. Findbugs or PMD. Nowadays many tools come with a large number of base rules that cover common stuff. Most likely some are of use. Modularity and layering is enforced with reference checking, e.g. Macker or SonarJ. With some fantasy (and enough computing power at your build machine) you can create quite sophisticated checks.