Slipping deadlines by months and sometimes by years, exceeding estimated cost by many times, defects in production, and projects that never complete are common issues in the software industry.

In my view this is due to the softness of software. One definition of “Softness” is: Easily molded, cut, or worked1. Software can be easily changed; it’s not like changing the design of a house after it’s built. If that’s what you think, think twice. Is it really easy to change software?

Changing current code of software does not make a mess like tearing down a section of a house does. However, changing software takes time, and that means it costs to change software. If the one who changes the code is not the original writer, it may take more time to understand the logic of the previous code before doing any changes. What’s more, changing code could introduce new issues that may or may not get caught before shipping the product. If caught, these would require more re-work until all issues are fixed. Sometimes, this becomes a never ending cycle of fixing-introducing more issues - fixing.

Therefore, is the softness of software just an illusion caused by the apparent cleanliness of the demolition and rework of software? What if software was not soft, but once it’s coded, it can’t be changed? Would this have resulted in more rigorous ways of perfecting initial requirements, or betters ways of modeling software? If software was not soft, it would have meant that once the product gets out of development, if it does not pass quality control, it would be rejected, and not sent back to developers for bug fixes, resulting in developers having to re-build the product or parts from scratch. Would this have resulted in more disciplined development methods that ensure what’s released from developers are of production quality?

I don’t have answers to these questions. However, thinking of software as “not soft” may open up new avenues for perfecting software engineering.

1. softness. Dictionary.com. The American Heritage® Dictionary of the English Language, Fourth Edition. Houghton Mifflin Company, 2004. http://dictionary.reference.com/browse/softness (accessed: November 14, 2008).

How to hire good software engineers, is a problem which I have had to face for a long time. How can you really gauge the ability of a candidate during an interview? This is further made difficult by the fact that there are no standard requirements to be met for a software engineer. Anyone who could learn a programming language and do a bit of programming can call (and they do) himself or herself a software engineer.

A few months ago, I read the book: Peopleware, Productive Projects and Teams by Tom DeMarco and Timothy Lister. In Chapter 15, the authors draw an analogy between hiring a juggler for a circus and hiring an engineer or designer. No one would hire a juggler based only on an interview without seeing him perform. The same should be applicable to engineers. Yet how many engineers (in our case software engineers) are hired based only on an interview?

In the past, I have suffered due to bad hiring decisions at interviews. In order to prevent this, recently we decided to make it mandatory for the each candidate to go through a pre-interview test that consists of an exercise to write working code on a computer to solve a given problem, and to come up with designs on paper for a few given software problems. Today’s the first time we tried it. So far two candidates have taken the test, and we can already see that it gives us much more information about their skills than an interview.

As someone who plays multiple roles in life (including many roles at work), for some time, I have been trying to find the ideal productivity tool. Having experimented with many such tools, I am yet to find the perfect solution. However, the good news is that Chandler 1.0  has finally been released. Today I downloaded and installed Chandler 1.0.2 for Linux on my laptop running Ubuntu 8.04.

Chandler is an open source note to self organizer which I had previously tested a few months back only to find that its performance was inadequate for my purposes. I also sometimes ran into bugs that made it unusable. So far, I have been able to use 1.0.2 without any major issue. Performance has been acceptable, but the real test is when I use it for about a week and enter 100s of records. As an advocate of open source software, I hope the product has finally become usable.

Apart from Chandler, I have used two other excellent productivity tools. Both are hosted services that you can access over the Internet. One is Remember the Milk which has a great way to manage lists. The tagging is also quite powerful. The other is Todoist which can have any number of levels in a task hierarchy making it well suited for managing projects.

Getting started with Chandler on Ubuntu
The binary available on Chandler website was built for Ubuntu 7.10. However with the help of a discussion thread, I managed to get Chandler 1.0.2 installed on Ubuntu 8.04.

Related Posts: Dreaming in Code, Don’t Worry be Crappy, Building Software - Engineering or Art?

I am getting closer to the end of the book: “Dreaming in Code”. Chapter 11 is titled “The Road to Dogfood”. Eat you own dogfood is a concept that recommends using your own products. Chandler team had taken more than three years before they had a product that could be shipped or at least could be used internally. In my view, they were too ambitious in trying to ship a great product in the first attempt.

This reminds me of the commencement speech made by Guy Kawasaki at the Carnegie Mellon Silicon Valley graduation last summer. One of his advice was “Don’t worry be Crappy”. The following is taken from his blog:

An innovator doesn’t worry about shipping an innovative product with elements of crappiness if it’s truly innovative. The first permutation of a innovation is seldom perfect–Macintosh, for example, didn’t have software (thanks to me), a hard disk (it wouldn’t matter with no software anyway), slots, and color. If a company waits–for example, the engineers convince management to add more features–until everything is perfect, it will never ship, and the market will pass it by. - Guy Kawasaki.

On hindsight, a similar vision coupled with an agile development methodology would have helped Chandler. An agile methodology like Scrum or Exreme Programming would have made the team focus on releasing the most essential features of the product early without waiting for all the functionalities to be in the product. This would have been ideal for Chandler as its vision was only defined at a very high level. Early releases of a usable system with less features would have expedited the process of fixing bugs as well as enabled the feature definitions to take shape with more early feedback from the user community.

Today, I spent a few hours going through our bug list. This is a process that is important but not that enjoyable.

Every software project must have a bug tracker (This is #4 in The Joel Test). The purpose of a bug tracker is to track all identified issues of the system in a central place. In David Allen’s popular Getting Things Done (GTD) methodology, this is the step: collect.

A popular tool like Bugzilla could be used as the bug tracker.

For us at OrangeHRM, it’s the bug tracker on SourceForge. The main process in which bugs get reported is QA testing. However, bugs could also get reported by developers, customers, or any other user of the software. Therefore, it is inevitable the bugs reported on the tracker would not be well organized. I have observed the following issues in bugs reported by various sources to the bug tracking system:

1. The mentioned priority of the fix is not necessarily the correct priority in the context of the project.
2. Feature requests and product enhancements get logged as bugs.
3. Misunderstood expected behavior reported as bugs.

Therefore, it is important that we implement the second step (also similar to GTD) of processing without letting developers directly pick bugs from the tracker. Ideally processing could be done by the project manager or whoever that best understands the project situation and is the decision maker of the project. Each bug should be analyzed and the following should be determined:

1. Is it really a bug that needs to be fixed? If not the tracker should be updated with the decision. Sometimes it may be required to consult the reporter before making a final decision.
2. What’s the real priority of the bug in the context of the project.

Once the real bugs and real priorities are identified, they should be added to the project planning tool used in the project. We use XPlanner for project management as our process is derived from Extreme Programming.

I usually create a story named “Bug Fixes” in each iteration in XPlanner and add all bugs as tasks under the story with its priority prefixed. (E.g. [0] - Immediate, [1] - High, [2] - Medium, [3] - Low, [4] - Trivial). XPlanner automatically sorts the list based on the prefix used such higher priority bugs appear on the top of the list. I ask the developers to pick bugs from the top of the list; fix them; and move down in the list. I repeat this process frequently such that when new higher priority bugs are added, they appear at the top of the list.

This process assures that we don’t miss any reported issues, and that we address the issues based on the real priority and we don’t waste time on lower priority tasks.

Chapter 10 of Dreaming in Code is titled: “Engineers and Artists”. This chapter discusses the age old debate about whether building software is an engineering discipline or whether it’s more like a creative process similar to producing art.

In my view it’s both an engineering discipline as well as a creative process. However, building software should primarily be an engineering discipline. I use the word “should” as in practice, this has not been true. This is both based on my personal experience as well as what is discussed in the book. The following quote from the book is a good way to put it.

“If builders built houses the way programmers build programs, the first woodpecker to come along would destroy civilization - Gerald Weinberg”

My argument that building software should primarily be treated as an engineering discipline is based on the comparison of impact of faulty software and not so good art. Although artistic creativity is important, if the software does not perform as intended, there could be serious consequences.

The impacts of 20 famous software disasters range from lives lost and endangered to millions of dollars lost. Although the impact of failure of all software would not be as catastrophic as this, all software failures would affect lives of people one way or the other. Therefore, it is paramount that building software is treated primarily as an engineering discipline.

The challenge is to perfect building software as an engineering discipline, which is not as mature as other engineering disciplines.

Images taken from en.wikipedia.org

Related Posts: Dreaming in Code, Don’t Worry be Crappy , In Search of an Ideal Personal Productivity Tool