Test Driven Development (TDD) is great. When I started unit testing and TDD with JUnit about six years ago, it was a relatively new concept. I was impressed by the way it helped development, and led to developing better code. However, it’s only now that I realize how little I knew about unit testing back then.

I can remember an XP project where we developed an enterprise application which had two layers of session beans, and entity beans that got persisted in an Oracle database. We used to have unit tests (that’s what we called them) testing all layers, accessing the session beans and propagating all the way down to the entity beans and to the database. As a result of testing all layers, running the whole suite of tests took us about one and a half hours. We were not convinced that our so called unit tests added any value to the project as it took us so much time to use them properly. So we ended up running only groups of tests together, and not the whole suite.

Recently I stumbled upon an article by Michael Feathers that literally describes the same issue we faced. He lays down the following rules:

A test is not a unit test if:
1.It talks to the database
2.It communicates across the network
3.It touches the file system
4.It can’t run at the same time as any of your other unit tests
5.You have to do special things to your environment (such as editing config files) to run it.

The main takeway from this is that the purpose of a unit test is to test the functionality of a particular unit of code. It need not test all the layers of functionality upon which the class being tested depends (e.g. entity beans, databases etc.). In our case, we could have used mock objects to separate the business logic from the underlying persistence layers. This should have allowed us to simply test the business logic of our classes using unit tests without effectively writing comprehensive but bulky functional tests. We could have defined our classes with dependency injection in mind. Dependency injection is the practice of writing testable code by separating code for object creation and business logic, and passing all dependent objects through the constructors of objects.

The Google tech-talk below provides a great explanation of this concept:

So, just because you use an xUnit framework and write automated tests for your classes, they don’t necessarily become unit tests.

A topic which often surfaces in discussions about XP is, whether Stories in XP are the same as Use Cases under a different name. Apparently, they seem to be the same. However, if you analyze a little deeper it’s possible to identify subtle differences between the 2 approaches.

In my view, the main difference is their purpose of existence. The main purpose of a user story in XP is to identify the scope of the project as early as possible and to track project progress. While a use case tries to capture requirements at the required level of detail. The XP approach is to have the story defined in simple words and to get more details from the customer before development starts.

User Stories in XP

1. Purpose: Estimation, identification of scope early, and tracking progress.
2. Characteristics: Simple; any customer visible functionality.
3. Approach: Meet customer face-to-face and get details before implementing.
4. Suitability: Projects that need early implementation.

Use Cases as used in the Unified Process

1. Purpose: Describe all tasks that users will need to perform with the system.
2. Characteristics: Can be scaled to have many details; functional and behavioral requirements.
3. Approach: Gathered mainly initially (inception & elaboration phases).
4. Suitability: complex large projects.

Today I watched a talk from the Google Tech talk series by Elisabeth Hendrickson on YouTube. This talk provides a good overview of testing in an agile project. She talks about:

• Agile methodologies in general
• Test driven coding
• User stories and acceptance test cases
• Documenting test cases
• Testing when all features are not complete.

Watch Video:

To understand this talk, it requires no previous knowledge of agile methodologies as the presenter starts with an explanation of agile methodologies.

We adopt some of these practices at OrangeHRM, which uses a hybrid process that includes chosen practices from both agile and plan-driven methodologies. I am planning watch the whole video again with the team and have a discussion. It’ll be interesting to identify how we could further improve the way we do things.

The search for the so called “silver bullet” or the cure-all in software development has been there for ages. Many methodologies were developed, each claiming to be the solution. However, still there is no one universally accepted methodology that can solve all the inherent issues in software development. While all these approaches try to find an ideal development methodology for success in software projects, an important variable is usually forgotten. That is, the role of a customer in the success of a software project.

A good customer has an enormous impact on the success of a software project. A bad customer may fail a project which has the best engineering and project management skills.

The acronym CRACK ¹ represents 5 qualities of a good customer (or customer representative).

• Collaborative, and willing to work with the project team toward the same goals, initially with requirements, and later with deployments and any integrations.
• Representative of the end users of the software, and is able to speak on behalf of them.
• Authorized to make decisions that have a significant impact on the project. If the customer is not authorized, there could be delays in proceedings until the necessary approvals are received.
• Committed, willing to see the project to the end, and willing to put effort to eliminate inevitable hurdles that software projects face.
• Knowledgeable about the organization and the requirements.

Here’s a comparison of two customers I have worked with in the past (Each quality is scored on a scale from 1-5, 5 being excellent)

Project #2 continued to be in an uncertain state with periods of inactivity and late change requests leading to indefinite delays in project completion. Although Project #1 had its share of issues, most the issues were related to engineering (e.g. performace), and with a little bit extra work, the team was able to overcome them.

It’s good to know what sort of a customer you’re dealing with before you start a software project. Although most of the time we don’t have the luxury to pick and choose customers, knowing the customer characteristics makes is possible to do the necessary precautionary process adjustments.

[1]Boehm, Barry, and Richard Turner. Balancing Agility and Discipline. Addison-Wesley, 2006.

I am getting closer to the end of the book: “Dreaming in Code”. Chapter 11 is titled “The Road to Dogfood”. Eat you own dogfood is a concept that recommends using your own products. Chandler team had taken more than three years before they had a product that could be shipped or at least could be used internally. In my view, they were too ambitious in trying to ship a great product in the first attempt.

This reminds me of the commencement speech made by Guy Kawasaki at the Carnegie Mellon Silicon Valley graduation last summer. One of his advice was “Don’t worry be Crappy”. The following is taken from his blog:

An innovator doesn’t worry about shipping an innovative product with elements of crappiness if it’s truly innovative. The first permutation of a innovation is seldom perfect–Macintosh, for example, didn’t have software (thanks to me), a hard disk (it wouldn’t matter with no software anyway), slots, and color. If a company waits–for example, the engineers convince management to add more features–until everything is perfect, it will never ship, and the market will pass it by. - Guy Kawasaki.

On hindsight, a similar vision coupled with an agile development methodology would have helped Chandler. An agile methodology like Scrum or Exreme Programming would have made the team focus on releasing the most essential features of the product early without waiting for all the functionalities to be in the product. This would have been ideal for Chandler as its vision was only defined at a very high level. Early releases of a usable system with less features would have expedited the process of fixing bugs as well as enabled the feature definitions to take shape with more early feedback from the user community.

Today, I spent a few hours going through our bug list. This is a process that is important but not that enjoyable.

Every software project must have a bug tracker (This is #4 in The Joel Test). The purpose of a bug tracker is to track all identified issues of the system in a central place. In David Allen’s popular Getting Things Done (GTD) methodology, this is the step: collect.

A popular tool like Bugzilla could be used as the bug tracker.

For us at OrangeHRM, it’s the bug tracker on SourceForge. The main process in which bugs get reported is QA testing. However, bugs could also get reported by developers, customers, or any other user of the software. Therefore, it is inevitable the bugs reported on the tracker would not be well organized. I have observed the following issues in bugs reported by various sources to the bug tracking system:

1. The mentioned priority of the fix is not necessarily the correct priority in the context of the project.
2. Feature requests and product enhancements get logged as bugs.
3. Misunderstood expected behavior reported as bugs.

Therefore, it is important that we implement the second step (also similar to GTD) of processing without letting developers directly pick bugs from the tracker. Ideally processing could be done by the project manager or whoever that best understands the project situation and is the decision maker of the project. Each bug should be analyzed and the following should be determined:

1. Is it really a bug that needs to be fixed? If not the tracker should be updated with the decision. Sometimes it may be required to consult the reporter before making a final decision.
2. What’s the real priority of the bug in the context of the project.

Once the real bugs and real priorities are identified, they should be added to the project planning tool used in the project. We use XPlanner for project management as our process is derived from Extreme Programming.

I usually create a story named “Bug Fixes” in each iteration in XPlanner and add all bugs as tasks under the story with its priority prefixed. (E.g. [0] - Immediate, [1] - High, [2] - Medium, [3] - Low, [4] - Trivial). XPlanner automatically sorts the list based on the prefix used such higher priority bugs appear on the top of the list. I ask the developers to pick bugs from the top of the list; fix them; and move down in the list. I repeat this process frequently such that when new higher priority bugs are added, they appear at the top of the list.

This process assures that we don’t miss any reported issues, and that we address the issues based on the real priority and we don’t waste time on lower priority tasks.