hy are so many articles and books written about development methodologies? I believe it’s because improving the efficiency of a development team is a never-ending battle. Technology keeps changing and teams must adapt or die, but adapting is time-consuming. Customers need fast development cycles with higher quality. With the advent of offshore outsourcing, developers now come from different countries, in different time zones, with different cultures. And as everybody does, developers tend to concentrate on tasks that are interesting and usually not the most productive.

In this complex development world, I have encountered very interesting issues as a team manager and developer. This article presents a formalized list of the most troublesome facts of development I’ve come across, and it offers some suggestions for handling them. (The original version of this listing was posted on my blog.)

Fact 1: Maintenance cost is the biggest enemy of efficiency.
Maintenance cost pressure naturally leads development teams to low efficiency. Maintenance costs tend to be more or less proportional to the number of lines in your code, especially when it is not properly tested. As the number of lines increases, the team discovers that the number of bugs tends to grow for internal reasons (bugs) or external ones (not really bugs but need to be corrected anyway).

An external reason may be a change in the way the code is used. For instance, your code works perfectly with JRE 1.4, but JRE 1.5 is here and the show must go on. Your code must run with it, and if it does not, you have a bug from your customer’s point of view.

Because the perfect code does not exist, you must think differently. You must take maintenance into account even before you code. Think about how to test before thinking about how to code. Moreover, you must determine specifically how to have your servers test your code for you?a test that cannot be automated is only half a test. Then the team will be able to test its own code effortlessly in any new environment and extend the tests quickly and efficiently.

You must find a way to enable every line of code to be automatically (and fully) tested. This is not just an issue of unit testing, because unit tests cannot test everything. In fact, tests are more efficient when they are separated into layers (more on the layers later).

Fact 2: Developers don’t like to test, and they don’t test their code the way they should.
Code that works in one place is supposed to work every place. This is absolute nonsense in real life, but developers actually live in a virtual world where everything is supposed to follow norms without encountering bugs. For instance, a J2EE application that works in JBoss is supposed to work seamlessly in WebSphere as well. (LOL) Therefore, developers need to test their code on all targeted platforms with minimum effort. Otherwise, the tests just won’t be run. If they are, they won’t be run properly. The way a developer generally works is code, compile, and forget.

I believe you need a framework to run the tests on every platform and store the results in a database. To sum up the process: write tests in any language, automatically test on every targeted platform, and store the results. Then you can review the history of the test results on any platform for every version of the code.

I know some tools espouse those features. For example, the BuildBot open source project seems to be working on it, at least in the near future.

Fact 3: When a bug is discovered, it takes much more time to isolate it than to correct it.
You must react quickly when a developer modifies code and checks it into the code repository. Test the committed code promptly, so that if a test fails, the developer remembers what he or she just did and where the bug originated.

If the developer doesn’t remember, he or she can look at the code history and see the differences between the two versions. The bug can be isolated in the code that changed. The developer won’t have to spend hours to find the bug.

Fact 4: It takes time for a developer to switch from one project to another.
Often, setting up a development environment for a project is difficult. It may be nearly impossible without very good knowledge of the project. When it takes too much time to set an environment up, developers tend to stay in the same environment for too long. A developer should be able to switch from one project to another without technical difficulties. The “project culture” is difficult enough.

I advocate using project management tools like Apache Maven. A project described in Maven tends to be very easy to set up in Eclipse. For Java projects, just download your code from SVN, type “maven eclipse”, refresh, and it works. All your unit tests are ready to run against your code. Whoa! A whole project set up in less than 30 seconds.

For technologies other than Java (Python, C, etc.), Maven with standard plug-ins is not enough.

Fact 5: Bugs increase as the number of developers increases.
Without the proper methodology, the number of bugs tends to grow with the number of developers. So the quality decreases with the complexity of the project. One must work against this tendency with the simple principles of transparency, code reviews, and automated tests.

Transparency ensures that anybody can look at the code and determine who has modified what, when it was modified, and if possible, why it was modified. Transparency puts more pressure for quality on the individuals in a team.

Code reviews enable any developer to discover and prove a bug’s existence. It’s amazing how powerful they are for improving quality.

Fact 6: Tests have several layers.
Writing good tests is a tough challenge. For example, writing unit tests on existing code is very difficult, and they cannot test everything. Functional (QA) tests are powerful for challenging your application, but they usually run slowly and fail to give enough information about the cause of the problem.

So you must divide and conquer. Running huge tests is useless if the fast ones do not pass. Create test layers from the simplest to the most complex:

• Level 0: Unit tests on the developer’s own environment
• Level 1: Unit tests on a build server, started on (almost each) commit
• Level 2.1: Integration tests, where you try to have several modules connected together (You replace the mock objects with the actual modules you created. For instance, you would use an actual XML parser instead of a mock one.)
• Level 2.2: Integration tests, with access to the system (socket, disk, etc.) (Here you can test the actual access to the system.)
• Level 3: Functional tests, where you test the visible part of your application (actual functionalities seen by the user) (This layer is certainly not enough.)
• Level 4: Performance tests
• Level 5: Customer use (OK, this is not a test layer, but it certainly is a source of surprising new bugs.)

You need to start the more complex tests only if the simple ones pass.

The motto of this methodology is a bug should never appear twice. Therefore, for every bug you find, you should create a test that guarantees that it will never happen again. Sometimes the test is a unit test (which is better), sometimes it is an integration test.

The sooner you get the error, the sooner you correct the issue. So you’d better begin running tests at Level 0 rather than Level 3, and the code quality tends to increase naturally with the number of tests.

Fact 7: You must have a restricted number of languages.
If you choose the best tool for every task, the number of technologies you use tends to grow fast. If you always take the quickest path to the short-term goal, the complexity of the system grows. Therefore, you should sometimes choose not to use the best tool. Using only one or two languages in a team is better than using more, because training people on new technologies costs a lot. For instance, Ruby on Rails certainly is a very productive tool but it’s one more language and technology. One should consider the whole cost before deciding to develop that new web site with this technology. This policy certainly does not apply if your main activity is creating web sites, because in that situation Ruby may become your team’s main language.

In my opinion, choosing a limited number of programming languages is the way to go. You must also assess the technologies in use in your company. For instance, is it very productive to use four Web services frameworks? One likely is enough.

Fact 8: Tracking change and innovation is necessary but difficult.
It is very easy to miss out on a tool that can save you a lot of effort. If you don’t know about it, you can’t try it. Tracking changes and innovation in development frameworks and methodologies prevents this. Thanks to this type of tracking, I discovered almost all the tools I now use: Eclipse, Maven, Tomcat, Apache, LDAP, CruiseControl, TestNG, Python, etc. Most of them saved me weeks, if not months of development time.

The difficulty is in choosing the tools (it takes time to choose) and preventing your team from having too many technologies at the same time. Sometimes, it’s better to wait a bit to capitalize on one technology before implementing a new one.

Another challenge is choosing the new technologies to evaluate. It’s not easy to choose between all the competing frameworks and evaluate only one or two.

Fact 9: To stay productive, a developer must not switch activities too often.
A developer takes 7 to 15 minutes to become efficient at resolving a complex problem. Switching from one activity to another (phone, email, etc.) interrupts the developer’s progress. For instance, a team where the developers answer tech service calls every 20 minutes is catastrophic. One must not allow users to call the developers directly.

Instead, one must set up a tool to gather questions, bugs, nice-to-haves, and requests, so that the developers can concentrate on their jobs. Tons of available tools can gather the demands, including Mantis and Jira.

Remember, a developer is not a tool; a developer is a human being. Why do managers keep asking developers to remember?store?their demands? Aren’t the managers smart enough to write their needs down?

Fact 10: Defining the architecture is as important as coding.
Coding without architecture is like driving at night without lights. You may arrive at your desired destination, but the journey is dangerous and slow. Architects must review the organization’s products often and anticipate problems. Without the architect’s input, the complexity of the product grows each time a new functionality is added.

Even when the coding was all right, projects that I know took the fastest path to the result usually went wrong. This is where an architect should have provided an overall view of the product architecture and proposed alternative coding paths.

Fact 11: Developers have different priorities than product owners do.
A developer tends to work on “fun” tasks, while the product owner wants something useful for his or her clients. The development process must lead to a better approach, where the developers find it exciting to work on annoying issues.

From what I’ve seen, Scrum seems to be a very nice method to use for this. It proposes a process in which the developers commit themselves to delivering results on time, and pride is a strong and powerful motivation! While developers are challenged to deliver on time, managers are challenged to think twice before requesting features that will be of no use. Have a look. Scrum seems really great.

Fact 12: Coding conventions are efficient; they must be imposed.
Coding conventions free developers from needing to understand the form of the code, allowing them to concentrate on the code’s core meaning instead. In the teams I’ve worked in, we ended up setting conventions for the directories where we put the code, for the file names, and for the form of the code itself (the naming of the classes, methods, variables, etc.). Many other conventions can be proposed.

Some SVN add-ins even forbid commits when the code does not follow the conventions. Great!

Nip Efficiency Issues in the Bud
Most development processes and methodologies aim to address efficiency issues. This article pointed out some of the development facts I’ve observed that are at the heart of these issues. I hope you can learn from my experience, and while no one solution is a silver bullet, I recommend having a close look at Scrum because it enhances the commitment of every actor in the development chain.