s we near the fifth anniversary of the Agile Manifesto, the shift from Waterfall software development to Agile methodologies and practices continues to gain momentum. At this year's Agile 2006 Conference in Minneapolis, MN, there was much discussion about whether Agile had "crossed the chasm" yet (a la the book "Crossing the Chasm" by Geoffrey A. Moore), but there was no debate about whether Agile had reached the chasm. Because proper application of Agile techniques and processes is demonstrating success in greater numbers all the time—especially in the most difficult situations—interest in adopting Agile is at an all time high.

But once an organization decides to adopt an Agile development regimen, there is still plenty of difficult research and decision-making to be done. In particular, navigating the various Agile methodologies available can be confusing to the Waterfall-minded. The most popular Agile methodologies in use today appear to be Extreme Programming (XP), Scrum, Feature Driven Development (FDD), Lean Software Development, Agile Unified Process (Agile UP or AUP), Crystal, and Dynamic Systems Development Method (DSDM). This two-part article will briefly explore the Agile value system, and analyze how these methodologies manifest these values. Particular care will be taken to highlight the similarities and differences between each methodology and we will conclude with a brief analysis of the varying business contexts that may make one methodology more appropriate than others. (Read part 2 here.)

Understanding Agile
Agile software development is a style of software development characterized by an emphasis on people, communication, working software, and responding to change. Its principles are summarized in its seminal declaration, the Manifesto for Agile Software Development. One of the most notable characteristics of this manifesto is that it is worded as a value statement and not a concrete plan or process. This underscores Agile's nature as a style of development consistent with the values laid out in the manifesto.

All Agile methodologies engage in an iterative workflow and incremental delivery of working software in short time-boxed iterations. An iteration is essentially a small release of software. Generally during each iteration many activities will occur in parallel, such as requirements, coding, and testing. Iterations are typically a fixed length (although this length varies between the methodologies) and thus are referred to as time-boxed. The time allocated to each iteration is sometimes referred to as a cycle time.

Agile methods also differ in the activities and artifacts that they produce. Methodologies that have many artifacts (or work products) and steps along the way are referred to as higher ceremony. Conversely, methodologies with less emphasis on these are referred to as lower ceremony. The number of people involved and the number of "sign offs" also helps to define the amount of ceremony around a particular methodology. It is important to strike an appropriate balance between the two in any given context.

Extreme Programming
Extreme Programming (XP) is one of the more well known Agile methodologies. As its name suggests it is a programmer-centric methodology that emphasizes technical practices to promote skillful development through frequent delivery of working software. Although XP (and Agile methods as a whole) is often characterized as less rigorous then traditional techniques, this could not be farther from the truth. XP got its name when its founders asked the question, "what would happen if we took each technique/practice and performed it to the extreme? How would that affect the software process?" An example of this is the practice of code reviews. If code reviews are good, then doing constant code reviews would be extreme; but would it be better? This led to practices such as pair programming and refactoring, which encourage the development of simple, effective designs, oriented in a way that optimizes business value. In truth, full adoption of all of XP's practices requires a high level of discipline, teamwork, and skill.

One of the characteristic distinctions between XP and other methodologies is its cycle time and level of ceremony. XP recommends very short iterations between one and four weeks. XP is also a very low ceremony methodology. Minimal artifacts in an XP project include story cards, code, and unit tests.

But XP is most commonly known for its technical practices. At the heart of XP are four core values: communication, simplicity, feedback, and courage. From these values 13 practices are derived. Theses practices are:

1. Planning Game: Plan work incrementally.
2. Small Releases: Release as quickly as possible to increase time to market, and get feedback as soon as possible.
3. Metaphor: If possible, define a metaphor for the system being developed. For example, the shopping cart metaphor is widely used to describe an online ordering system.
4. Simple Design: Use the simplest design that will work for the functionality (user story) being implemented. Do not design for things that may never actually be used.
5. Testing: Test everything, and try to automate the testing if possible.
6. Refactoring: Instead of designing the entire system up front, design as you go, making improvements as needed. Change the implementation without changing the interface to the functionality, and use automated testing to determine the impact of the refactoring.
7. Pair Programming: Programming in teams of two (or three) allow for a discussion to occur in real-time that addresses requirement, design, testing, and programming concerns.
8. Collective Code Ownership: Anyone on the team can make a change to any code at any time.
9. Continuous Integration: The entire code base is constantly being rebuilt, and retested in an automated fashion.
10. Sustainable Pace: Ideally, team members do not need to work more than 40 hours per week to meet project deadlines. Burning the midnight oil is chunked by management in favor of consistent, predictable, repeatable delivery.
11. Whole Team: The team functions as a whole. Members are encouraged to be more generalized than specialized. Learning about all technologies and requirements is encouraged.
12. Coding Standards: In order to maximize communication, coding standards are defined by the team, and used to ensure consistent coding practices.
13. Onsite Customer: Having constant and direct access to the customer allows the team to work at the fastest possible speed.

These practices support one another, as illustrated in Figure 1, and thus care should be taken when modifying any of them, or deciding not to include one or more of these practices in a project.

Figure 1. XP Practices. The practices in XP support one another and thus care should be taken to understand these relationships when modifying XP.

Another distinction between XP and other Agile methodologies is its approach to requirements gathering. The primary requirement artifact in XP is the user story. Visibly, a user story is nothing more than a note card with a short description on it. But user stories actually are comprised of the card (reminder of promised functionality), conversations between developers and requirements donors, and tests (all types: unit, integration, acceptance, etc.).

There are two primary types of planning activities that are referred to in XP parlance as planning games: the release and iteration planning games. And each planning game is characterized by three phases: exploration, commitment, and steering.

Release planning begins with the customer writing story cards and prioritizing them. Programmers then estimate the stories from which a velocity can be derived. Velocity describes how much work the team can accomplish in any given time frame. The customer then chooses a scope for the release based either on desired functionality or release date. The steering activities for release planning occur at iteration boundaries when progress towards the planned release date can be tracked and adjustments can be easily made.

Iteration planning follows a similar pattern to release planning. Each iteration begins with developers taking stories and breaking them into tasks. Programmers accept responsibility for tasks and estimating the tasks that they are responsible for. Each programmer's load is compared to their historical performance to determine whether anyone is overcommitted and to allow the workload to be balanced across the team. Throughout the iteration programmers partner to complete tasks by writing unit tests and code. Periodically throughout the iteration a member of the team checks on the progress of the team and communicates this information to all team members so adjustments can be made.

There are a small number of roles on an XP project. These include:

• Customer—The customer creates and prioritizes the stories that need to be implemented. Because the customer sets the priorities she controls what will be delivered in any given release and can control the release date by adding and removing stories.
• Programmer—Programmers collectively estimate stories, but individually accept responsibility for tasks which they estimate, write tests, and code.
• Coach—The coach monitors the software development process, mentors team members on XP process and techniques, and focuses the team's attention on potential problems or optimizations.
• Tracker—The tracker monitors the progress of the team and alerts them when adjustments to the schedule or a re-balancing of tasks might be required.

Scrum
A common misconception is that Scrum is an acronym: SCRUM; not so. Scrum is named after the Scrum in Rugby, which is a mechanism to restart the game after an accidental infraction - such as the ball going out of bounds. It's the general idea of a team huddling together to move the ball toward the goal.

Scrum is a project management framework for managing Agile projects. Its primary goal is to deliver software that, after each and every iteration, provides the highest business value. Scrum is based on a 30-day iteration called a "Sprint." Technically Sprints can be either two or four weeks, but the generally accepted default is usually four weeks.

A fundamental Scrum principal is that project teams should be self organizing. This means that team members don't follow a prescriptive plan or set of tasks, but organize themselves initially based on the goals for the Sprint, and subsequently on a daily basis through daily scrum meetings. Recommended team size is from four to nine members.

Every day at the same time, the project team meets to discuss the project. Members are expected to stand during the entire meeting to encourage short meetings. Meetings are targeted to complete in 10 to 15 minutes. Each member in turn answers three questions:

1. What did I do since the last Scrum meeting?
2. What do I plan on doing between now and the next Scrum meeting?
3. Do I have any roadblocks?

This report is not a status meeting. In other words, team members do not report on what percent a task is done. They report on what work they did, why it was being done, and whether it has been completed or not. When something is encountered that is impeding progress, it is considered a roadblock. As roadblocks are reported, the team decides how to handle them, including escalating them immediately to the appropriate channel.

When multiple teams are involved in a project, a hierarchy of daily scrum meetings may occur, sometimes referred to as a Scrum of Scrums. An example would be where three teams are involved in working on three separate but related Sprints. Each team would hold their daily Scrum, and then one member from each team would convene together for an additional Scrum to make sure team coordination occurs. Information from this additional Scrum is fed back into the separate team Scrums the following day.

Scrum also defines only a small numbers of roles on the team:

Product Owner—The Product Owner is responsible for representing the customer or user of the product the team is building. Ideally this is the actual customer, but when the customer is not available for a project, this role is filled by someone representing the customer's interests; he or she is also responsible for being the liaison between the project team and the customer.

Scrum Master—Scrum originated as a way to deal with troubled or emergency projects. The person brought in to refocus the project team is called a Scrum Master. This role is primarily to facilitate, report, and focus the team on the highest priority work and to remove any roadblocks that may impede team progress. Certification training for Scrum Masters exists, supporting three different levels of practice.

Team Member—All other members of the team are considered general team members. This includes developers, architects, project managers, testers, database administrators—everyone. The team concept is important to Scrum and only the key roles of Scrum Master and Product Owner are called out.

Scrum uses the term "backlog" to refer to a list of items that are related. Three backlogs are defined by the Scrum process: product backlog, release backlog, and sprint backlog.

Figure 2. Scrum Burndown Chart. A Scrum Burndown Chart helps to provide visibility of the progress of the team and the work remaining. The straight line represents an ideal iteration where work is completed in a perfectly steady and evenly distributed manner. The more erratic line represents the work that is actually completed over time by the team.

The product backlog is a list of all the requirements to deliver the product. These are usually defined at a high level, and include estimates used to scope the Sprint Backlogs. The Release Backlog is pulled from the Product Backlog, which contains requirements. This list must be in priority order, with each entry on the list having a unique priority. Here, each requirement also includes a more granular estimate than that given in the product backlog.

At the start of each Sprint, the project team breaks down items from the Release Backlog, starting at the top (most important), and adds these into the Sprint Backlog. Once enough items have been selected to fill the Sprint, the Sprint Backlog is locked. Estimates include total time to complete each item, including but not limited to analysis, design, coding, testing, documentation, etc.

There is another item related to Backlogs that is key to Scrum. The Burndown Chart (see Figure 2) is used to indicate the number of remaining Sprint Backlog items yet to be completed in the current Sprint. A daily record is maintained showing the team's progress in achieving the goal of the Sprint.

At the end of a Sprint, the team meets with any interested stakeholders to demonstrate what work has been completed, and to evaluate priorities for the next Sprint. In addition, any outstanding roadblocks are also discussed, as well as their impact and possible solutions.

One final note is that since Scrum focuses on the project management aspects of a project and specifies no technical practices, it integrates well with other Agile methodologies. It is commonly combined with XP, but will work with other approaches as well.

Lean Software Development
While most of us might joke that Lean Software Development (a.k.a.: Lean) is a new way to lose unwanted pounds and keep them off, that wouldn't be far from the truth. Lean is a methodology that aims to trim the fat from the software process, starting with requirements, and including how business looks at the systems they request.

Derived from the Lean Manufacturing processes that companies such as Toyota have perfected over the past couple of decades, Lean's goal is to meet the challenge of defining, building, and delivering complex software systems that are exactly what the business really needs to stay competitive. Lean is similar to Scrum in that it focuses much more on the project management aspects of software development rather than the technical ones, specifically targeting the cost and ROI attributes of a project.

Lean pays a good deal of attention to gathering the "right" requirements. Requirements are measured based on their impact to the business, and must be defined in clear, complete, and verifiable ways. Incomplete, missing, wrong, unverifiable, and conflicting requirements are filtered out during the requirement process.

Due to this focus on requirements, the Customer plays an absolutely vital role in the Lean process. From a manufacturing point of view, it helps to think of Lean as being similar to new product development; this helps to crystallize the role and value that the Customer plays. Constant feedback from the Customer about the business value and functional requirements being addresses by the development team is a core component of the Lean approach.

Based on a wealth of quantitative metrics, Lean realizes that many projects fail due to their configuration and management. Projects with a silo'd, isolated, or "not my problem" attitude will face swift changes when Lean steps in. Lean attempts to also address resource issues such as the team not having the right skill sets, not enough team members, or too much turnover. In this sense Lean is very "root cause"-oriented.

Where some methodologies enforce strict definition between team roles, Lean prefers a more cross-functional approach. Team members are not only cross trained on functional and technical aspects of the system, they also team up with various team members to understand the business value for system features, and the business problems the system is expecting to solve.

Lean Manufacturing evolved from the work of Dr. W. Edward Deming's Total Quality Management (TQM), which can be boiled down to two main concepts:

1. Process is important.
2. People build and improve the process.

TQM has been refined for almost half a century now and has proven itself in the manufacturing realm. TQM is very metric intensive, and metrics play a big part of Lean Software Development as well.

A newer addition to the Lean Software Process is the Theory of Constraints (by Eliyahu M. Goldratt, et. al.), which is comprised of two primary tenants as well:

1. Identify constraints.
2. Improve/remove constraints.

The Theory of Constraints is a body of knowledge on how to effectively manage business organizations as self-contained systems.

Lean Software Development promotes seven Lean principles. The methodology revolves around these principles, and all other aspects of Lean are designed to reinforce them. The seven principles are:

1. Eliminate Waste: Eliminate anything that does not add customer value.
2. Build Quality In: Validate and re-validate all assumptions throughout the process. If a metric or practice no longer has value, discard it.
3. Create Knowledge: Use short iterative cycles to provide quick, constant feedback to ensure the right things are being focused on.
4. Defer Commitment: Don't make decisions until enough is known to make the decision—a sound understanding of the problem and the tradeoffs of potential solutions is required.
5. Deliver Fast: Minimize the time it takes to identify a business problem and deliver a system (or feature) that addresses it. (a.k.a. time to market).
6. Respect People: Empower the team to succeed.
7. Optimize the Whole: Use cross-functional teams to keep from missing important, possibly critical aspects of the problem and of the system designed to solve it.

Again, since Lean focuses on the project management aspects of a project and specifies no technical practices, it integrates well with other Agile methodologies, such as XP, that focus more on the technical aspects of software development.

Feature Driven Development
While most Agile methodologies start with a handful of principles or a specific set of processes, the center of Feature Driven Development (FDD) is the domain model. Creating a model of the domain is the foundational step for FDD, which requires collecting domain knowledge from all domain experts (sometimes called Subject Matter Experts or SMEs), and integrating this knowledge into a unified model (or set of models) representing the problem domain. Once this model and any other requirements have been evaluated, a rough plan is drawn up to determine what resources will be needed to build the masterpiece. A small set of features are identified for a team to work on for a period of time recommended to last no more than two weeks. Once the initial set of features is delivered, another set is tackled. Multiple teams can work in parallel on different sets of features, with all activity being tracked on a per feature basis.

The basic unit of work in FDD is a feature, which is defined as a small, client-valued function expressed in the form:

<action><result>[of|to|for|from]<object>.

This expression can be restated as: an action, causes a result, to an object. The section in the brackets is optional to make the feature easier to read. An example of a feature is: Calculate the monthly interest rate for an adjustable rate mortgage. Here "calculate" is the <action>, "monthly interest rate" is the <result>, and "an adjustable rate mortgage" is the object participating in this feature.

Features can be combined into Feature Sets, and Feature Sets can be aggregated into Subject Areas. Requirements are usually gathered in a top down approach, defining all the Subject Areas for the system, breaking these down into Feature Sets, and eventually down into Features, from which tasks can actually be defined and estimated. Feature Sets typically reflect a business activity, and Subject Areas commonly correspond to general business practices. Some possible Feature Sets are "Making a Deposit," "Making a Withdrawal," "Viewing an Account Balance," all of which may be part of the "Managing an Account" Subject Area.

FDD specifies five specific processes that are to be followed, in this specific order:

1. Develop an Overall Model
2. Build a List of Features
3. Plan by Feature
4. Design by Feature
5. Build by Feature

Again, it is important to understand that absolutely everything is planned, built, managed, and tracked on a per-feature basis. Other units such as feature sets and subject areas are available for higher level planning and reporting, but the key universal unit is the feature.

FDD relies on specific roles and responsibilities more than almost any other Agile methodology. FDD also tends to move away from shared ownership of code and artifacts that other Agile methods promote, and the team roles reflect this. The nine roles defined for FDD are:

1. Project Manager—Responsible for all administrative aspects of the project, including the financial and reporting ones.
2. Chief Architect—Responsible for the overall design of the system, including running all design sessions, code reviews, and technology decisions.
3. Development Manager—On the hook for the daily development activities. Coordinating the development team and their activities, and dealing with resource issues.
4. Chief Programmer—A senior developer involved in ongoing design and development activities, and is assigned to be responsible for one or more Feature Sets.
5. Class Owner—A developer who works under the direction of a Chief Programmer to design, code, test, and document features as they are implemented.
6. Domain Expert—Any business related stakeholder who can define required features that the system should provide. These would include clients, users, and various analysts; anyone who has knowledge of how the business works that could impact the system.
7. Tester—Responsible for verifying that each feature performs as defined.
8. Deployer—Deals with not only the actual deployment of code to various environments, but also the definition and/or conversion of data from one format to another.
9. Technical Writer—Responsible for creating and maintaining all the online and printed documentation that a user will need for the final system.

FDD uses several unique and specific mechanisms to report project activity and progress. The least unique of these is the feature list and the task list. Many Agile methodologies use some sort of list to track requirements and work done on requirements. In FDD, these lists all correspond to a specific feature.

Figure 3. Feature Set Progress Report. The Feature Set Progress Report illustrates progress of feature development in each subject area. Light green Feature Sets are in progress and on schedule, dark green Feature Sets are completed, and grey Feature Sets are behind schedule.

Capturing exactly where a feature is in its development cycle is done using a table that tracks six specific milestones:

1. Domain Walkthough
2. Design
3. Design Inspection
4. Code
5. Code Inspection
6. Promote to Build

These milestones are tracked by the date each is completed and by the Chief Programmer responsible for that specific feature.

Feature milestones are rolled up into a table showing feature sets. This allows project stakeholders to see how the project is progressing. Things can be rolled up into subject areas as well if there is value in doing so. Completed features are also tracked across the entire project using a line graph. This graph shows the cumulative total by day or week of all features that have been completed.

The final group of reports is the Feature Set Progress Report, which is very unique to FDD (see Figure 3). This report is color coded and shows each Feature Set in the project, what percent complete it is, how many features are in each area, the name of the Chief Programmer for each Feature Set, and the month and year when each is targeted to be completed.

Three More to Go
We couldn't cover all the Agile methodologies in one article. In fact, we can't cover all the Agile methodologies in two articles either but we hit all the big ones. In the next article we continue this exploration by analyzing the Agile Unified Process, Crystal, and DSDM. And we bring all this data together with a great summary of all the different approaches and look at the contexts in which each methodology excels. Finally, we offer tips when choosing a methodology or combination of methodologies for your project.