verything right or wrong with a software project is management’s fault. Either management staffed the right people or the wrong people. Management was absent or involved.
Management is hard, and there are numerous factors that can cause success or failure of a project. In the best situation you have great people who do great work. Software managers can succeed despite themselves if they happen to staff a top-notch team?even though the managers, themselves, might not be very competent. Any successes that a top-notch team achieves are the manager’s achievement. Failure, however, is harder to blame on the team because a manager must be able to solve problems as they come along. This article focuses on tips and knowledge you can use when managing an agile software project.
Aiming for Agile
Lean, Extreme Programming (XP), and Scrum are probably the three most well-known agile frameworks, but in my opinion, none of the three is sufficient on its own. Probably the most well-known at this point (at least in my region) is Scrum. I hear folks referring to iterations as “sprints” more and more. Managers are sending their folks to Scrum training left and right, and yet, Scrum is naked without XP. Scrum doesn’t offer much guidance on engineering practices, and as long as the project involves software, engineering practices are key. When managing your team, ensure you provide adequate engineering guidance. I’ve found that you can get great results when you manage according to Lean principles, run the project inside the Scrum framework, and create the software through Extreme Programming; the methodologies are quite complementary.
Instead of going into an exhaustive explanation about each agile method, I’ll leave that research to you, and instead talk specifically about what things to pay attention to while managing your project and striving for agility. Just think of agility as the ability to change course and adapt continuously to customer or product owner needs. The software should remain just as maintainable as on the second day of its life and the pace of the project and subsequent releases should remain stable.
The Fallacy of Fixed Scope
One of the reasons “scope creep” is such a well-known term in our industry is because scope hardly ever stays the same. Even in fixed-scope projects, scope changes. That’s because customers and users don’t understand 100 percent of the scope they need to solve their business problem. They might know 50 or 80 percent. In highly-regulated environments, that knowledge might even be in the 90s, but then the communication factor comes into play. How do you communicate scope? Do users or developers or managers attempt to write it down in prose? Do they have long verbal conversations? Even if the software customer theoretically understood 100 percent of the scope required to solve a problem, they cannot communicate 100 percent of that understanding up front.
Agile project management understands that scope is a moving target. The larger the scope, the more it will change. It’s not that the scope itself changes so much, but the understanding of that scope changes. Even with the theoretical 100 percent understanding, the information transition to the development team is a variable and not a perfect medium; therefore, to the team, scope will appear to have changed even if it has not.
Now, let’s move away from the theoretical 100 percent understanding to the more realistic 50% understanding. Imagine you have a common environment where a customer has a business problem that needs solving, and he thinks he has a pretty good idea what will solve it; hence, the 50 percent understanding. To manage this environment, both the team and the customer must understand the core of the business problem and be completely committed to solving it. The scope the customer has in mind might be relevant, or it may not. The software manager should foster an environment where questioning assumptions is welcome. In this environment, the software team will apply a critical eye to every story on the table and evaluate whether that story will contribute to the end goal. Every story must stand up to criticism to prove worthy of the team’s time. Working on the right thing is as important as making the thing worked on right. Work with the customer to determine together what scope will solve the customer’s problem in the best, fastest, and cheapest manner. In most cases the team will be able to cut scope from the project.
The Unspoken Quality Variable
When managing a team, it’s important to understand who you have. You know how much you pay your people, and you should understand how much you are getting from them. All software engineers are not created equally, and there is no such thing as a project that will require exactly five developers. Let’s take, for example, a team of six developers who each get $10,000, and compare their output to a team of three more experienced developers making $20,000 each. (I’m intentionally giving unrealistic salaries because salary numbers are not relevant here, just the relative size.) Studies referenced in Code Complete by Steve McConnell show that you can get developers who are roughly ten times as productive for only twice the price. The key lies in discerning each developer’s skill. It’s entirely possible that a smooth-talking developer on par with your team of six has negotiated a higher salary without the skill to back it up. That would be your mistake and his gain. Hopefully, if you’ve made your way into management, you can discern people’s skills.
In an agile project, you tend to employ more generalists than specialists. They will all work together to meet the commitment for each iteration. Staffing a team of specialists limits communication and creates bottlenecks, which then create queues. Refer to Lean software principles for information on queuing theory.
Key Performance Indicators (KPIs)
When managing an agile software project, you’ll be interested in tracking some metrics to help you gauge how the project is going. Just remember, whatever metrics you track will improve. Make sure you don’t track the wrong things. For instance, if you track number of lines of code as a productivity measure, you will see productivity soar, at least according to your metric. It’s natural to improve the measure, or game the system, as you might say; therefore, track what you really want: software delivered and customer satisfaction. Start with the following metrics and hone your tracking over time:
- Defect-free stories delivered. Each story represents a unique behavior in the software from which the customer benefits. Stories delivered with defects aren’t as valuable, so don’t count those.
- Customer satisfaction. Ultimately, you are creating software for a customer. Without the customer, you wouldn’t be creating the software. Keeping them happy is a great metric. It’s simple. Just ask their satisfaction level on a scale from one to ten.
- Consistent velocity. Iteration over iteration, velocity should remain constant. If it continues to change, something is wrong. After about the sixth iteration, treat any significant change in velocity as a cause for alarm. If the team make-up is consistent, the velocity should remain constant. A slowing of velocity could be a sign of a codebase that is less than maintainable.
|Editor’s Note: This article was first published in the May/June 2008 issue of CoDe Magazine, and is reprinted here by permission.|
Managing the Customer
Part of your job is managing the development team, but you must also manage the customer. Customer satisfaction depends so much on managing expectations. Customers don’t like surprises, so you shouldn’t deliver them. The customer should know what to expect in the iteration demo. If you deliver less than your team committed to, the customer gets an unpleasant surprise. If, for example, a team member must be out for paternity leave, inform the customer that it will affect the output of the next iteration.
A great way to keep the customer informed and to avoid surprises is by inviting them to the daily stand-up meeting. Every morning, have the team go over three items:
- What I accomplished yesterday.
- What I hope to accomplish today.
- What barriers are impeding progress.
Armed with this daily knowledge, the customer will start to feel like part of the team instead of an outsider. The customer will have up-to-date knowledge and won’t be greeted with any surprises. The daily stand-up meeting, weekly iteration demos, and constant communication will keep the customer happy and involved. It’s important to pull the customer into the process; otherwise, there will be an “us vs. them” environment, and that is dysfunctional. When your team is in competition with the customer, neither will win.
Bugs vs. Defects
Defects are gaps between what your team committed to and what they delivered. If the team commits to a story and they find a problem with the functionality of that story, you have a defect on your hands, and the story isn’t finished. You should only count stories as “done” when your customer accepts them without defects. It does no one any good if you merely log a defect and take credit for the story anyhow.
A bug is anything that bothers users. I’m not sure we’ll ever get away from bugs, but constant communication sure helps. In managing the software process, it’s important to understand the difference between a bug and a defect. Bugs are not negotiable. If the behavior of the software bugs a user, you have a bug, end of story. If your team and your customer agreed upon the behavior beforehand, then you have a defect, not a bug, and your team should correct defects immediately.
A defect is a gap in the story contract and acceptance criteria. A bug is not a defect since it’s an annoyance that has yet to be discussed. Bugs pop up all the time, and if the customer is motivated enough, he’ll craft a story to get rid of the bug and put that story on the backlog. A customer shouldn’t have to think about defects, however. When someone finds a defect, the team should drop everything and fix the defect right away. A defect signifies that a story which the team has already labeled “done” is, in fact, not finished. Finish the story and move on. Institute a zero-defect policy. Don’t count a story as done until it meets all acceptance criteria. You can’t get away from bugs, but you sure can eliminate defects.
At the end of every iteration, have a short time where the team can reflect and craft ways to improve even more. A fun way to do this is to list feedback under three headings:
- Good (what you would like to continue).
- Bad (what you would like to stop).
- Ugly (a fun category if an issue was particularly messy).
Consider all items as action items. Your team should continue and enhance “good” items. Your team should stop, change, or overhaul bad and ugly items. A good manager will try to remedy the items listed under “bad” and “ugly.” Most of the time, you are the only one who can change these items. The team will remedy the items under their control, and the items that remain are more organizational problems. Don’t just ignore these organizational problems. It may be difficult, but as management, it’s your job to handle issues such as poor facilities, equipment, software licenses, etc. Who said management was easy?
Keep retrospectives short. Thirty minutes is a good cap on a retrospective meeting. You don’t want it to linger on into a discussion on agile theory. After all, you’re in charge, so if your team starts a long discussion on how to move forward, you may have to make the call.
Software system design can enable or cripple your team. Let pain and developer annoyance be your guide. If an aspect of the software is painful or annoying, invest the time to fix it. Treat that time as an investment. That investment should yield returns. For instance, a shared development database often causes pain because one developer who alters the schema for a new feature will often break the system temporarily for the rest of the team. Having a database instance on each developer machine quickly solves that particular problem. Be sensitive to pain and don’t take “we can’t do anything about that” as an answer.
Determining Iteration Length
An iteration is the length of time a team works before demonstrating the new state of the system to the product owner. Scrum advocates one month. XP advocates one or two weeks. I strongly advocate one week. Iterations are different than releases. Iterations represent a cycle in software delivery. Lean stresses the reduction of cycle time as a means to improve throughput. Shortening cycle time to one week will help keep the team focused on exactly what they need to deliver. It’s hard to get distracted from essential stories when there is only one week to deliver them. Some folks might think that you can’t get anything done in only a week. If that is true, then you have some serious other problems that you need to tackle. If you are just beginning to embrace agile values, then you might be working with systems where even fixing a typo and releasing to production is a two-month ordeal. If that is the case, then you need to improve your build and deployment process so that it can be done in an hour, not two months. If you have some monumental issues you are battling, you need to invest effort into fixing those problems. As the manager, everything right or wrong in your organization is your fault. Take the initiative to mitigate the problems.
One-week iterations squeeze your process and force dysfunctions to surface. With short cycle times, anything slowing down your organization will become very obvious. Problems can’t hide when you employ short cycle times. Use one-week iterations as a tool to discover existing problems. Tackle them one by one until you have a smooth-running team. You’ll then be able to deliver significant value in a single one-week iteration.
Hiring and Firing
As a manager you want to do more hiring than firing. Staffing is your responsibility. I’ll now discuss what I call a “net-negative developer.” This developer does not have low productivity. These are developers no one misses when they’re out sick. These developers might have good intentions, but team velocity is slower because of them. In fact, the team could move faster if these people went on vacation. Have the courage to make the staffing changes necessary to improve your team. Hold high standards for your team members. I understand that large corporate environments might have policies against firing without egregious performance. As a manager, it is your responsibility to navigate the political landscape to remove things that hinder your team’s performance. A net-negative developer is a constant tax on your team’s productivity. Have the courage to do what is right not only for your team but your customer or product owner.
In contrast, recognize when you have a “multiplier.” In performance reviews, you might not be able to list specific items directly attributable to this person, but it seems that whatever project the multiplier is on goes very well. This developer multiplies the efforts of the rest of the team. Perhaps this person removes roadblocks or works to streamline your process. Either way, this person might double the productivity of the entire team just by doing what they do. Recognize and reward this person. They are a valuable asset to your organization. Work to identify this type of person in your hiring process as well.
Agile Values and Principles
Before I conclude, I would like to walk through the values and principles in the Agile Manifesto. As a manager, it is your responsibility to set the culture for your department. You must define the values and principles your department operates by, so it’s important to differentiate between practices and principles. Practices can be prescribed. Scrum and XP prescribe practices. They also prescribe principles. Agile does not prescribe practices. Agile is a classification of software methods based on values and principles. The practices of XP and Scrum are examples of ways to implement agile principles.
The four values of agile software development are to emphasize:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
The Agile Manifesto places a heavy emphasis on people over tooling. Because software teams must deliver software, the teams that follow agile processes value delivering software over other artifacts that are not. . .software. These four values define an agile mindset. You can find the twelve principles of agile software development online. I would like to go through each one individually:
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
- Deliver working software frequently, from every couple of weeks to every couple of months, but prefer shorter timescales.
- Business people and developers must work together daily throughout the project.
- Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- Working software is the primary measure of progress.
- Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity?the art of maximizing the amount of work you don’t have to do?is essential.
- The best architectures, requirements, and designs emerge from self-organizing teams.
- At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Delivering software is the valuable service that your team provides to your customers. A status report is nice, but in the absence of working software, it is hardly a substitute. An architecture diagram is often pretty, but there is no ROI in the marketplace if users cannot use the system. You should elevate delivering software to the top of the priority list. Whatever else also has to be delivered can be delivered after the working software. We also emphasize continuous delivery of software. This means we do not do a single release. We release early and often so that the customer can use the software as it evolves.
A traditional software team might take a full year to deliver a large software project. Twelve months into the project should not be the first time the customer can change his mind. In contrast, an agile software team will demonstrate the state of the software early and often, allowing the customer to evaluate whether or not it perfectly meets their needs. The software team can incorporate any feedback early and often. If the customer has made a big mistake, the software team should be able to help the business by reacting quickly and changing direction. Failure to accurately predict the future should not be an unrecoverable mistake on the part of the customer.
Prefer a short cycle time. Agile software teams deliver full releases frequently. Customers can use full releases, whether for internal use or for release to real users. The more real feedback the customer can give, the quicker the agile software team can make essential changes. Often, customers do not know what will work until they can use a software feature. As they use released features, they’ll come up with ideas for improvement.
There is no substitute for humans working with other humans. Relationships between humans are the key to good working environments. Business folks and developers who do not know each other have a huge disadvantage when compared to teams where all involved see each other on a daily basis.
You cannot manage software with assembly-line principles. Software is a specialized form of knowledge work, and folks on a project team need to be highly trained and experienced. You cannot swap team members out like standardized parts on the Army’s M-16 assault rifle. Each team member is unique and has unique ideas. As a manager, you should motivate them and set them loose. Remove any environmental roadblocks such as inadequate tools. Give them the responsibility for delivering the project. Then give them the authority to do what is necessary.
Some studies have concluded that only seven percent of communication is verbal, or in the words themselves. Tone of voice accounts for 38 percent and non-verbal clues accounts for 55 percent. Therefore, if you choose email as your only means of discussion, you are losing 93 percent of available communication bandwidth. That is no way to run a software project. Using face-to-face communication leverages the full power of communication.
There is no substitute for showing the real software to the customer. A status report has no teeth.
The architecture of the software system should promote an environment where technical complexity rises linearly. In contrast, many software systems have exponential complexity graphs. This causes future changes to be more and more difficult to make. Over time, a system may become unchangeable because of its complexity. You should keep the complexity curve linear for sustained development.
Keeping the software design clean will enable you to change it as necessary. Duplication is a particularly egregious software sin. As logic gets duplicated, changes to business rules might require similar changes in multiple places. This hinders agility.
Companies often face the buy vs. build decision. Even after you have decided to build a custom software system, you also have the opportunity to purchase components as opposed to crafting every line of code yourself. A common buy vs. build decision today is to use an object-relational mapping library instead of writing every line of database access code by hand. Equally important is helping the customer simplify the proposed solution if the problem at hand can be solved with a software system of significantly less complexity. Keep everything as simple as possible. Be wary of developers who are proud of complex code. Instead, rewrite complex code, and make developers proud of simple code.
You have presumably hired good people. Put them together so they can work to refine the process. Unilaterally prescribing a rigid process will ultimately frustrate your people and will not lead to maximum throughput.
Use retrospectives to discover how to become more effective. Harness the intelligence of every team member in this process.
It is not easy to manage an Agile software project. You will never know it all, so expect to be a continuous learner. Never hesitate to listen to someone who has an idea for improvement. As a manager, every good idea does not have to be yours. In fact, if every good idea is yours, you probably have some staffing problems. Always delegate authority along with responsibility, and empower your people to succeed. Reach beyond your company into the community and other parts of the industry. Share ideas, and do not be afraid to admit fault. Change immediately when you discover you are going in the wrong direction, but do not change any part of your process without evidence that it is not working. Hire and keep good people. Give them the tools and authority to succeed. Then get out of their way.