Posted by Gigi Sayfan
on May 2, 2016
The ability to hire excellent software developers is a limiting factor. There is a real shortage, and as a result there is serious competition for every qualified candidate. As a manager on the lookout for talented developer I experience it personally. There are two very clear trends here. Candidates who put themselves on the market will get queries from more companies than they can handle. In one hiring platform that I use shows competing offers and it is common to see a well-qualified candidate receiving 20+ invitations for initial interviews. The other trend is that the big companies out there: Google, Facebook, Uber, etc., use their war chests to offer unheard of levels of compensation. An engineer decides which company to join based on many factors. But, if a big company offers $50K - $100K more than competing offers (in total compensation) then it's very likely the engineer will pick the big company. How can a small startup compete? By using its relative strengths - speed. This is where Agile hiring methods can give you the advantage.
What if you compress the entire hiring process so that you can get an actual job offer out to a candidate, before they have even had a chance to set up phone interviews with most of the other companies who might be interested in them? In the best case scenario, maybe your competitors are so slow that they didn't get around to even sending to initial invitation. I will not discuss here how to find those candidates.
Let's start from the assumption that you have some source of candidates. With Agile hiring your focus is to minimize the time from zero to offer (or rejection). The typical hiring process consists of:
- Review of resume/application/cover letter
- Phone interview[s]
- Coding challenge
- Face-to-face interviews
- Reference checks
- Background check
- Offer letter
The goal is to avoid wasting valuable time by rejecting inadequate candidates early in the process. Let's see what we can get rid of. The phone interview can go away. I very rarely reject a person based on the phone interview. If the original application is solid, the candidate will make it to the next phase. The coding challenge can go away if the candidate has a decent GitHub account. The reference checks can go away. I never got any useful information whatsoever from a reference the candidate provided. So, here is the Agile process:
- Applications are processed by an experienced technical person who can look beyond buzzwords and evaluate coding competency based on public code.
- Immediate invitation to face-to-face interviews, or if problematic, video conference interviews (easier to schedule).
- Until the interview exchange information with the candidate for expected compensation and role. (This may be a phone conversation, but not a gating review to delay the face-to-face interview that was already scheduled.)
- At the end of the face-to-face interview the candidate should have an offer in hand.
Negotiation may take place and background checks can be conducted later. That's it. You could have a new engineer within 2-3 days since first contact vs. 2-3 months. Think about it. If you're worried about wasting people's time with too many interviews, consider having fewer people interview each candidate. It is not a rule of nature that every team member must meet and approve every candidate. If you are confident in your filtering abilities, then maybe just the hiring manager and two other people need to speak with the candidate.
Posted by Gigi Sayfan
on April 14, 2016
Before the Agile Manifesto there was Extreme Programming. I discovered it through the http://c2.com wiki — the first wiki and was instantly impressed. Extreme programming is indeed the extreme. There are values such as communication, simplicity, feedback and courage. Later respect was added. But, courage is what caught my attention. Too often, I see people operate out of fear, which is simply debilitating. Even the "move fast and break things" movement is not courageous because they hedge their bets and concede that they'll break things.
Extreme Programming is different. It doesn't make excuses. It doesn't hide behind trade-offs such as, if we move fast, things will get messy. No, the Extreme Programming stance was different. What happens if we take all the best practices and turn the knob to 11?
There are many practices and you can read about them detail here. Some of them were revolutionary at the time. How did extreme programming fare? It depends. On one hand it ushered the era of Agile and, as such, was revolutionary. But, on the other hand it was too strict to follow exactly. The Chrysler Comprehensive Compensation (C3) project that was the real world test bed for Extreme Programming was cancelled after 7 years and a plethora of other Agile methods exploded on the scene. Many of the original extreme programming practitioners such as Ward Cunningham, Kent Beck and Martin Fowler became well-known thought leaders, published successful books and continued to advance the state of software development. I definitely learned a lot about software development from reading and trying to practice variants of Extreme Programming.
Posted by Gigi Sayfan
on March 21, 2016
Agile methods can accelerate development significantly compared with traditional methods. This is true in particular for programming where fast edit-test-deploy cycles are possible even on a massive scale (look at Google or Facebook for good examples). However, the database is often another world. If you keep a lot of data in your database (and who doesn't?) then changes to the database schema may not be as swift as regular code changes.
There are several reasons for that. The data is often the most critical asset of an organization. Losing/corrupting the data might lead to the demise of the company. Imagine a company permanently losing all their user information, including billing. This means that any such change should be taken with extreme caution and with lots of measures to detect and revert any change. This is not easy to do safely. Another reason is that the database if often at the core of many applications and some database changes lead to cascading changes in those applications.
Sometimes, schema changes, such as splitting one table into two tables, require data migration from the old to a new schema. If you have a lot of data that may be a long process that takes days or weeks because you still need to support the live system while the migration is ongoing.
So, is the database doomed to be this thorn in the side of Agile development? Not necessarily. There are some strategies you can employ to minimize both the risk and the effort to make database changes. First, use multiple databases. The data can often be split into multiple independent data sets. Any change to one of them will not impact the others. Another approach is use schema-less databases, where the application deals with the variety of implicit formats--at least during migration periods. Finally, you can get really good at managing database schema changes and build the mechanisms to support it, migrate data and ensure you have working recovery mechanisms. This takes a lot of work, but it is worth it if you work on innovative development and need to evolve your database schema.
Posted by Gigi Sayfan
on March 15, 2016
Flow is a mental state where you're in "The Zone." Time flies by, everything just works and you're totally focused on the task at hand and apply yourself fully. You can see basketball players attain it sometimes where they just can't miss a shot. But, you can also observe and experience it programming. It's those amazing sessions where in several hours or a day you churn out something that typically takes weeks.
These 100x productivity periods are highly coveted. Not just because of how much you get done, but also because it is one of the best feelings in the world when you totally realize your potential. To get into flow you need an obstacle free environment--no distractions, all available resources at hand. Everything must be streamlined.
In software development, Agile methods provide the best breeding ground for getting into flow. Your tasks are broken down into small enough chunks that you can comprehend them, the automated tests, continuous integration and deployment smooth the process of verifying code. Pair programming is arguably another fantastic technique to get into flow. The driver is just programming, while the navigator deals with all the stuff that typically would distract the driver if they were programming alone. For example, looking up documentation, checking the status of the latest test run and scouring StackOverflow for that helpful snippet. The most important part might be that you can't just go HackerNews binging in the middle of a pair programming session. When you have two people working together, one keeps the other honest as far as focus goes. I recommend that you explicitly experiment and find the best way for you to achieve flow. It is well worth it.
Posted by Gigi Sayfan
on March 9, 2016
Many Agile methodologies promote a daily standup meeting. The idea is that people are uncomfortable standing for a long period of time, so it will help keep the meeting short. The goal of the daily meeting (typically first thing in the morning) is to get a sense of what everybody is working on and if there are any obstacles. But, is the daily standup meeting effective?
I have worked for many companies that implemented some variant of Agile development and some of them implemented daily standup meetings as well. Based on my observations, the daily meeting is just a ceremony that doesn't really contribute much. Let's consider two different teams: small and large. The small team is generally fewer than five people. The large team is likely more than ten people. Teams that have between 5 and 10 will behave as either small or large depending on the team members and the culture. Small teams often work in the same room/open space. They already know what everybody does and obstacles can be addressed on the fly by turning around and talking to your teammate.
Large teams may be split physically and functionally and often work on multiple projects or separate aspects of the same project. The daily standup meeting either takes a long time when every member takes the time to explain what they're working on and what issues they face or is kept short by rushing through and having each team member mumble a few sentences. There is no time for questions and often people are not really interested in the daily struggles of others and just want to get on with their day. Any serious discussion requires a separate meeting with the relevant people.
Given those concerns, what are the benefits of the daily standup meeting? It can contribute to team spirit, serve as a launch pad to a more in-depth meeting and occasionally someone may have insight about how to overcome a particular obstacle. The team lead may gain some benefit by getting a concise summary from the whole team, and finally it ensures that everybody is in the office at the same time!
Posted by Gigi Sayfan
on February 29, 2016
Agile development practices have become mainstream as large and small organizations recognize Agile development's superiority over other approaches for most projects. Organizations that follow the Agile method for development can produce real business value continuously. But, how do you deliver that value to your users?
Typical marketing departments will run large campaigns, sometimes synchronized with big industry events, sometimes with big launch events for important product releases. This is all in stark contrast to the development organization. If the developers added capability X to the product, why wait several months and deliver it to the users with capabilities Y and Z together. Yes, it makes for a more impressive presentation when you show X, Y and Z. But, you actually left value on the table by not delivering X when it was ready.
Agile marketing is all about being in tune with the development organization and also the market. Some of the values of Agile marketing are:
- Big plans are eschewed in favor of responding to changes
- Many small experiments over a few large bets
- Discovery over prediction
- Adaptive and iterative campaigns over big-bang campaigns
There are many benefits to this approach since the marketing organization is in tune with both the customers and the development organization. This allows communicating changes or better understanding of user needs to the developers in a continuous fashion (build the right thing). The users in turn are delighted because their wishes and requests are getting satisfied much more quickly.
The lack of big announcements about upcoming future products and capabilities takes away the risk of committing early to something that is not well understood or possible given current technology. As a result, users will not be disappointed when you fail to make good on your promises. This goes against the grain of the "establishment" but if you work in an organization that follows Agile development practices, consider pushing towards more Agile marketing as well. Take advantage of the great synergy that can exist between the various groups of your organization.
Posted by Gigi Sayfan
on January 21, 2016
Agile methodologies have been used successfully in many big companies, but it is often a challenge. There are many reasons: lack of project sponsorship, prolonged user validation, existing policies, legacy systems with no tests - and most importantly culture and inertia. Given all these obstacles how do you scale Agile processes in a big organization? Very carefully. If you're interested in introducing Agile development practices into a large organization, you can try some of these techniques:
- Show don't tell - Work on a project using Agile methods. Get it done on time and on budget using Agile methods.
- Grow organically and incrementally - If you're a manager it's easy. Start with your team. Try to gain mindshare with your peer managers - for example, when collaborating on a project, suggest the use of Agile methods to coordinate deliverables and handoffs. If you're a developer, try to convince your team members and manager to give it a try.
- Utilize the organizational structure - Treat each team or department as a small Agile entity. If you can, establish well-defined interfaces.
- Be flexible - Be willing to compromise and acknowledge other people's concerns. Try to accommodate as much as possible even if it means you start with a hybrid Agile process. Changing people and their habits is hard. Changing the mindset of veteran people in big companies with established culture is extremely difficult.
Finally, if you are really passionate about Agile practices and everything you've tried has failed, you can always join a company that already follows agile practices, including many companies from the Fortune 2000.
Posted by Gigi Sayfan
on January 13, 2016
- The form of internal documentation appropriate for an organization following agile practices.
- Generating external documentation as an artifact/user story.
The first meaning is typically a combination of code comments and auto-generated documentation. A very common assertion in Agile circles is that unit tests serve as live documentation. Python, for example has a module called doctest in which the documentation of a function may contain live code examples with outputs that can be executed as tests which verify the correctness.
Behavior Driven Development
BDD is putting a lot of emphasis on even specifying the requirements in an executable form via special DSLs (domain specific languages), so the requirements can serve as both tests and live human readable documentation. Auto-generated documentation for public APIs is very common. Public APIs are designed to be used by third party developers who are not familiar with the code (even if it's open source). The documentation must be accurate and in sync with the code.
The second meaning can be considered as just another artifact. But, there are some differences. Typically, when generating external documentation for a system it is centralized. You have to consider the structure and organization and then address the content as a collection of user stories. Unlike code artifacts, external documentation doesn't have automated tests. Documentation testing is an often neglected practice. Which is fairly typical because the documentation itself is often neglected. However, some types of external documentation are critical and must serve contractual or regulatory requirements. In these cases, you must verify that the documentation is correct.
Posted by Gigi Sayfan
on January 4, 2016
Agile practices have proven themselves time and again for development and evolution of software systems. But, it's not clear if the same agile approach can benefit user-facing aspects such as public APIs, user interface design and user experience. If you change your API constantly, no sane developer will use it. If your user interface design or experience keeps shifting users will get confused and angry that they have to face a new learning curve whenever you decide to make a change. Sometimes, users will be upset even if the changes are demonstrably beneficial, just because of switching costs. Remember users didn't subscribe to your agile thinking and are just interested in using your API/product.
What's the answer then? Do you have to be prescient and come up with the ultimate API and user interface right at the beginning? Not at all. There are several solutions that will allow you to iterate here as well. But, you have to realize that iteration on these aspects should and will be slower and more disciplined.
Possible approaches include A/B testing, keeping the old API/interface available, deprecating previous APIs, backward compatibility, testing rapid changes on groups of users that sign up for beta. In general, the more successful you are, the less free you are to get rid of legacy. Probably the best example is Microsoft which still allows you to run DOS programs on the latest Windows versions and used a variety of approaches to iterate on the Windows desktop experience, including handling the frustration from users whenever a new version of Windows comes out. Windows 10 is a fine response to the harsh criticism Windows 8 endured.
Posted by Gigi Sayfan
on December 14, 2015
Micro service architectures gain a lot of mindshare recently for good reasons. Let's assume you did the hard work and you have switched your system to use micro services. What are the benefits to an agile team? What would an agile micro service looks like?
The best thing about a micro service is that it is micro. It has a very well scoped functionality and it's easy to wrap your head around what it's doing. That means it is very easy to deliver significant features or improvements to a micro service within a single sprint. It also means that it is easy for a new engineer that has never worked on the micro service to pair with another engineer and quickly get up to speed.
The interactions with other services are through well-defined interfaces, so it is easy to use dependency injection to mock them and create automated unit tests. The narrow scope of the micro service helps to also to ensure that the test coverage is adequate. Refactoring anything inside the micro service is very simple because there is no need to coordinate with other services. It also aids in scaling a system incrementally.
At some point, every assumption you made when you designed the smaller system will come back to bite you when you scale — that data fits in memory, the data fits on the hard disk, the data can be stored reliably in one data center, the server can return all the data without timing out, etc. One approach is to try to think about all these eventualities and engineer them into the system from the beginning. This is typically a very bad decision. Your system might not scale to the level of engineering you built in for years, the time to market will be significantly higher if you try to build infinitely scalable system from the start, operations and maintenance will be more difficult. But, the worse thing is that when your system actually scales to the levels that require such engineering effort you'll discover that your well-crafted super-engineering from three years ago completely missed the mark and you have to start from scratch.
Micro services can help with that too. You can often isolate scalability, fault tolerance and high-availability into dedicated micro services that you build later and weave with your existing business logic micro services. The loosely coupled nature of micro services lends itself very well to agile evolution of the system.