Posted by Gigi Sayfan
on August 23, 2016
Building software used to be simple. You worked on one system with one executable. You compiled the executable and if the compilation passed, you could run your executable and play with it. Not anymore--and trying to follow Agile principles can make it even more complex. Today systems are made of many loosely-coupled programs and services. Some (maybe most) of these services are third-party. Both your code and the other services (in-house and third-party) depend on a large number of libraries, which require constant upgrades to keep up-to-date (security patches are almost always mandatory). In addition, these days, a lot more systems are heavily data-driven, which means you don't deal with just code anymore. You have to make sure your persistent stores contain the data for decision making. In addition, many systems are implemented using multiple programming languages, each with their own build tool-chain. This situation is becoming more and more common.
To follow Agile principles and allow an individual developer to have a quick build cycle of edit-built-test requires significant effort. In most cases it is worth it. There are two representative cases: small and large:
In the small case, the organization is relatively small and young. The entire system (not including third-party services) can fit on a single machine (even if in a very degraded form). In the large case, the organization is larger, it's been around for longer and there are multiple independent systems developed by independent teams.
The big case can often be broken down into multiple small cases. So, let's focus on the small case. The recommended solution is to invest the time and effort required to allow each developer to run everything on their own machine. That may mean supporting cross-platform development even though the production environment is very carefully specified. It might mean creating a lot of tooling and test databases that can be quickly created and populated.
It is important to cleanly separate that functionality from production functionality. I call this capability system in a box. You can run your entire system on a laptop. You may need to mock some services, but overall each developer should be able to test their code locally and be pretty confident it is solid before pushing it to other developers. This buys you a tremendous amount of confidence to move quickly and try things without worrying about breaking the build or development for other people.
Posted by Gigi Sayfan
on June 24, 2016
The traditional view of productivity and how to improve it is completely backwards. Most people think of productivity as a personal attribute of themselves (or their subordinates). X does twice as much as Y, or yesterday I had a really good day and accomplished much more than usual. This is a very limited view and it doesn't touch on the real issue.
The real issue is organizational productivity. The bottom line. The level of unproductively increases with scale. This is nothing new. It is one of the reasons that miniature startups can beat multi-billion corporations. But, most organizations look at the inefficiencies introduced with scale as a process or a communication problem, "If you improve the process or the communication between groups, then you'll improve your situation." There is some merit to this idea, but in the end, larger organizations still have a much greater amount of unproductivity compared to smaller organizations.
The individual employees at the bottom of an organizational hierarchy work approximately as hard as individual startup employees. The middle management does its thing, but is unable to eliminate or significantly minimize this large-organization unproductively tax. In some cases, there is a business justification for the larger organization to go more slowly. For example, if you have a mature product used by a large, and mostly happy, user base then you don't want to change the UI every three months and frustrate your existing users with weird design experiments. You might want to do A-B testing on a small percentage, however. The current thinking is that this unavoidable. Large companies just can't innovate quickly. Large companies either create internal autonomous "startups" or acquire startups and try to integrate them. But, both approaches miss out on important benefits.
I don't have an answer, just nagging feeling that we shouldn't accept this unproductively tax as an axiom. I look forward to some creative approaches that will let big companies innovate at startup-like speeds, while maintaining the advantages of scale.
Posted by Gigi Sayfan
on June 9, 2016
In Rational Tech Adoption, I discussed how to decide about whether or not to adopt new technologies. But that decision is not context-free. There is always a switching cost. If the switching cost is too high, you might forgo potentially beneficial upgrades whose benefits are less than the switching cost.
In order to benefit from new technologies, you need to build agility right into your stack. Can you easily switch your relational database from MySQL to PostgreSQL? How about from MySQL to MongoDB? Can you move from Python 2 to Python 3? How about from Python to Go? Are you running your servers on AWS? Can you move them to the Google cloud platform? How about switching to a bare metal virtual service provider? Those are not theoretical questions. If you build successful systems that accumulate a lot of users who stay around for many years, those questions will come up.
Traditionally, the answer was always, "Nope. Can't switch," and you got stuck with your initial choices or, alternatively, a super complicated project was launched to upgrade some aspect of the system. The good news is that similar to testability, performance and security, if you follow best practices such as modularity, information hiding, interaction via interfaces and build loosely coupled systems, then it will be relatively simple to replace any particular component or technology.
The stumbling block may often be your DevOps infrastructure and not the code itself. The whole notion of DevOps is still new to many organizations and there isn't much awareness regarding the need to quickly switch fundamental parts of your DevOps infrastructure. If you consider these aspects when you build your system, you'll have the chance to reap enormous benefits as your system scales and evolves.
Posted by Gigi Sayfan
on May 25, 2016
My favorite Agile methodology, Extreme Programming, has five main values: Simplicity, Communication, Feedback, Respect and Courage. The first four are relatively uncontested — everybody agrees on these four. Courage is different. Courage ostensibly flies in the face of safety, security, stability and risk mitigation. But, this is not what courage is about. Courage is actually about trust. Trust yourself, trust your team, trust your methods and trust your tools. But, trusting doesn't mean being gullible or blindly trusting things will somehow work out. Your trust must be based on solid facts, knowledge and experience. If you have never done something you can't just trust that you'll succeed. If you join a new team you can't trust them to pull through in difficult times. Once you gain trust, you can then be courageous and push the envelope knowing that if anything goes wrong you have a safety net.
When trying something new, always make sure you have plan B or a fallback. Start by asking "what if it fails?" This is not being pessimistic. This is being realistic. Sometimes, the downside is so minimal that there is no need to take any measures. If it fails, it fails and no harm done. Let's consider a concrete example: suppose you decide with your team that you need to switch to a new NoSQL database with a different API. What can go wrong? Plenty.
The new DB may have some serious problems you only detect after migration. The migration may take longer than anticipated. Major query logic may turn out to depend on special properties of the original DB. How can even think of something like that? Well, if your data access is localized to a small number of modules and if you can test the new DB side-by-side and you have a pretty good understanding of how the new DB works, then you should be confident enough to give it a go. To summarize — being courageous is not taking unacceptable risks and being impervious to failure. It is taking well-measured risks based on sound analysis of the situation and full trust that you know what to do if things go south.
Posted by Gigi Sayfan
on May 12, 2016
"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." — Brian Kernighan (co-inventor of the C programming language).
You shouldn't take it literally of course, but there is a grain of true to both parts: debugging is harder than programming and also, if you write your code too cleverly, you'll have trouble debugging it. One lesson to take away is that you should write clear and simple code that is easy to understand and troubleshoot when something goes wrong.
Another way to look at it is that when something does go wrong you want all the help you can get to fix it. One of the main the Agile practices is pair programming, where two programmers collaborate in front of a single screen and keyboard — probably the least popular in the Agile arsenal. However, pair debugging is taking all the goodness of pair programming into the debugging experience. You don't have to feel you're all alone against the inscrutable code. Your partner will help you through tight spots and see things you don't notice. Sometimes just explaining the problem to your partner may shed a light on what's going on. I've had a very good track record where the combined efforts of the pair found the root cause after a single programmer was stuck for a long time banging their head against the metaphorical wall.
My recommendation would be that when you face a difficult problem, try to solve it on your own, but as soon as you run out of ideas or feel overwhelmed call in the cavalry. Describe what you know and what you have done so far and you'll be surprised that often that alone will get you unstuck. If not, your partner will probably have some ideas regarding how to move forward. Happy pair debugging.
Posted by Gigi Sayfan
on May 2, 2016
The ability to hire excellent software developers is a limiting factor. There is a real shortage, and as a result there is serious competition for every qualified candidate. As a manager on the lookout for talented developer I experience it personally. There are two very clear trends here. Candidates who put themselves on the market will get queries from more companies than they can handle. In one hiring platform that I use shows competing offers and it is common to see a well-qualified candidate receiving 20+ invitations for initial interviews. The other trend is that the big companies out there: Google, Facebook, Uber, etc., use their war chests to offer unheard of levels of compensation. An engineer decides which company to join based on many factors. But, if a big company offers $50K - $100K more than competing offers (in total compensation) then it's very likely the engineer will pick the big company. How can a small startup compete? By using its relative strengths - speed. This is where Agile hiring methods can give you the advantage.
What if you compress the entire hiring process so that you can get an actual job offer out to a candidate, before they have even had a chance to set up phone interviews with most of the other companies who might be interested in them? In the best case scenario, maybe your competitors are so slow that they didn't get around to even sending to initial invitation. I will not discuss here how to find those candidates.
Let's start from the assumption that you have some source of candidates. With Agile hiring your focus is to minimize the time from zero to offer (or rejection). The typical hiring process consists of:
- Review of resume/application/cover letter
- Phone interview[s]
- Coding challenge
- Face-to-face interviews
- Reference checks
- Background check
- Offer letter
The goal is to avoid wasting valuable time by rejecting inadequate candidates early in the process. Let's see what we can get rid of. The phone interview can go away. I very rarely reject a person based on the phone interview. If the original application is solid, the candidate will make it to the next phase. The coding challenge can go away if the candidate has a decent GitHub account. The reference checks can go away. I never got any useful information whatsoever from a reference the candidate provided. So, here is the Agile process:
- Applications are processed by an experienced technical person who can look beyond buzzwords and evaluate coding competency based on public code.
- Immediate invitation to face-to-face interviews, or if problematic, video conference interviews (easier to schedule).
- Until the interview exchange information with the candidate for expected compensation and role. (This may be a phone conversation, but not a gating review to delay the face-to-face interview that was already scheduled.)
- At the end of the face-to-face interview the candidate should have an offer in hand.
Negotiation may take place and background checks can be conducted later. That's it. You could have a new engineer within 2-3 days since first contact vs. 2-3 months. Think about it. If you're worried about wasting people's time with too many interviews, consider having fewer people interview each candidate. It is not a rule of nature that every team member must meet and approve every candidate. If you are confident in your filtering abilities, then maybe just the hiring manager and two other people need to speak with the candidate.
Posted by Gigi Sayfan
on April 14, 2016
Before the Agile Manifesto there was Extreme Programming. I discovered it through the http://c2.com wiki — the first wiki and was instantly impressed. Extreme programming is indeed the extreme. There are values such as communication, simplicity, feedback and courage. Later respect was added. But, courage is what caught my attention. Too often, I see people operate out of fear, which is simply debilitating. Even the "move fast and break things" movement is not courageous because they hedge their bets and concede that they'll break things.
Extreme Programming is different. It doesn't make excuses. It doesn't hide behind trade-offs such as, if we move fast, things will get messy. No, the Extreme Programming stance was different. What happens if we take all the best practices and turn the knob to 11?
There are many practices and you can read about them detail here. Some of them were revolutionary at the time. How did extreme programming fare? It depends. On one hand it ushered the era of Agile and, as such, was revolutionary. But, on the other hand it was too strict to follow exactly. The Chrysler Comprehensive Compensation (C3) project that was the real world test bed for Extreme Programming was cancelled after 7 years and a plethora of other Agile methods exploded on the scene. Many of the original extreme programming practitioners such as Ward Cunningham, Kent Beck and Martin Fowler became well-known thought leaders, published successful books and continued to advance the state of software development. I definitely learned a lot about software development from reading and trying to practice variants of Extreme Programming.
Posted by Gigi Sayfan
on March 21, 2016
Agile methods can accelerate development significantly compared with traditional methods. This is true in particular for programming where fast edit-test-deploy cycles are possible even on a massive scale (look at Google or Facebook for good examples). However, the database is often another world. If you keep a lot of data in your database (and who doesn't?) then changes to the database schema may not be as swift as regular code changes.
There are several reasons for that. The data is often the most critical asset of an organization. Losing/corrupting the data might lead to the demise of the company. Imagine a company permanently losing all their user information, including billing. This means that any such change should be taken with extreme caution and with lots of measures to detect and revert any change. This is not easy to do safely. Another reason is that the database if often at the core of many applications and some database changes lead to cascading changes in those applications.
Sometimes, schema changes, such as splitting one table into two tables, require data migration from the old to a new schema. If you have a lot of data that may be a long process that takes days or weeks because you still need to support the live system while the migration is ongoing.
So, is the database doomed to be this thorn in the side of Agile development? Not necessarily. There are some strategies you can employ to minimize both the risk and the effort to make database changes. First, use multiple databases. The data can often be split into multiple independent data sets. Any change to one of them will not impact the others. Another approach is use schema-less databases, where the application deals with the variety of implicit formats--at least during migration periods. Finally, you can get really good at managing database schema changes and build the mechanisms to support it, migrate data and ensure you have working recovery mechanisms. This takes a lot of work, but it is worth it if you work on innovative development and need to evolve your database schema.
Posted by Gigi Sayfan
on March 15, 2016
Flow is a mental state where you're in "The Zone." Time flies by, everything just works and you're totally focused on the task at hand and apply yourself fully. You can see basketball players attain it sometimes where they just can't miss a shot. But, you can also observe and experience it programming. It's those amazing sessions where in several hours or a day you churn out something that typically takes weeks.
These 100x productivity periods are highly coveted. Not just because of how much you get done, but also because it is one of the best feelings in the world when you totally realize your potential. To get into flow you need an obstacle free environment--no distractions, all available resources at hand. Everything must be streamlined.
In software development, Agile methods provide the best breeding ground for getting into flow. Your tasks are broken down into small enough chunks that you can comprehend them, the automated tests, continuous integration and deployment smooth the process of verifying code. Pair programming is arguably another fantastic technique to get into flow. The driver is just programming, while the navigator deals with all the stuff that typically would distract the driver if they were programming alone. For example, looking up documentation, checking the status of the latest test run and scouring StackOverflow for that helpful snippet. The most important part might be that you can't just go HackerNews binging in the middle of a pair programming session. When you have two people working together, one keeps the other honest as far as focus goes. I recommend that you explicitly experiment and find the best way for you to achieve flow. It is well worth it.
Posted by Gigi Sayfan
on March 9, 2016
Many Agile methodologies promote a daily standup meeting. The idea is that people are uncomfortable standing for a long period of time, so it will help keep the meeting short. The goal of the daily meeting (typically first thing in the morning) is to get a sense of what everybody is working on and if there are any obstacles. But, is the daily standup meeting effective?
I have worked for many companies that implemented some variant of Agile development and some of them implemented daily standup meetings as well. Based on my observations, the daily meeting is just a ceremony that doesn't really contribute much. Let's consider two different teams: small and large. The small team is generally fewer than five people. The large team is likely more than ten people. Teams that have between 5 and 10 will behave as either small or large depending on the team members and the culture. Small teams often work in the same room/open space. They already know what everybody does and obstacles can be addressed on the fly by turning around and talking to your teammate.
Large teams may be split physically and functionally and often work on multiple projects or separate aspects of the same project. The daily standup meeting either takes a long time when every member takes the time to explain what they're working on and what issues they face or is kept short by rushing through and having each team member mumble a few sentences. There is no time for questions and often people are not really interested in the daily struggles of others and just want to get on with their day. Any serious discussion requires a separate meeting with the relevant people.
Given those concerns, what are the benefits of the daily standup meeting? It can contribute to team spirit, serve as a launch pad to a more in-depth meeting and occasionally someone may have insight about how to overcome a particular obstacle. The team lead may gain some benefit by getting a concise summary from the whole team, and finally it ensures that everybody is in the office at the same time!