Agile Guide

Introduction

In one of his first development jobs, Kent Beck was given an assignment that he estimated would take himself and another programmer six weeks to complete. But at the beginning of the project, his partner was reassigned to another task, leaving Kent to complete the project alone. Working on his own, he completed the project in twelve weeks—twice the allotted time, a reasonable adjustment considering he had half the team members he had been promised. Still, his managers were unhappy. They harassed Kent for the six-week “delay.” He felt like a failure even though his original estimate had been right on the nose.

This is just a sampling of the kinds of expectations that burdened software developers in the 1990s. Companies were using sweeping, inflexible processes, like Waterfall—which had been designed in the 1970s to manage massive spacecraft projects, not meet the changing demands of software development in the internet age. As a result, it was difficult to make changes to a project that had already started, working software wasn’t delivered until late in the process, and quality suffered.

Developers had enough. Independently of one another, programming leaders started to create their own “lightweight” processes as alternatives to the “heavyweight” processes in place at many companies. These new processes came in many forms—Extreme Programming, Scrum, and more—but their adherents sensed they shared similar philosophies. In February 2001, a group of seventeen programmers met in a ski lodge in Utah to try to find common ground.

Nestled in the snow-covered Wasatch mountains, Agile was born.

What is Agile software development?

Agile, in the words of its founders, “is the ability to create and respond to change.” It’s an approach to software development that prioritizes flexibility, collaboration, and frequent, small deployments. It’s not a software development framework, rather, Agile is a mindset—a set of values and principles that helps companies make good decisions as they adapt to changing circumstances. The creators of Agile codified these values and principles in the Agile Manifesto and the Twelve Principles of Agile Software, which we’ll discuss in more detail in the coming sections.

Agile arose as a reaction to the problems inherent to heavyweight software development processes, like Waterfall. The issue with heavyweight practices is that their success typically depends on following a strict plan, which can easily fall apart when teams face unanticipated obstacles. While some teams can fix this by getting better at planning, in most cases, change occurs that no one could have predicted—you have to reassign developers to fix an urgent issue, your customer changes their mind, or your competitor releases a feature that renders the one you were working on obsolete. In Agile, instead of trying to be predictive and anticipate every possible eventuality, you become adaptive and work on becoming flexible.

While these ideas weren’t entirely new—iterative development practices had been around since the 1950s—the creation of Agile marked a new chapter. It’s an adaptive development philosophy, not a defined process. Agile united developers with shared values, even if their individual approaches diverged. Crucially, it helped organizations rethink their software development processes—today 97 percent of organizations report using Agile methods. Let’s explore this philosophy in more detail and discover what makes it so compelling.

The Agile Manifesto

The seventeen developers who met at that ski lodge in Utah—the “Agile Alliance,” as they fashioned themselves—created the Manifesto for Agile Software Development to summarize their new philosophy, along with the Twelve Principles of Agile Software to further support and explain their thinking.

The manifesto is a fascinating document for a number of reasons—“not the least of which was getting 17 people to agree to it,” wrote Agile alliance members Martin Fowler and Jim Highsmith. At just 68 words long, it’s easily digestible yet expresses some quite profound ideas about what developers should value and prioritize. Let’s have a look.

The manifesto is notable, first of all, for its introduction. The signatories wanted to make it clear that—while they are considered experts and leaders in their field—software development is an ongoing practice, and they’re still learning. The choice of the word “uncovering” was deliberate, Martin and Jim say, “to assure (or frighten) the audience that the Alliance members don’t have all the answers.” They simply wanted to share what they had learned and were continuing to learn.

The manifesto is also unusually “mushy” for a software development document (to borrow the phrasing of Agile Alliance member Bob Martin). It emphasizes creating human connections, both with teammates and customers. That’s by design: “Software development is a people business,” said James Newkirk in a talk for the Agile Alliance. Building software, especially on a large scale, is a collaborative process—there are many stakeholders, both internal and external. The Agile Alliance wasn’t arguing that having processes in place or negotiating contracts are bad practices, just that they’re inadequate for successfully navigating software projects. Collaboration and teamwork take precedence.

Finally, the manifesto turns time-honored project management wisdom on its head. “Traditional project management practices assume that achieving a plan equals project success equals demonstrated customer value,” write Martin and Jim—but this simply isn’t true in the modern software development landscape. Today’s projects tend to be more volatile, with frequently changing demands. A project’s initial conception has little bearing on its ultimate success. It’s more important for teams to be flexible than to execute a plan to the letter. Or, as Martin and Jim put it, “facilitating change is more effective than attempting to prevent it.”

Perhaps most significantly, the Agile Manifesto is not prescriptive. There are no rules about how teams should structure their workflows or how often they should meet. Rather, the manifesto offers a mindset that teams can adopt to help them prioritize the right things. When teams are faced with decisions, they can think back to the manifesto and use it to help them make choices in line with their values.

The Twelve Principles

The Agile Alliance further supported its manifesto with twelve guiding principles. These help explain the creators’ thinking and explore the manifesto’s practical implications in more depth. In the words of the Agile Alliance, these are “the guiding practices that support teams in implementing and executing with agility.” Let’s take a look:

While the Agile Manifesto promotes an idealized vision of software development, these Twelve Principles lay the foundational practices that help teams achieve it.

Why Agile?

The simplicity of the Agile Manifesto and the Twelve Principles principles can be deceiving. While the ideals expressed in the manifesto are easy to understand, putting them into practice is harder than it looks—not to mention counterintuitive to the way many organizations function. Large, established companies typically have many processes in place. They may want to keep exhaustive software documentation and negotiate carefully worded contracts to cover themselves for liability purposes. They might not be equipped to respond to adapt to change without extensive change management planning. So why bother with Agile at all? Let’s take a look.

Members of Agile teams are happier

This isn’t just anecdotal: Researchers at Baylor University have found a positive relationship between developers’ use of Agile methods and their job satisfaction. The study found that members of Agile teams have more autonomy—Agile developers tend to be actively involved in project management and have a higher degree of control over the work they do—which promotes job satisfaction and employee well-being.

Agile increases team productivity

Since the early 2000s, case studies across a wide range of organizations have shown significant productivity gains from adopting Agile. According to the most recent State of Agile Report, 51 percent of teams cite productivity gains as a reason for adopting Agile. This may not be surprising. After all, Agile minimizes wasted time and resources by eliminating redundant meetings, unnecessary documentation, low-value features, and so on. But Agile teams may also be more effective because they’re happier—as demonstrated by a University of Michigan study, positive teams are more productive.

Agile improves communication and transparency with stakeholders

Unlike plan-driven project management techniques, which attempt to scope the entirety of a project’s requirements at the beginning, Agile methods focus on cultivating an ongoing conversation between developers and stakeholders. This fosters better communication between developers, executives, and customers—whether internal or external. With more visibility into the development process, stakeholders feel more involved and can offer valuable feedback on an ongoing basis.

Agile results in better software

As laid out in the Twelve Principles, teams using agile methods make smaller deployments more frequently. These shorter iterations allow for faster feedback, meaning that teams can correct problems quickly and design quality into products from their earliest stages. One study found that teams using Agile used QA practices much more frequently than teams using Waterfall and that these QA practices were built in from the earliest phases of software development.

Agile empowers managers to do high-value work

Agile empowers teams to work more autonomously—releasing managers from the obligation of micromanaging their staff. Agile managers still conduct regular check-ins and offer their advice and support where needed, but they can trust that their teams will get the work done. Instead of constantly assessing everyone’s progress, managers can focus on the big-picture problems they’re meant to be solving, such as defining company direction and vision, prioritizing strategic goals, and assigning the right people to the right projects.

Common Agile Misconceptions

Agile isn’t a framework for software development

Agile is a philosophy that teams can use to make decisions about how best to develop software. While many Agile teams do use frameworks like Scrum, these processes alone aren’t Agile. As Agile expert James Newkirk explains, “Process itself is the means to an end. It is not the end.” The end goal, as stated in the Agile Manifesto, is “finding better ways of developing software.” Agile teams will have varying processes and approaches, but they are all working towards the same goal.

Agile isn’t anarchy

While some managers might flinch when they hear phrases like “self-organizing teams,” Agile does not mean chaos and anarchy. Agile teams don’t reject the process, they simply don’t let the process take priority over their teammates or the goals they’re working toward. In fact, Agile teams do typically adopt some kind of software development lifecycle methodology, like Scrum, Lean Development, or Kanban—they just ensure the process is serving their needs rather than sacrificing their needs for the process.

Agile principles aren’t just for software teams

This is a contentious one, as the Agile Manifesto and Twelve Principles were written specifically with software teams in mind. But other teams can also benefit from adopting Agile principles—for instance, NPR uses Agile methods to create new radio programming, and Lonely Planet uses Agile in their legal team. However, Agile works best when the conditions mirror the ones faced by software developers. The Harvard Business Review summarizes these conditions as follows: “The problem to be solved is complex; solutions are initially unknown, and product requirements will most likely change; the work can be modularized; close collaboration with end users (and rapid feedback from them) is feasible; and creative teams will typically outperform command-and-control groups.”

Agile in Action

Agile teams are united in their shared goal—developing better software—and the values they use to get there. They all prioritize effective collaboration and communication, put a premium on working software, and focus on becoming flexible as a way to respond to inevitable change. But where they differ is in implementation. Different Agile teams may use very distinct practices and frameworks to achieve the goal of creating better software. This is a good thing—being Agile is about results, not the process you use to get there. What works for one team may not work for another. There are no cookie-cutter solutions or “best” processes. Every team has to determine what works best for them.

Agile Practices

Here are some of the most common Agile practices used across a variety of different Agile frameworks. While these practices alone won’t make your team Agile, they can be useful tools to support your Agile journey.

Sprint

A sprint, sometimes called an “iteration,” is a timebox for development, usually one to four weeks in length. Before the sprint, teams select a limited number of tasks from their backlog to focus on, with the goal of completing them all by the end of the sprint. (Non-development teams can also use sprints to prioritize and execute their tasks). The short, iterative nature of the sprint makes it easy for teams to adjust their direction and prioritize new work as needed, allowing them to be flexible. Working in this fixed, consistent rhythm can also make it easier for teams to make accurate estimations about how long new projects will take to complete.

Planning session

Before beginning a new sprint or other iteration, Agile teams often meet for a planning session. This planning session typically has two parts: Scoping and planning. During the scoping part of the meeting, the team forecasts the number of tasks they think they can reasonably manage and chooses a limited number of priority tasks from their backlog to complete over the duration of the sprint. Then, they switch to planning—they determine how they’re going to complete the tasks they selected and assign responsibilities to different team members.

Standup

Standups are short, daily check-in meetings—typically no more than 15 minutes in length—where every team member reports their progress. As the name implies, standup meetings are often conducted standing to promote alertness and quick meeting resolution. Standups are an opportunity to catch up, resolve issues or blocked tasks, and identify ways that team members can help and support one another.

Test-driven development

Test-driven development flips the traditional software development life cycle on its head. Typically you would write code, then create tests to verify that the code is functional, reliable, usable, and secure—but in test-driven development, you start by designing tests and then writing the code to pass them. A Cal Poly study found that software developers who use test-driven development “write software modules that are smaller, less complex, and more highly tested than modules produced by their test-last counterparts.” This is important because smaller units of more heavily tested code generally have fewer defects—programmers catch issues and resolve them well before deployment.

Pair programming

Pair programming is a style of programming in which two developers share a monitor—one person, known as the “driver,” writes code while the other, known as the “navigator,” reviews it. While they work, they explain their thinking to one another (this is why pair programming is sometimes also referred to as “programming out loud”). Every few minutes, the driver and navigator switch roles, ensuring they spend equal time on writing and reviewing. In the process, the two developers learn from each other and gain a better understanding of their own programming habits.

Agile Frameworks

Below, we’ve described some of the most important Agile frameworks used in software development. While not exhaustive, this list should nonetheless give you a good overview of common Agile frameworks and help you determine which, if any, may be right for your team.

Scrum

Scrum is the most popular Agile framework. Used by over 12 million people worldwide, it’s no wonder that in many peoples’ minds, Agile is synonymous with Scrum. “Simple to understand, difficult to master,” the Scrum framework is centered around “sprints”—defined work periods of anywhere between one to four weeks—with an emphasis on frequent, small deployments. A Scrum team has three main roles:

• The Product Owner: Acts as the voice of the customer and prioritizes tasks from the product backlog to be completed each sprint
• The Scrum Master: Acts as the team’s coach and facilitates teamwork by ensuring everyone understands the process, removing obstacles, and helping to maintain good relational dynamics
• The Development Team: A cross-functional, self-organizing group that both plans and executes their tasks over the course of one sprint

Before each new sprint, the Product Owner recommends tasks from the team’s product backlog to focus on and the team plans their sprint. For the duration of the sprint, the team has daily “scrums”—brief meetings to discuss progress and resolve any issues or blocked tasks. At the end of the sprint, they host a sprint review with external stakeholders to update them on their progress. Finally, the team meets on its own for a sprint retrospective to discuss what went well and what they can improve on for next time before the whole process begins again.

Extreme Programming

Though not as common today, Extreme Programming (XP) is one of the oldest and most disciplined Agile frameworks. Started at Chrysler in the 1990s, the framework emphasizes a high quality of software alongside a high quality of life for developers. In XP, one team member, designated “the customer”—typically a business representative—provides requirements, sets priorities, and directs the project. The team cultivates a “whole team” mentality in which members sit together, use collaborative practices like pair programming, and are encouraged to become generalists, not specialists. The goal is to create seamless code, as though it was written by a single, highly competent individual.

Lean Software Development

Lean Software Development, established by husband-wife team Tom and Mary Poppendieck, applies the principles of lean manufacturing—pioneered by Toyota factories in the 1950s—to software. As product development, whether physical or digital, became more closely connected with software, it made sense to treat them as parts of the same process. In the words of the Poppendiecks, “Today, most software development is not a stand-alone process, but rather a part of developing products or services. Thus lean software development might be considered a subset of lean product development.” Lean teams abide by the following seven core principles:

1. Eliminate waste
2. Amplify learning
3. Decide as late as possible
4. Deliver as fast as possible
5. Empower the team
6. Build quality in
7. See the whole

Kanban

An offshoot of Lean Software Development, Kanban also applies lean manufacturing theories to software development (and any kind of knowledge work). But it goes a step further, creating a defined process for teams to follow to plan, track, and optimize their work—which you can read all about in our Kanban Guide. The most iconic feature of the process is a Kanban board, a tool that teams use to track their work as it moves through the software pipeline from planning to completion.

This board acts as a mirror, allowing teams to visualize their work, identify blocked tasks, and optimize their throughput. In the words of David J. Anderson, who codified Kanban as a process for knowledge workers, “Kanban is an approach that drives change by optimizing your existing processes.” Kanban can also be combined with Scrum to form a hybrid, Scrumban.

Agile Teams

Yet practices and frameworks pale in comparison to the most important component of Agile—your team. Like the manifesto says, “Individuals and interactions over processes and tools.” But, outside of any given framework, what should an Agile team look like?

The most important quality of any Agile team is that it’s self-organizing. The eleventh principle from the Twelve Principles of Agile Software states that “the best architectures, requirements, and designs emerge from self-organizing teams.” That means the same group of team members should scope, plan, and execute their projects. This applies whether the team is collocated, i.e., in the same physical space, or distributed, i.e., remote. In order for a team to effectively self-organize, it needs to be cross-functional—team members need to have the skills and expertise to take a project from initiation to completion. Ideally, as team members work together, they will become generalists and learn to perform multiple functions. In the next section, we’ll delve into how to create an Agile team and set it up for success.

Is Agile right for your team?

Before we take you step-by-step through the process of implementing Agile in your organization, it’s important to consider whether it’s a good fit. Agile tends to work well for development teams and in knowledge work environments that face similar challenges to software development, according to the HBR. In the most favorable conditions for Agile, the problems to be solved are complex, no one knows the solution at the outset, requirements change, users can provide feedback, and creativity tends to work better than following a strict plan. For teams who do marketing, product development, supply chain management, legal work, or any number of other functions that face similar challenges, Agile may be a good fit. But for teams who perform routine functions—like sales calls, accounting, or purchasing—Agile won’t be as effective as a more traditional project management framework.

Pick a flexible project management tool rather than the “perfect” framework

Don’t worry too much about picking the “right” framework—being Agile means constantly adjusting your project management process and workflows. Nothing is set in stone. If your chosen framework doesn’t work for you, there’s nothing to say you can’t change it later. But in order to give yourself this flexibility, it’s important to manage your tasks in a way that allows you to modify your system without causing chaos.

Many Agile frameworks promote the idea that teams should make work progress and bottlenecks visible, and traditionally, they used a whiteboard and sticky notes to do that—after all, your work doesn’t get more visible than being on the wall for all to see. However, arranging your tasks physically has the hazard of making your team less Agile. It makes changing frameworks a hassle (who wants to physically rearrange every sticky note?). Not to mention that it’s inaccessible for remote workers and can cause old tasks to get lost or thrown away, making it difficult to reference past projects.

That’s why today, most Agile teams prefer using digital project management tools like Backlog. These digital tools are accessible anywhere with a Wi-Fi connection, making them an ideal, flexible solution for modern teams. Look for one that has an integrated boards feature, which will make it easy to keep work visible without the hassle of a physical board.

Start small, but do start

Many organizations drag their heels on adopting Agile, taking weeks to evaluate frameworks and deliberate on which is best for their business. Alternatively, they’ll try to launch Agile as a massive change program, forcing everyone aboard the new system—whether they want it or not. Neither is a particularly sustainable solution. “The most successful introductions of Agile usually start small,” writes the HBR. Small, incremental change tends to be more effective in the long run.

If you’ve decided that Agile is right for your organization, try starting with just a small group of volunteers and allowing them to experiment. Follow a “staircase approach” by changing just one thing at a time—not only are steady, incremental changes more sustainable, but they also allow you to effectively measure your progress. By altering just one variable at a time, you’ll gain a clear picture of what is and isn’t working, allowing you to make effective changes. Once Agile is working well for this team, you can gradually allow it to spread to other areas of the business. Your initial volunteers will become your most enthusiastic advocates for Agile—and they can, in turn, mentor other teams as they begin their Agile journeys.

Create a “definition of done”

The idea of being “done” a task can be notoriously ambiguous—both in software development and in other fields where Agile is appropriate. Sure, you might be “done” writing your code, but have you created test data, tested your code, ensured it’s deployable, and written documentation? (Similarly, a content marketer might be “done” writing a blog post, but have they copyedited it, SEO-optimized it, and saved it in the blog queue?).

But knowing what work is done—especially at the end of a sprint or other work increment—is essential for planning and optimizing your workflow going forward. That’s why it’s so critical to create an agreed-upon definition for when a task can be considered complete. Some teams may find it useful to create a “done” checklist that their members can refer to. That way, when someone says, “I’m done,” you’ll know exactly what they mean.

Manage resistors and reluctant adopters

Agile has had its fair share of resistors. Many managers once fell into this camp, perhaps reluctant to believe that IT had come up with the solution to their productivity woes. Even the Harvard Business Review didn’t write about Agile until 2016—fifteen years after the original manifesto was created. While Agile has gained widespread acceptance today, there will still be detractors and reluctant adopters.

That’s why it’s so important to take a slow, measured approach when starting Agile. It gives people time to adjust to the new system and feel like they have a part in shaping it. It also gives managers a chance to engage with individuals who might be showing resistance. In these cases, it’s best to try to understand why someone is hesitant about Agile: Is it because they like the old system and don’t want to change? Have they had poor experiences at other organizations? Generally speaking, being empathetic is a more effective strategy than trying to force a change on someone who doesn’t want it. Be patient and try to understand their point of view—they might just need some time to come around.

From “doing” Agile to “being” Agile

Agile is a simple philosophy to understand but one that takes time to master. Many companies start by “doing” Agile—adopting a framework, like Scrum, and practices, like sprints, daily standups, and test-driven development. This is an important starting point, but your Agile journey shouldn’t end there. To truly become Agile, you need to adopt the mindset of agility and make it an integral part of your organizational culture. You need to foster an environment where people feel secure to experiment and fail. In fact, in order to achieve Agile, you can’t allow “being Agile” to become your main goal. Rather, look at Agile as the means to create better software (or other outputs), a happier and more productive workforce, and delighted customers.

Ready to begin your Agile journey? Backlog is the ideal, flexible project management tool designed specifically for Agile teams. Features like boards make it easy to keep work visible so you can quickly identify blocked tasks and optimize your workflow. With an all-in-one project management and version control platform, Backlog allows team members to manage their projects—and their code—from one centralized location. Get started for free.

Further Reading and Recommended Resources

• The Agile Manifesto is where it all began
• The writers of the Agile Manifesto, the Agile Alliance, have compiled all their favorite Agile resources—blog posts, ebooks, webinars, and more. Becoming a member is free!
• Harvard Business Review: Embracing Agile is an exceptionally thorough article on what Agile can (and cannot) do for your business, co-authored by Jeff Sutherland, one of the founding members of the Agile Alliance.
• The State of Agile Annual Report has been the definitive voice on Agile adoption and maturity in the industry for 14 years.