page updated on July 18, 2015

Does Agile Development Allow Long-Term Planning?

What we now call "Agile software development" started as a reaction to heavyweight development processes in the '80s and '90s. Kent Beck's 1999 Extreme Programming Explained served as an early exploration of the space, while the Agile Manifesto laid down governing principles.

In the past 15 years, agile has grown from XP to include a suite of methods including Scrum, Lean, and Kanban. It's also become a credible way to develop software in the face of uncertainty.

One of the principles of XP often seen in agile development is the reminder to build only enough software for the features you need at the moment. This is YAGNI, or You Aren't Gonna Need It. Critics of agile (and especially XP) claim that this is anathema to long-term planning, and that the practice leads to brittle software with huge changes in design resulting from poor planning.

While mistakes can (and do) happen, mistakes don't have to happen. Agile teams can (and do) plan for the long term without sacrificing their flexibility.

What is an Agile Backlog?

It's a truism in software that you can't do everything at once. You can only write one line of code at a time and you can only finish one feature at a time. In the 40 hours you work every week, you can accomplish a finite amount of work.

The question of what work you should do is up to a stakeholder. The question of how much you can do in a week is one of resources and constraints.

Given these constraints, agile development tries to push the question of what to do to the stakeholder. What's more important? Which feature would you rather have sooner? Which feature can wait? The question of how long it will take belongs to the developers. Thus, given a list of features, developers provide their best estimates as to what they can deliver in the next week or so and the stakeholder sets priorities. If there's too much work to do this week, something will have to slip to next week. If there's not enough work, something can move ahead from next week.

For this planning to work, the stakeholder needs a large stack of potential features—far more than the team can get done in the next two weeks. In a mature project, this stack may include several weeks of work, if not months.

The stack isn't static. It's likely to change from week to week. That's one of the goals, in fact. Agile development intends to support changes in business goals at the granularity of a week. If the needs of the business change, new features can enter the stack as frequently as they need and other features can get pushed out further.

Developers work on the features the stakeholder decides have the highest priorities. They follow the principle of YAGNI and build (and design) only as much as they need to support the current features. After all, the business priority of next week may change.

In practice, there's less churn than this description makes it sound. (Though if a business is flailing, very few project management methods can save it.) After the first few weeks of development, the backlog tends to grow stable. Developers work through features as they come in. Items at the top of the priority stack get the most attention. Items further down get less attention.

Even though the backlog tends to stabilize, making development velocity and feature delivery dates reasonably predictable, the combination of weekly planning and YAGNI tends to convince agile critics that there's no long term planning or architecture going on. In some cases that may be true, but it doesn't have to be.

Peekahead Planning with an Agile Backlog

What's the point of architecture, after all? The same as long-term planning—to reduce the risk of rework by building software that allows you to keep developing it. To some extent, the relentless focus on automated testing and regular refactoring (thanks, XP) helps agile developers avoid making messes, but if you only ever keep the short term, week-to-week development in mind, you can get into situations where you wished you'd known a feature was coming so you could lay more groundwork for it when working on something related.

Other times, it's just nice to know what you're about to get into when thinking about who's going to be on vacation, what holiday is coming up, or even just if you can plan some extra time for refactoring or system upgrades or something else.

Here's the secret: your backlog is a good roadmap to what your business wants from development. If your stakeholder is doing a good job about capturing feature requests and setting priorities, your backlog is organized into desired features and can be put against a calendar. (You already know how much work you can get done in a week; plan it out.)

If you have your stakeholder's attention, you can have a regular conversation to figure out what's likely, unlikely, and probably a wishlist idea. This works best at a high-level conversation about business values. Where's the business headed? What deals are in the works? What's changed since last time you had this conversation?

Should you plan out architecture as a result of this discussion? Maybe. It's up to you and what works best for your team. You're still balancing between delivering the work you promised this week and building software you might need in the future, but you have a source of information to help you make better decisions now.