The software development industry has long suffered from a dogged obsession with identifying, checking, detailing, double-checking, describing and documenting perfect and complete sets of requirements before allowing ‘development’ (typically only the coding or programming part of it) to start. Typically such a requirements phase consumed vast amounts of valuable energy, time and budget, causing gigantic delays before the core work of software development even begun. Besides this trauma, such a requirements approach often caused what became known as ‘analysis paralysis’. The obstinate desire for perfect requirements completely grinded projects as an overwhelming amount of information was collected, along with the abunding amount of requirement interconnections. We lost overview, we certainly couldn’t document it anymore, let alone explain it and get a sign-off. The insurmountable information overload caused a hard stop. Basically, projects already got off track during this preparation phase, got beyond recovery and were canceled before the real work could start.
All energy was spent on what in the end is only an intermediate deliverable. The requirements turned into a goal in themselves. It was ignored that only during implementation the real discovery of what works and what not works happens, when functional options are validated against technology and turned into usable versions of product. And, ultimately, real progress of any software product development effort is in working software, versions that can be released to and be tested by users.
This is reflected in the value statement of the Agile Manifesto:
“Working software over comprehensive documentation.”
as well as in one of the 12 Agile principles:
“Working software is the primary measure of progress.”
Scrum implements this through the demand to have Done Increments of software by the end of every Sprint, where a Sprint takes no more than 30 days, and often less.
This demand supports breaking away from the false illusion that there is a direct correlation between the perfection of a set of requirements and the value of the software produced. There is no such cause-effect linearity. The chance of success of a software development endeavor is not linearly related to the level of detail, documentation and completeness of its requirements. Any requirement, no matter how well documented and agreed upon can still become obsolete, its content may change, its appearance when coded may differ or its delivery may not result in the expected value and satisfaction. Regardless how well it was thought through in advance, documented and agreed upon.
Despite the adoption of Agile frameworks, and the widespread adoption of Scrum in particular, we have a long way to go.
In many environments, as part of what is called ‘Agile’, requirements are now captured as User Stories, sometimes even on index cards. However, the hidden desire for perfection hasn’t changed. The belief that more detailed requirements will lead to better software remained intact. The courage or insights to acknowledge that progress is not measured on intermediate artifacts is missing. The ‘analysis paralysis’ phenomenon is replaced by a ‘story card hell’ [1]. Efforts are undertaken to collect all stories that can possibly be thought of, and for all stories identified, attempts are made to have every detail documented, in a tool or on its index card.
The format has changed, the obsession with perfect requirements hasn’t.
In the system called ‘Scrum’ one, and only one, source of input is defined, Product Backlog.
All work for a product is captured in such Product Backlog, regardless whether the work is performed as a ‘project’ or not.
Product Backlog management in the empirical process that Scrum implements, holds that items in Product Backlog are added and changed as more is learned from the actual work. There is no need, nor a demand to have a full Product Backlog in place before development can start. A Product Owner having sufficient trust, ideas and funding often even provides the best starting position for innovation and new product development.
Product Backlog is incrementally managed:
- Ordered against time (for assumed value). Understanding that value remains an assumption until the work is released.
- Gradually decomposed.
- Regularly refined, cleansed and updated.
Scrum prescribes no specific format or tools for a Product Backlog, or the items on a Product Backlog. Scrum certainly has no obligation to use the format of User Stories for all items on the Product Backlog.
Product Backlog serves transparency.
Product Backlog is expected to hold all types of work required to deliver or enable delivery of releasable Increments of product by the end of every Sprint.
Although Product Backlog might hold references to other sources and deliverables, in itself it remains the single source of truth.
Scrum’s principles and rules may not create a requirements paradise, but as a framework it allows finding distinct ways to deal with requirements in a way that overcomes ‘analysis paralysis’ without ending up in a ‘story card hell’. Keeping Product Backlog minimal and removing items (the tea leaves effect) is probably the most overlooked Product Backlog management strategy, yet it will get you a long way. Progress is measured and visualized, Sprint by Sprint, upon Product Backlog converted into Done Increment and the new insights applied on open Product Backlog.
Still. I see the challenges.
We can do better a job providing inspiration to enact the simplicity of Scrum to better deal with complexity. We haven’t been clear enough in helping teams, organisations and Product Owners more effectively utilise Product Backlog, one Product Backlog.
The challenges in the more effective utilization of Product Backlog are to start using Product Backlog to decompose from goals, objectives, functions and domains to actionable work, certainly in scaled implementations of Scrum. Or to use Product Backlog to create value and impact, while managing dependencies. To use Product Backlog to forecast, create product roadmaps, align work with other products and report on progress, budget and value. To use Product Backlog at the program and portfolio level.
Product Backlog can serve these purposes. We could have made people be aware of it more.
Product Backlog is open to adding metadata and properties for and grouping notions of Product Backlog items. Such additional metadata allows creating different views into Product Backlog while upholding one Product Backlog as the single source of truth, while upholding total transparency. Transparency is not upheld through an immense inventory of requirements with an immense inventory of detailed descriptions, not even in one Product Backlog, let alone spread across separate whatever-type-of backlogs.
Many techniques and practices exist surrounding Product Backlog. They might require their own artifacts, yet they cannot replace Product Backlog.
Product Backlog serves to uphold maximal transparency in product development when employing Scrum. Product Backlog, in the end, is all the plan you need.
[1] The term ‘story card hell’ was first used by James Shore, in a 2005 presentation, http://www.jamesshore.com/Presentations/Beyond%20Story%20Cards.html
“Understanding that value remains an assumption until the work is released.”
—> I would emphasise “until work is used by the end-user and feedback is received”. I see challenges in organisations understanding ‘done’ as released, but not actually valuable for the end-user, or not even actually used by the end-user.
Inspired by the lean startup, I’ve been talking about how the lean startup approach introduces hypothesis -driven development:
– replacing “requirements” with assumptions identified, prioritising these according to uncertainty
– formulating a hypothesis – explicitly acknowledging what we know now is simply a belief – not a certainty
– thinking about what the learning objective is, thinking about a (lightweight) experiment to test the hypothesis and proven it wrong or right
this sounds very ‘scientific’ and in fact it is, but it makes the proces more objective- quite often ‘requirements’ are driven by opinions, gut-feelings, and (only) stakeholders’ preferences.