Closed-loop feedback control with Scrum

Scrum is a simple framework to manage complex challenges. Software delivery is a complex challenge. Software delivery encompasses a multitude of complex activities to create and evolve complex products in complex circumstances. Scrum embraces and emphasizes the complexity of software delivery by implementing the only process type that fits its complexity, empirical process control.

Complexity

There are many variables that have an impact on delivering software; requirements, skills, experience, people, teams, technology, integrations, market conditions, company strategies, budgets, regulations and dependencies, to name just a few. Not all known variables are controlled by the people doing the work, although they have to incorporate the impact on the work. For some known variables too much detail is needed to fully comprehend them. Even if a variable is known, its behavior -now or in the future- may be unknown, or different from what is anticipated.

Products are created by people with cognitive capacities. People are not robots, or replaceable pieces of machinery. The outcomes are hard to describe in an exact and detailed way; before, at the beginning or even as the work advances. Every product created is unique and what is appreciated can only be established when released. The activities to be undertaken aren’t predictable with any degree of high precision. They have not been performed before, in the same way, under the same conditions, several times. The environment evolves constantly. And, most annoying, not all variables are known. There is a high degree of unpredictability.

The degree of dynamism of a challenge requires the right process to be in place in order to have control:

Open-loop systems

An open-loop system is designed for execution of a series of preempted steps to result in a defined outcome in a single run. Such a system assumes a near-perfect predictability of the variables that influence the process as well as of the process activities themselves.

For larger problems, typically a chain of open-loop subsystems is created. The output of a subsystem is the input to the next subsystem. Although theoretically risks should be confined to the smaller subsystems, in practice the whole system’s vulnerability increases exponentially. In situations of turbulence and change, deviations and variances accumulate across the various subsystems, e.g. in timing and quality. It is not uncommon for the accumulated problems to surface only at the end of the final subsystem.

Predictive plans and hand-overs of work between separate functional groups are implementations of open-loop thinking. Predictive plans can only include known variables, their known details and their anticipated behavior, creating the illusion that no unknowns exist. Open-loop thinking invites lengthy upfront consideration of all elements of the plan, and ultimately attempt to foresee the unforeseeable.

An open-loop system is -by design- unable to cope with the amount of disruptions and unknowns typical for complex challenges like software delivery.

Closed-loop systems

Closed-loop systems implement frequent opportunities for inspection so adaptations can be implemented. The actual outcome of the system is compared in a timely fashion against a desired outcome. Desires may change. Variances or undesired results are eliminated or corrected in the next or in future runs. Not all variables and parameters need to be known precisely and in detail, as the process is self-correcting. The system requires and creates transparency. Reality is inspected, and exposed, so that appropriate adaptations are undertaken. The people performing the inspections have clear and agreed standards in place to inspect-and-adapt against. Inspection for reporting and status purposes is pointless. Inspection without adaptation is pointless.

Closed-loop feedback control with Scrum

Scrum embraces and stresses the complexity of software delivery by implementing empirical process control. Scrum replaces the open loops of traditional, phase-gate, staged or similar processes with closed-loop feedback. Scrum defines regular opportunities to inspect and adapt. Scrum enables players to halt the traditional rat race. The players are enabled to stop, reflect, learn from inspections, gather feedback over the output and change course, re-organize, update priorities, improve, adapt. Scrum brings reality back in the game. Scrum brings transparency. That is pleasant when all is progressing well. That is crucial when all is not progressing as hoped for. It allows re-positioning and correction.

In Scrum all work is organised in Sprints. Sprints are time-boxed to assure timely inspection and adaptation. A Sprint forms an ‘inspect and adapt’ cycle of a few weeks at most that wraps the 24-hours ‘inspect and adapt’ cycle of the Daily Scrum:

  • At the Daily Scrum the people doing the work inspect their progress, and identify their most important work to do next within the container of the Sprint. They use the Sprint Backlog, the Sprint Goal and a progress visualization to self-organize within the Sprint. The daily cadence assures they never get out of sync for more than 24 hours.
  • A Sprint is a cycle that starts with identifying and interiorizing the most important ideas instantiated on Product Backlog. Sprints end with an inspection of the product Increment that was actually released or could be released, as well as how it was built, the process, the interactions, the technology at play. As with all inspections, they are forward-looking. They serve the purpose of adapting.

All events of Scrum set a frequency for the inspection and adaptation process, where the artifacts contain the information to be inspected and adapted. Scrum describes the accountabilities needed to perform the inspections and adaptations.

Organizing work in Sprints allows people to take a breath, to break with the traditional rat races, and work at a sustainable pace. Explicit reflection moments are introduced that are crucial for humans to thrive in cognitive, creative work of high complexity, like software delivery is. Within a Sprint, additional feedback loops are created, e.g. through agreed work and development standards.

Note: above text is adapted from my book “Scrum – A Pocket Guide (A smart travel companion)”. If you have more time to spend, consider reading it.