What is Flow, Actually?
Last week, I was driving on a two-lane highway at rush hour. I had to put up with being stuck in a traffic jam. As often when I’m bored, I thought about software development (is that odd? Asking for a friend…). The highway, in particular, reminded me of a term that entered software jargon through disciplines like DevOps and Lean Thinking: flow. In my current situation, something on the highway prevented cars from going to their destination. Similar to blockers preventing your code from being finished or reaching production. Something impeded the flow (spoiler: it was a poorly positioned construction site). I observed that my understanding of flow in software development was quite the same as my understanding of how traffic jams work: a gut feeling. Following my curiosity, over the next few days, I kept looking deeper into the concept of flow. I wanted to discover a more nuanced understanding of the term flow - what it is and how to achieve it.
An important concept to grasp for understanding flow is value streams. A value stream incorporates all activities in developing a software product that are involved in providing some utility for customers. Typically, such activities involve coding, testing, design, integration, and deployment. The role of development teams is to move features or defects through a value stream by performing these activities. Typically, we denote things moving through the stream as work items.
What’s interesting about value streams is that as long as work items are still within a stream, they do not deliver any value to customers yet. Think of unfixed bugs or unfinished features. In that sense, it is necessary to finish all activities within a value stream before the work item can actualize any value.
What is Flow?
Flow describes a continuous, uninterrupted movement of work items through a value stream. It entails the reduction of obstacles that hinder work items to go through the value stream smoothly and steadily. In order to quantify flow and the degree of such obstacles, metrics exist which measure the speed at which work items go through a value stream. Lead time, for example, describes the time it takes a feature to reach production starting at the point a customer requests a change. Process time expresses the time between work starting and the code reaching production. That means lead time also includes activities that precede active work on an item, like the time a feature idea stays in the backlog before being pulled.
Shorter lead and process time obviously means that a development team finishes and releases work items faster. Usually, value streams containing smaller work items have shorter lead times because they are usually finished faster, bind fewer resources and are easier to test and deploy. Being able to release new ideas and bugfixes faster not only leads to a better product and happier customers but also makes experimentation easier and early feedback possible. A continuous flow of value also makes feedback more assignable. Large releases usually mean substantial disruptions: for your customers as well as for the deployment environment. It becomes hard to pinpoint what exactly caused observables changes in system and user behavior, like performance or customer engagement metrics to go down. If releases are smaller, you can pinpoint sudden changes in metrics and user behaviour more precisely.
Measuring Flow
While lead and process time describe the speed at which work items flow through a value stream, it describes only one dimension. You could, for example, have an incredibly short lead time but only deliver one work item per year. There is more to consider for establishing flow than only speed. The following five flow metrics show additional aspects:
Flow velocity describes the number of items being completed within a certain time frame. It measures how much value a stream delivers to customers.
Flow efficiency characterizes the fraction of time an item gets actively worked on while it moves through the value stream. This metric shows the amount of waiting time work items are exposed to.
Flow load defines the number of items currently in the value stream.
Flow distribution depicts the proportion of certain flow item types (features, defects, debts, …) represented in a value stream.
Flow time describes the duration an item needs to go through a value stream. This is what lead time and process time describe.
These metrics help us to get the subtlety of flow right. For example, flow distribution for mature legacy products might look different than for greenfield products, with a higher distribution toward defects and debts. In another example, a high flow load indicates congestion within the value stream which lowers flow time and flow velocity down the road.
Flow velocity and flow time are useful target metrics. It means that streams continuously deliver value and teams are able to react fast. Upstream from that, control flow load and make sure to have a high flow efficiency by removing blockers and wait times (more on that below). Use flow distribution to measure which kinds of work items you deliver and make sure that it matches the customers expectations.
What Hinders Flow?
There are some blockers to flow common in many organizations. Manual tasks are one aspect: activities that could be automated but fall back to manual labor, sometimes only executable by a single person. Waiting times are another common one. This is especially noticeable when teams have to wait for other teams. That’s often because of technical and organizational dependencies. Technical dependencies, like the need for other teams to change their APIs so you can implement a feature. What makes these dependencies even worse is when teams are not autonomous enough to make their own decisions and need to seek manual approval from centralized decision-makers.
It should be a primary goal to remove blockers and bottlenecks. The system as a whole can not move faster than its slowest part.
Fast Flow
Achieving fast flow resides on some fundamental principles:
Because large work items tend to take longer, bind a lot of resources and therefore are likely to congest value streams, favor smaller work items. They increase flow time and flow velocity while decreasing flow load.
Remove bottlenecks and waste: Introduce work in progress limits to circumvent congestion and automate manual tasks to reduce waiting times. On top of that, detect avtivities that do not add value and reduce or remove them. Take a look into Value Stream Mapping to to make such activities visible and explicit.
Fast feedback: find a way to incorporate feedback as fast as possible. Use the information to fix issues and refine the product iteratively.
Cross-functional and autonomous teams: break down silos and remove handoffs. Teams should be aligned towards a common goal and value stream. In order to move faster and establish responsibility, teams should be autonomous and able to make their own decisions. There should be no centralized decision making with lengthy approval processes.
Conclusion
There is more nuance to flow than is superficially visible. Metrics like lead time do not fully describe how work items pass through value streams. What makes this topic even more complex is that organizations do not have a single, linear value stream but a complex, interconnected network of multiple value streams and feedback loops. However hard such systems are to measure, the most important goal you should aim for is to release continuously and in small batches. Only then can you optimize iteration speed and shorten feedback cycles.