BS

The software development pipeline

Jan 14, 2019 · 4 min read

Pipelines.

They are very important in software development, in my opinion. If you do not get your pipeline right, you likely have problems within your team(s).

So, what is a pipeline?

Well, think of it has a conveyor belt in a factory. The software development pipeline is a conveyor belt. From projects, to requirements, to building/testing, to deploying, and to everything in between. Those processes/items are moving along the conveyor, ideally in a smooth fashion.

This is analogous to the Lean Philosophy. Two principles of lean are:

  1. Make the value-creating steps occur in tight sequence so the product will flow smoothly toward the customer.
  2. As flow is introduced, let customers pull value from the next upstream activity.

If you have a bottleneck in the pipeline, your conveyor belt is being strained, which causes issues down the line, at some point, somewhere.

For example (not making any assumptions about how the teams are formed in your organisation)

What we are aiming for is “Just in time delivery” for each process along the conveyor belt, another aspect of the lean methodology. If requirements have been delivered just in time to engineers, we know they add value now. If the requirements making their way to the engineering teams are 6 months old, are they still valid? Will they still add value? I’ll take a gamble and say at the very least, you won’t know.

The process flow, from initiation to deployment should be calm and seamless, in my opinion. And once you have that, you can work at that pace - constantly.

Quoting the agile manifesto

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

You can only achieve that if you have a decent pipeline.

Engineering teams

So, engineering teams. I’ve been involved in many engineering teams over the last decade or so, and they should always be able to occupy themselves without the oversight of management. I only bring this up, as I firmly believe that out of all the teams and processes in the pipeline, the engineers should be able to deal with the bottleneck, as long as you have either a strong culture or strong leader to set the expectations.

Too much work coming down the conveyor belt?

This should sit with the Product Owner, or Project Manager - a role like that. It’s a planning issue mainly. Yes the engineering team may be slacking or have issues estimating, but by and large, it’s a planning issue.

Even if the engineering team estimated incorrectly, the people planning projects should have plans to accommodate that. Estimating large pieces of work may be the first thing to eyeball if this is the problem, but that’s a post for another day.

Too little work

Then they should be adding value wherever they can.

Conclusion

Pipelines are key to delivering constant value to your customers, and it’s worth spending time on getting the pipeline well defined and running smoothly.