Feature Toggling

Mar 19, 2019 · 8 min read
#techniques #engineering #deployments #testing

As mentioned previously, we are documenting a whole bunch of stuff at my current gig. Documenting what tools and languages we should be using for our new products and platforms, but also what techniques we should be using. Techniques is an interesting area to be documenting, because how far do you go?

Taking a step back, I’ve always wanted to be a conference speaker, ever since I saw my boss do a talk at PHPNW back in 2008 / 9. However, two factors have always stopped me:

  1. Everything I do is “so basic”, people will laugh or pick fault, and
  2. I cannot stand in front of that many people and say meaningful words.

Why do I bring this up? Back to techniques.

What may be obvious to you now, was not obvious to you 10 years ago. You need to constantly be thinking about the new person in your team/company. The next generation.

With that in mind, here’s a post about Feature Toggling, something my teams have been using as part of software delivery for years. You may also be doing it, it may seem obvious, or you may not be doing, and you may not even agree 🤷‍♀️

Outline

A feature toggle (also feature switch, feature flag, feature flipper, conditional feature, etc.) is a technique in software development that attempts to provide an alternative to maintaining multiple source-code branches (known as feature branches), such that a feature can be tested even before it is completed and ready for release.

Source

Rationale

So why would we do this, it sounds like extra work? Here are some highlights:

In reality, feature toggling is the addition of some configuration, ideally dynamic, and then writing control statements in your code, to observe the feature toggle. When stated like this, it doesn’t sound impressive, but when you embrace feature toggling, you will have a more manageable platform.

Feature toggling is a really powerful technique

Decoupling deployments from product releases

All too often, customers get features when the action of “releasing the software” is complete. This is fraught with issues. If you have communicated the release to customers, and there is an issue with the deployment, then customers get a bad experience from the company. By utilising feature toggles, you can decouple the action of releasing, and providing new features to customers. The feature can be turned on during normal working hours, when the engineering teams are at full strength. [1]

This allows the philosophy of small and often software releases, which in turn, can reduce the risk of large releases introducing more change than customers and engineers can deal with.

Stability

When you are in control of more aspects of the system, you have more stability. If you decouple releases and feature introduction, you have more control. If you can stop integrations to 3rd parties, or toggle features causing performance issues, you have more control.

Working environment

Ideally we limit the amount of system outages on our platforms. But if a system outage is linked to a new feature, the ability to turn the feature off, reduces the anxiety around the business, whilst a fix is being investigated. If we manage the message around feature introduction, we can reduce this even more.

If you have a system outage due to a new feature, and no way to back out of this, that puts the engineering teams (and others) under pressure. If you can limit this by using feature toggles there is a victory to be had.

Calm working environments produce better results.

Branch management

I think it’s fair to say the industry have more or less agreed that short lived feature branches are the way to go. However there are still places that have long running branches.

Feature toggling allows for continuous deployments (discussed in more detail below), therefore the notion of having long running branches seems contrary to this technique. By limiting the amount of long running branches, we reduce: cost, code rot, and the management overhead.

When to use

Feature toggling is a really powerful technique, and can mean different things to different people.

Engineering toggling:

Feature/Product toggling:

This does not aim to be an exhaustive list, but provide some insight as to how this technique can be used. We will not go into depth in all of these aspects, but to provide a little more context, see below. For further reading and understanding, please follow the links at the bottom of the page.

Continuous deployment

Holding back releases until features are complete, make the releases larger. Large releases have more risk, as you are introducing more change to the platform. If something goes wrong, depending on the quality of your metrics/observability platform, you may struggle to pinpoint the issue since there was a vast amount of change on the platform.

The idea with feature toggles is you code behind the feature toggle. Let’s say your sprints are 2 weeks long, but it will take 3 weeks to implement a feature. You would define a feature toggle (using whatever feature toggling tool you choose) and then code your solution based on the toggle being on or off. In this example, at the end of your sprint you are 66% complete, but you deploy with the feature toggle off. You continue to work on the feature in the next sprint when you complete the solution. You then deploy the sprint with the feature toggle on, as the feature is complete. At this point you have two options: keep the feature toggle, or remove the feature toggle in your codebase.

The decision on what to do with the feature toggle is a discussion to have with your product owner. They may want a grace period whilst you collate metrics to see whether the feature is working, or whether the performance is acceptable, or maybe it was a promotion being run. If you have just deployed a new feature, and there is an issue, you have the ability to toggle the feature off, whilst investigations take place in a calm, and affective manner. This mitigates risk and removes the anxiety around change, because the team is in control of the situation.

This enables a constant release pipeline, and features may or may not meet the deadline. This becomes irrelevant because the date is defined (and does not need to move), as you can deploy with feature toggles.

A/B Testing

Let’s say a product owner (or engineering team) want to make a change to the system, be it UI, UX, Technology change/introduction etc. However, at this moment in time they do not have enough data to make a calculated decision. This is where a feature toggle may help. Let’s say we want to make a UI change, and some users should hit the new UI, and some should hit the old UI. You engineer the new solution, taking into account feature toggles. This time the feature toggle has a probability assigned to it, or certain customers are allocated a feature. This then allows us to test a feature in production.

The information collated from production and real users can then help the product owner decide if a feature is worth building in full, or not.

Integrations

If we are building a product into a 3rd party vendor, we probably want the ability to isolate ourselves and our customers’ experience, from external systems. What if the 3rd party system goes down, or is degraded in functionality. We need to be able to handle that appropriately for our customers, and not blame our experience on a vendor.

Feature toggling can help here. If coupled with some form of queuing system (if appropriate) you could toggle the integration off until the 3rd party vendor is back to 100%.

Anything we need to know?

Sounds too good to be true? Well, it kind of is. However, some advice

Cleanliness

As with everything in life, tidy up after yourselves. If your feature toggle is transient, remember to remove the code no longer required on either side of the toggle, and remove the toggle (configuration) itself.

This is just good practice 😇

Buy in

Engage with the stakeholders of the product and release pipeline so they can understand this technique. It’s beneficial for multiple roles: ops, engineering, product, release, deployment, sales.

Sharing this technique may bring other benefits to other people in your organisation. Sometimes people do not know what is achievable, so share.

Thanks to

I was lucky enough to write the vast majority of this post at work, and with a great colleague: Conrad

And luckily, it turns out, I’m OK to post this here.

Footnotes

See also

Content

Social

Recent