You’ve heard of feature flags, but what about operational flags? ⏯️
🎞️ Feature Flag Recap:
Feature flags are used to toggle features on/off while being developed and deployed.
They are usually temporary; once a feature is stable, the flag is removed from the code.
🧠 Operational Flags:
Operational flags are long-lived flags that turn system behavior on/off at runtime.
Not for releases, but for resiliency and manageability.
Well-placed Operational Flags can:
-
⛔ Disable an API call when a downstream service is in a brownout (e.g., reboot loop).
-
📨 Disable event publishing when consumers are backlogged.
-
🔌 Turn off non-essential dependencies during unexpected traffic spikes.
👨🏫 Real World Example:
I once had a service that published an event for every incoming request.
At one point, the downstream consumer started experiencing performance issues with their database.
This load was exaggerated by the incoming events being written to the database.
We couldn't just stop the downstream service because of the other functions it provided.
Luckily, we built an operational flag that could turn off the publishing of events.
We turned publishing off, giving the downstream consumer more time to address their performance issues.
We didn't build this flag with the consumer in mind; it was built in case we needed it.
⚠️ Not Plan A
Operational flags are not plan A; they are emergency switches that are useful when needed.
I try to build operational flags for any dependency or non-essential functionality in mission-critical systems. Just in case…