
Fuses: a simple pattern for preventing runaway automation
I’ve built and maintained countless “janitor” systems in my career: cronjobs which go and clean up old resources like user accounts which should no longer be active. On multiple occasions, one of those janitors has gone off the rails and locked the entire company out of their accounts.
Now, whenever I build systems like this, I include a “fuse”.
The fuse defines a maximum value I expect the system to handle at any one time. The number of accounts to delete, or number of permissions to revoke. If the janitor ever tries to exceed this value, the fuse “blows” and requires a human to intervene.





