Comment by JohnMakin
I have fought this battle for my entire career. The problem is, incentives are tough - often senior engineers on ops teams have built the system on duct tape and bubble gum, and have a very specific and unobtainable knowledge about all the quirks of it which frequently involve a ton of manual, esoteric shit to resolve without a lot of know-how.
Part of how I’ve made myself useful as a consultant sometimes in my career is by insisting on a strict set of processes that break down and identify bottlenecks like this. It’s almost always ops, it almost always can be automated, but there’s “no bandwidth or budget” to fix the problems so they expect superheroes to constantly save the day by pulling absurd hours or by inventing genius solutions that management buys into.
For me, as an IC or sometimes as a consultant, I try to chip away. We spend X minutes a week doing Y thing manually. what’s the easiest thing we can do to reduce X? and then slowly introduce whatever arises out of those processes, and then slowly “eat the elephant” one bite at a time.
when people come into a big messy manual legacy system they often, imho, make the mistake of wanting to tear it down and recreate it from scratch. having done several cloud architecture/IAC refactors in my career, I can tell you its definitely a way easier sell to management and other teams to whittle away at it than to recreate it entirely, and i think that’s also what the video is poking fun at. There’s always skeletons that your perfect solution will run into. Processes, standards, and monitoring will expose these dead bodies, but it’s not a one shot solution and I’m immediately skeptical when anyone proposes something in this space that is. Every situation is a snowflake.
> Every situation is a snowflake.
Yeah, I mean, I think this describes it. Ultimately the video had many thoughts and layers, starting with a charismatic lead who is proud or pretending to be proud about completely meaningless, arbitrary achievements and metrics trying to motivate his team for rejecting access requests. Then it goes on to this individual who just can't handle it. Which is relatable. I don't think I could be able to perform a job like this at all. So I think this individual there was just so badly misplaced for handling this type of work - he really wanted out from there, but unfortunately it is not always so simple. In an ideal world he would've worked in another role.