> even when doing your day to day data analysis job
This is a bad advice.
Usually coming from an over-engineering mentality (that many engineers suffer from, including myself).
Part of the engineering culture is to find the _right_ solution for the job. Not the most _engineered_ one.
There is no point to shoot at sparrows from a canon.
When your job is to cobble up disposable scripts - there is no place for “practices”, unit test for example. Thus there is nothing for a software engineer to pay special attention to. Therefore there is no value in suggesting that
While some may take that as “turn up your engineering knob” that leads to over engineering
While I am fairly good at overengineering and optimizing, I have always hated doing it. For the longest time I wasn't able to do it but then I learned about low level hacking, and throwing things into assembly. Then I had a friend enter the HFT field and he taught me a thing or two about optimization.
godbolt.org is fun to check out :)
But yea, I really like making quick scrappy prototypes that deliver business results quickly. I've always had that mindset rather than crafting good quality code. In software engineering teams I threw that mindset to the wayside because they'd get mad. Sometimes rightly so as tests have saved my life, but quite often I think SWEs focus too much on code and being technical rather than delivering impact. I think in part that's also because it seems that in most places I've worked at product owners are "supposed to" think about that. The dev is only supposed to think about the technical implications, maybe the UX implications (depending on the company) but the moment a dev thinks about business implications, I've gotten quite a few times to "stay in my lane".
In most cases the care for code quality isn't needed, sometimes it is. I'm taking a page from Jonathan Blow here (the game dev that created The Witness and Braid). He'd develop everything unoptimized and only later when he'd run into performance issues would he optimize it.
How is this bad advice?
> keep up your software engineering best practices
It's not a best practice to over-engineer solutions, and indeed, it is a best practice to find the right solution.