Comment by Random_BSD_Geek 2 days ago
Polar opposite of my experience. To achieve the technical equivalent of changing a lightbulb, spend the entire day wrangling a dozen tools which are broken in different ways, maintained by teams that no longer exist or have completely rolled over, only to arrive at the finish line and discover we don't use those lightbulbs anymore. Move things and break fast.
Yeah 100%. I found it immensely frustrating to be using tools with no community (except internally), so-so documentation, and features that were clearly broken in a way that would be unacceptable for a regular consumer product. If you have a question or error not covered by an internal search or documentation, good luck, you'll need it. Literally part of the reason I left the company.
Well, you're supposed to read the code and figure it out. And if you can't, you're not good enough an engineer. According to people at Meta.
No matter what, tools will be broken. Having access to the source and being able to land a diff to fix the issue is awesome imo.
Same as Google. Many internal tools have painful interfaces and poor or documentation because the hiring bar was high and it was acceptable to assume that the user's skill level is high enough to figure it out. That attitude becomes a bigger problem when trying to sell tools to the public (e.g. Google Cloud Platform).
As an outsider, I was always under the impression that Google had a tradition of engineering excellence (robust tools, clean and while tested code following strict guidelines), while Meta has more of a Hacker culture (move fast and break things).
Google also has traditions that created Broccoli Man: https://www.youtube.com/watch?v=3t6L-FlfeaI
Agreed. I often get my work done using open source build instructions and tools and then when everything works I port it to internal infra. Other people are the opposite though, which for open source based code bases has a nasty side effect of the work having no upstream able tests!
But you're both talking about different things. The tools are both often left in disuse, lacking documentation, etc. But they also have a really tight integration with each other that allows for unparalleled visibility and ability over enormous systems with many moving parts.
It's been awhile, but I recall fighting with the massive checkout sizes to do anything of consequence with the internal tooling causing the vms to run out of disk space and corrupt my work. I got very used to rsyncing to my laptop every few minutes and rebuilding the vm multiple times per day. Totally frustrating and pointless waste of time.
Large checkouts is a solved problem now https://github.com/facebook/sapling/blob/main/eden/fs/docs/O...
IMO there's a mix of a few really good, widely used, well-supported tools as well as a long tail of random tiny tools where the original team is gone that are cruftier.