Comment by andrewmcwatters a day ago
I know this comment isn't terribly helpful, so I'm sorry, but it also sounds like Go is entirely the wrong language for this use case and you and your team were forced to use it for some corporate reason, like, the company only uses a subset of widely used programming languages in production.
I've heard the term "beaten path" used for these languages, or languages that an organization chooses to use and forbids the use of others.
> The original developers chose Go because they thought it was a good fit for our use case.
I don't completely get this. If you are memory requirements are strict, this makes little to no sense to me. I was programming J2ME games 20 years ago for Nokia devices. We were trying to fit games into 50-128kb RAM and all of this with Java of all the languages. No sane Java developer would have looked at that code without fainting - no dynamic allocations, everything was static, byte and char were the most common data types used. Images in the games were shaved, no headers, nothing. You really have to think it through if you got memory constraints on your target device.
The strictness of our memory requirements wasn’t made apparent until years later. For background, the application is a system daemon running on end-user servers. So every byte we allocate, every cycle of CPU we use, is a byte or cycle taken away from our customers. We don’t provide firm guarantees or formal SLAs on our memory usage, but we do try to minimize it whenever possible. Because what we don’t want is for someone to upgrade agent versions and suddenly our daemon is using significantly more memory and causing OOMs, or using more CPU and force customers to scale out their fleet. Our p99 cpu and memory usage right now are more or less the same as they were two years ago (RSS is under 40MB last I checked)
So it’s not like we’re running on a machine with only kilobytes of RAM, but we do want to minimize our usage.
Wow. I bet that required some decent clever thinking that you would have only typically thought about in C or C++.
No, Go isn’t actually that widely used at my company. The original developers chose Go because they thought it was a good fit for our use case. We were particularly looking for a compiled language that produces binaries with minimal dependencies, didn’t have manual memory management, and was relatively mature (I think Rust was barely 1.0 at the time). We knew we wanted to limit memory usage, but it was more of a “nice to have” than anything else. And Go worked pretty well. It was in production for a couple years before we started getting burnt by these issues. We are looking at porting this to Rust, but that’s a big lift. This is a 50K+ line code base that’s pretty battle tested.