Comment by written-beyond 2 days ago
Idk I built a production system and ensured all data transfers, client to server and server to client were proto buf and it was a pain.
Technically, it sounds really good but the actual act of managing it is hell. That or I need a lot of practice to use them, at that point shouldn't I just use JSON and get on with my life.
There are plenty of it-doesn't-just-work things about JSON though. Sending binary data or 64-bit integers is a huge pain. Or maps with non-string keys, or ordered maps. Plus JSON doesn't scale well with message size because it doesn't use TLV so parsing any of a message requires parsing all of it.
It's not some perfect format.
That said, I'm disappointed with Protobuf too. Especially the handling of optional/default fields. I believe they did eventually add an `optional` tag so you can at least distinguish missing vs default field values.
The lack of required fields makes it very annoying to work with though. And no, there's no issue with required fields in general. The only reason it doesn't have them is because the implementation in Protobuf 2 caused issues and Google had to have a smooth transition from that.
If you're starting a greenfield project it seems silly to opt in to Google's tech debt.
If you're thinking "but schema evolution!!" well yeah all you need is a way to have versioned structs and then you can mark fields as being required for ranges of versions. So you can still remove required fields without breaking backwards compatibility.
Check out Lite³, a schemaless binary format: https://github.com/fastserial/lite3
Supports 64 bit ints, raw byte datatype, zero-copy parsing, does not require schema and can be converted to JSON for readability while retaining all field names.
What issues did you have? In my experience, most things that could be called painful with protobuf would be bigger pains with things like JSON.
Making changes to messages in a backwards-compatible way can be annoying, but JSON allowing you to shoot yourself in the foot will take more time and effort to fix when it's corrupting data in prod than protobuf giving you a compile error would.
Well at the bare minimum setting up proto files and knowing where they live across many projects.
If they live in their own project, making a single project be buildable with a git clone gets progressively more complex.
You now need sub modules to pull in your protobuf definitions.
You now also need the protobuf tool chain to be available in your environment you just cloned to. If that environment has the wrong version the build fails, it starts to get frustrating pretty fast.
Compare that to json, yes I don't get versioning and a bunch of other fancy features but... I get to finish my work, build and test pretty quickly.
I think the it-just-works nature and human readability for debugging JSON cannot be overstated. Most people and projects are content to just use JSON even if protos offer some advantages, if not only to save time and resources.
Whether the team saves times in the longer when using protos is a question in its own.