Comment by dfabulich

Comment by dfabulich 2 days ago

24 replies

View on Hacker News

> With JSON, you often send ambiguous or non-guaranteed data. You may encounter a missing field, an incorrect type, a typo in a key, or simply an undocumented structure. With Protobuf, that’s impossible. Everything starts with a .proto file that defines the structure of messages precisely.

This deeply misunderstands the philosophy of Protobuf. proto3 doesn't even support required fields. https://protobuf.dev/best-practices/dos-donts/

> Never add a required field, instead add `// required` to document the API contract. Required fields are considered harmful by so many they were removed from proto3 completely.

Protobuf clients need to be written defensively, just like JSON API clients.

HelloNurse a day ago

The blog seems to contain other similar misunderstandings: for example the parallel article against using SVG images doesn't consider scaling the images freely a benefit of vector formats.

Reply View 8 replies
  • chrismorgan a day ago

    https://aloisdeniel.com/blog/i-changed-my-mind-about-vector-... seems fairly clearly to be talking about icons of known sizes, in which case that advantage disappears. (I still feel the article is misguided and that the benefit of runtime-determined scaling should have been mentioned, and see no benchmarks supporting its performance theses, and I’d be surprised if the difference was anything but negligible; vector graphic pipelines are getting increasingly good, and the best ones do not work in the way described, and could in fact be more efficient than raster images at least for simpler icons like those shown.)

    Reply View 7 replies
    • HelloNurse a day ago

      > seems fairly clearly to be talking about icons of known sizes, in which case that advantage disappears.

      That's the point: obliviousness to different concerns and their importance.

      Among mature people, the main reason to use SVG is scaling vector graphics (in different contexts, including resolution-elastic final rendering, automatically exporting bitmap images from easy to maintain vector sources, altering the images programmatically like in many icon collections); worrying about file sizes and rendering speed is a luxury for situations that allow switching to bitmap images without serious cost or friction.

      Reply View 0 replies
    • eviks a day ago

      Are there display pipelines that cache the generated-for-my-device-resolution svgs instead of doing all the slower parsing etc from scratch every time, achieving benefits of both worlds? And you can still have runtime-defined scaling by "just" rebuilding the cache?

      Reply View 2 replies
      • HelloNurse 6 hours ago

        The best place for such a cache is a GPU texture, and in a shader that does simple texture mapping instead of rasterizing shapes it would cost more memory reads in exchange for less calculations.

        Reply View 0 replies
      • chrismorgan 21 hours ago

        Increasingly I think you’ll find that the efficient format for simple icons like this actually isn’t raster, due to (simplifying aggressively) hardware acceleration. We definitely haven’t reached that stage in wide deployment yet, but multiple C++ and Rust projects exist where I strongly suspect it’s already the case, at least on some hardware.

        Reply View 0 replies
    • cosmotic a day ago

      Icons are no longer fixed sizes. They're are numerous dpi/scaling settings even if the "size" doesn't change.

      Reply View 2 replies
      • chrismorgan 21 hours ago

        The article goes into that, it’s making a sprite map of at least the expected scaling factors.

        Reply View 1 reply
        • rerdavies 9 hours ago

          There are no "expected" scaling factors anymore.

          Reply View 0 replies
cookiengineer a day ago

Isn't the core issue just language and implementation differences of clients vs servers here?

I went all in with Go's Marshalling concept, and am using my Gooey framework on the client side nowadays. If you can come around Go's language limitations, it's pretty nice to use and _very_ typesafe. Just make sure to json:"-" the private fields so they can't be injected.

[1] shameless drop https://github.com/cookiengineer/gooey

Reply View 0 replies
klysm 2 days ago

It’s also conflating the serialization format with contracts

Reply View 7 replies
  • ablob a day ago

    I feel like that's fine since both things go hand in hand anyway. And if choosing the JSON-format comes with a rather high amount of contract-breaches it might just be easier to switch that instead of fixing the contract.

    Reply View 3 replies
    • peterkelly a day ago

      Unless a violation of that contract can lead to a crash or security vulnerability...

      Reply View 2 replies
      • ablob a day ago

        The post is about changing the serialization-format so enforcing contracts becomes esier; and I am defending the post, so I don't understand what you're hinting at here.

        Reply View 0 replies
  • shortrounddev2 a day ago

    Most web frameworks do both at the same time to the point where having to write code which enforced a type contract after deserializing is a delabreaker for me. I eant to be able to define my DTOs in one place, once, and have it both deserialize and enforce types/format. Anything else is code smell

    Reply View 2 replies
    • Seattle3503 a day ago

      I'm in the same boat. I mostly write Rust and Python. Using serde_json and Pydantic, you get deserialization and validation at the same time. It allows you to de-serialize really "tight" types.

      Most of my APIs are internal APIs that accept breaking changes easily. My experience with protobufs is that it was created to solve problems in large systems with many teams and APIs, where backwards compatibility is important. There are certainly systems where you can't "just" push through a breaking API change, and in those cases protobufs make sense.

      Reply View 1 reply
      • masklinn a day ago

        > My experience with protobufs is that it was created to solve problems in large systems with many teams and APIs

        Also significant distribution such that it’s impossible to ensure every system is updated in lockstep (at least not without significant downtime), and high tail latencies e.g. a message could be stashed into a queue or database and processed hours or days later.

        Reply View 0 replies
bloppe a day ago

Skew is an inherent problem of networked systems no matter what the encoding is. But, once the decoding is done, assuming there were no decoding errors in either case, at least with protobuf you have a statically typed object.

You could also just validate the JSON payload, but most people don't bother. And then they just pass the JSON blob around to all sorts of functions, adding, modifying, and removing fields until nobody knows for sure what's in it anymore.

Reply View 2 replies
  • happymellon a day ago

    > You could also just validate the JSON payload, but most people don't bother.

    I don't think I have ever worked somewhere that didn't require people to validate inputs.

    The only scenario could be prototypes that made it to production, and even when its thrown over the wall I'll make it clear that it is unsupported until it meets minimum requirements. Who does it is less important than it happening.

    Reply View 0 replies
  • Ghoelian a day ago

    The convention at every company I've worked at was to use DTO's. So yes, JSON payloads are in fact validated, usually with proper type validation as well (though unfortunately that part is technically optional since we work in php).

    Usually it's not super strict, as in it won't fail if a new field suddenly appears (but will if one that's specified disappears), but that's a configuration thing we explicitly decided to set this way.

    Reply View 0 replies
gipp a day ago

I think the OP meant something far simpler (and perhaps less interesting), which is that you simply cannot encounter key errors due to missing fields, since all fields are always initialized with a default value when deserializing. That's distinct from what a "required" field is in protobuf

Reply View 1 reply
  • orphea a day ago

    Depending on the language/library, you can get exactly the same behavior with JSON.

    Reply View 0 replies
Pet_Ant a day ago

> Protobuf clients need to be written defensively, just like JSON API clients.

Oof. I'd rather just version the endpoints and have required fields. Defensive is error-prone, and verbose, harder to reason about, and still not guaranteed. It really feels like an anti-pattern.

Reply View 0 replies