Comment by brabel 2 days ago
Mandatory comment about ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility. Yes, it's a bit ugly but if you stick to the DER encoding it's really not worse than Protbuf at all. Check out the Wikipedia example:
https://en.wikipedia.org/wiki/ASN.1#Example_encoded_in_DER
Protobuf is ok but if you actually look at how the serializers work, it's just too complex for what it achieves.
> someone will throw TeletexString or BMPString
ASCII with escapes and UCS-2.
> horribly bloated and over-engineered.
It's no more or less complicated than XML, JSON or CSV. Which is why you can use ASN.1 to serialize to and from all these formats. ASN.1 provides you an additional level of schema above these. It simply allows you to describe your problem.
I find ASN.1 far more sane and useful than something like JSON Schema which is just as "bloated and over-engineered." It turns out describing data is not a simple problem.
Nope, TeletexString is ITU T.61, a.k.a codepage 1036. So Backspace (0x08) is OK, but Tab (0x09) is not.
What, your implementation does not include CP1036 to Unicode translation table? Sorry, it's no longer ASN.1, it's now ASN.themafia.
Oh, it does? Then how about xmlhstring vs hstring, are you handing difference properly?
What about REAL type? Does your asn.1 library include support for arbitrary-precision floating point numbers, both base 2 and 10? And no, you cannot use "double", I am sure there is an application out there which uses base 10 reals, or has 128 bits of mantissa.
ASN.1 is full of overengineered, ancient things like those, and the worst part - once you actually start using it to interoperate with other software, there is a good chance you'll see them. If you want something that people actually implement fully, choose something else.
> Nope, TeletexString is ITU T.61
Yes, read the standard, it's ASCII with special escape sequences. Which I don't have to render, I only have to convey them correctly across the network.
> What, your implementation does not include CP1036 to Unicode translation table? Sorry, it's no longer ASN.1, it's now ASN.themafia.
Why would I need the table?
> Oh, it does? Then how about xmlhstring vs hstring, are you handing difference properly?
What exactly needs to be "handled?"
> Does your asn.1 library include support for arbitrary-precision floating point numbers
Yes, because there are third party libraries which supply this functionality, so it's hardly any special effort to implement.
> ancient things like those
So no one ever needs arbitrary precision integers? You'll eventually need them for some application. Now all you have is ad-hoc implementations and almost no software to interoperate with them or verify them.
> If you want something that people actually implement fully, choose something else.
Name anything else with the same features yet is easier to implement "fully." Seriously, go read the JSON Schema specification. This is a _hard_ problem. If you think you've found an easy solution it's likely you've just left most of the functionality on the floor. And now we have to ask "is software X compatible with software Y?" Obviating the entire point of a standard.
I also think ASN.1 DER is better (there are other formats, but in my opinion, DER is the only good one, because BER is too messy). I use it in some of my stuff, and when I can, my new designs also use ASN.1 DER rather than using JSON and Protobuf etc. (Some types are missing from standard ASN.1 but I made up a variant called "ASN.1X" which adds some additional types such as key/value list and some others. With the key/value list type added, it is now a superset of the data model of JSON, so you can convert JSON to ASN.1X DER.)
(I wrote a implementation of DER encoding/decoding in C, which is public domain and FOSS.)
> Protobuf is ok but if you actually look at how the serializers work, it's just too complex for what it achieves.
Yeah. I do remember a lot of workloads at Google where most of the CPU time was spent serializing/deserializing protos.
ASN.1 is way overengineered to the point of making it hard to support. You don't need inheritance for example.
it is not necessary to use or to implement all of the data types and other features of ASN.1; you can implement only the features that you are using. Since DER uses the same framing for all data types, it is possible to skip past any fields that you do not care about (although in some cases you will still need to check its type, to determine whether or not an optional field is present; fortunately the type can be checked easily, even if it is not a type you implement).
Yes but I don't want to worry about what parts of the spec are implemented on each end. If you removed all the unnecessary stuff and formed a new standard, it'd basically be protobuf.
I do not agree. Which parts are necessary depends on the application; there is not one good way to do for everyone (and Protobuf is too limited). You will need to implement the parts specific to your schema/application on each end, and if the format does not have the data types that you want then you must add them in a more messy way (especially when using JSON).
Thank you! Now I don't have to be the one saying this. Props if you use OER over DER. But since OP needs available tooling they might as well go to flatbuffers, which is much better than PB much like OER is much better than DER.
I... wouldn't use it for a greenfield project either unless I got good at porting Luke Howard's Swift ASN.1 stack to whatever language I might be using that isn't C. For C I'd just use Heimdal's awesome ASN.1 compiler and be done. Even then I would be tempted to use flatbuffers instead, or else I'd have to go implement OER (a bunch of work I don't really care to do).
The problem with ASN.1 -- the only real problem with ASN.1, is lack of excellent tooling.
> ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility.
After working heavily with SNMP across a wide variety of OEMs, this flexibility becomes a downside. Or SNMP/MIBs were specified at the wrong abstraction level, where the ASN.1 flexibility gives mfgs too much power to do insane and unconventional things.
Yeah same, ASN.1 was a nightmare when I was dealing with LTE
I've been working on and with Kerberos and PKIX for decades. I don't find ASN.1 to be a problem as long as you have good tooling or are willing to build it. The specs are a pleasure to read -- clear, concise, precise, and approachable (once you have a mental model for it anyways).
Of course, I am an ASN.1 compiler maintainer, but hey, I had to become one because the compiler I was using was awesome but not good enough, so I made it good enough.
I'm curious what made it a nightmare for you.
The compiler I [occasionally] work on does not abuse comments for directives. All directives in that compiler are out of band.
ASN.1 has too much stuff. The moment you write "I made ASN.1 decoder/encoder", someone will throw TeletexString or BMPString at it. Or inheritance, as morshu9001 sad. So at this point:
- You can support all those features, and your ASM.1 library will be horribly bloated and over-engineered.
- You can support your favorite subset, but then you cannot say it's ASN.1 anymore. It will be "ASN.brabel", which only has one implementation (yours). And who wants that?
(unless you are Google and have immense developer influence... But in this case, why not design things from scratch, since we are making all-new protocol anyway?)