Comment by morshu9001 a day ago
I'm still confused how these ISO 2022 strings even work, and the ASN1 docs discourage using the UniversalString and GraphicString types. All these different string types are intimidating if I just want unicode/ascii, and even if I were using an obscure encoding, I'd use generic bytes instead of wanting asn1 to care about it.
Oh gosh. Fair enough that this exists and something uses it, but I'd absolutely want to handle that on the ends only, not get asn1 involved in parsing it.
Oh GeneralString is madness. It's pre-Unicode madness. It exists because Unicode didn't exist in 1984, but people still wanted to be able to exchange text in multiple scripts, which necessitated being able to "switch codesets" in the middle. It's... yeah, it's.. it's nuts. I've _not_ implemented GeneralString, and practically no one needs to even when specs say to. E.g., in Kerberos the strings are GeneralString, but all the implementations just-send-8 and do not attempt to interpret any codeset switching escapes.
> I'm still confused how these ISO 2022 strings even work
There is C0, G0, C1, and G1 sets (C0 and C1 are control characters and G0 and G1 are graphic characters), and escape sequences are used to select the C or G set for bytes with or without the high bit set. Graphic string does not allow control characters and General string does allow control characters.
You probably do not need all control characters; your schema should probably restrict which control characters are allowed in each context (although the ASN.1 schema format does not seem to have any way to do this). This way, you will only handle the control characters which are appropriate for your use.
This is messy, although canonical form simplifies it by adding some restrictions (this is one of the reasons why DER is better than BER, in my opinion). TRON code is better and is much simpler than the working of ISO 2022. (Unicode has a different kind of mess; although decoding is simpler, actually handling the decoded characters in text is its own big mess for many reasons. Unicode is a stateful character set, even though the encoding is stateless; TRON code is the other way around (and with a significantly simpler stateful encoding than ISO 2022).)
> the ASN1 docs discourage using the UniversalString and GraphicString types
UniversalString is UTF-32BE and GraphicString is ISO 2022 without control characters. By knowing what they are, you should know in which circumstances they should be considered useful or not useful; I think that they should not be discouraged in general (although usually if you want Unicode, you would use UTF-8 rather than UTF-32, there are some circumstances where you might want to use UTF-32, such as if the data or program is already UTF-32 for other reasons).
(The data type which probably should be avoided is the UTC time type, which is not Y2K compliant.)
> All these different string types are intimidating if I just want unicode/ascii
If you only want ASCII, use the IA5 type (or Visible if you do not want control characters); if you only want Unicode, use the UTF-8 string type (or Universal if you want UTF-32 instead for some reason). ("IA5" is another name for ASCII that as far as I can tell hardly anyone other than ITU uses.)
However, Unicode is not a very good character set, and they should not force or expect you to use it.
As I had mentioned before, you do not need to use or implement all of the ASN.1 data types; only use the ones appropriate for your application (so, if you do not like most of the types, then don't use those types). I also made up some additional nonstandard ASN.1 types (called ASN.1X), which also might be useful for some applications; you are not required to use or implement these either.
> However, Unicode is not a very good character set, and they should not force or expect you to use it.
Unicode is an excellent character set, and for 99% of cases (much more probably) it's absolutely the best choice. So one should choose Unicode (and UTF-8) in all cases unless there is an excellent reason to do otherwise. As time passes there will be fewer and fewer cases where Unicode is not sufficient, so really we are asymptotically approaching the point at which Unicode is the only good choice to make.
This is all independent of ASN.1. But it is true that ASN.1 has built-in types for Unicode and non-Unicode strings that many other protocols lack.
Have you written up anything about ASN.1X anywhere? I'd love to take a look.
> Have you written up anything about ASN.1X anywhere? I'd love to take a look.
ASN1_BCD_STRING (64): Represents a string with the following characters: "0123456789*#+-. " (excluding the quotation marks). Each octet encodes two characters, where the high nybble corresponds to the first character and the low nybble corresponds to the second character.
ASN1_PC_STRING (65): Represents a string of characters in the PC character set. Note that the control characters can also be used as graphic characters.
ASN1_TRON_STRING (66): Represents a string of characters in the TRON character set, encoded as TRON-8.
ASN1_KEY_VALUE_LIST (67): Represents a set of keys (with no duplicate keys) and with a value associated with each key. The encoding is the same as for a SET of the keys, but with the corresponding value immediately after each key (when they are sorted, only the keys are sorted and the values are kept with the corresponding keys).
ASN1_UTC_TIMESTAMP (68): Represents a number of UTC seconds (and optionally fractions of seconds), excluding leap seconds, relative to epoch.
ASN1_SI_TIMESTAMP (69): Represents a number of SI seconds (and optionally fractions of seconds), including leap seconds, relative to epoch.
ASN1_UTC_TIME_INTERVAL (70): Represents a time interval as a number of UTC seconds. The number of seconds does not include leap seconds.
ASN1_SI_TIME_INTERVAL (71): Represents a time interval as a number of SI seconds (which may include fractions).
ASN1_OUT_OF_BAND (72): This type is not for use for general-purpose data. It represents something which is transmitted out of band (e.g. a file descriptor) with whatever transport mechanism is being used. The transport mechanism defines how a value of this type is supposed to be encoded with whatever ASN.1 encoding is being used.
ASN1_MORSE_STRING (73): Represents a string of characters in the Morse character set. The encoding is like a relative object identifier, where 0 means an empty space, and others is like bijective base 2 with 1 for dots and 2 for dashes, with the high bit for the first dot/dash, e.g. 4 means A and 8 means U.
ASN1_REFERENCE (74): A reference to another node within the same file. (Not all implementations will support this feature.) The encoding is like a Relative Object Identifier; the first number is how many times to go to the parent node (where 0 means the reference itself), and then the rest of the numbers specify which child node of the current node to go to where 0 means the first child, 1 means the second child, etc. It can reference a primitive or constructed node of a BER file, but you cannot specify a child index for a child of a primitive node, since primitive nodes cannot have child nodes. At least one number (how many levels of parents) is required, but any number of numbers is potentially possible.
ASN1_IDENTIFIED_DATA (75): Data which has a format and/or meaning which is identified within the data. The encoding is always constructed and consists of two or three items. The first item is a set of object identifiers, object descriptors (used only for display), and/or sequences where the first item of the sequence is a object identifier. The receiver ignores any items in this set that it does not understand. The second item in a ASN1_IDENTIFIED_DATA can be any single item of any type; it is interpreted according to the object identifiers in the first set that the receiver understands. The third item is optional, and if it is present it is a key/value list of extensions; the keys are object identifiers and the values are of any type according to the object identifiers. The default value of this key/value list is an empty key/value list.
ASN1_RATIONAL (76): Stored as constructed, containing two integers, being the numerator and the denominator. The denominator must be greater than zero. If it is canonical form, then it must be lowest terms.
ASN1_TRANSLATION_LIST (77): A key/value list where the keys identify languages. If the key is null then it means the default in case no language present in this list is applicable. The types of the values depends on the application (usually they will be some kind of character strings).
In addition, the same number for the BMP string type can also be used for a UTF-16 string type, and there is a "OBJECT IDENTIFIER RELATIVE TO" type which encodes a OID as either relative or absolute (in canonical form, it is always relative when possible) in order to save space; the schema will specify what it is relative to. ANY and ANY DEFINED BY are allowed despite being removed from the most recent versions of standard ASN.1. (The schema format for these extensions is not defined, since I am not using the ASN.1 schema format; however, someone who does use it might do so if they need it.)
There is also SDER, which is a superset of DER but a subset of BER, in case you do not want the mess of BER but do not want to require strictly canonical form either; and also SDSER which uses the same encoding for types and values than SDER but but length works differently in order to support streaming better.
As is usual, you do not have to use any or all of these types, but someone might find them useful for some uses. I have used some of them in my own stuff.
ASN1_BCD_STRING can be just IA5String with a constraint attached...
Your time types can be just an INTEGER with a constraint attached... (In Heimdal we use INTEGER constraints to pick a representation in the programming language.) E.g.,
-- 64-bit signed count of seconds where 0 is the Unix epoch
ASN1_UTC_TIMESTAMP ::= INTEGER (-18446744073709551616..18446744073709551615)
Out-of-Band ::= [APPLICATION 100] NULL
ASN1_REFERENCE is... interesting. I've not needed it, but some RPC protocols support intra-payload and even circular references, so if you have a need for that (hopefully you don't), then your ASN1_REFERENCE would be useful indeed.
ASN1_IDENTIFIED_DATA... ASN.1 has EMBEDDED-PDV, open types, and the TYPE-IDENTIFIER class -- there are many ways to do this in ASN.1. See https://github.com/heimdal/heimdal/blob/master/lib/asn1/READ...
ASN1_RATIONAL is just a tagged sequence of numerator and denominator, with a constraint that the denominator must not be zero.
OBJECT IDENTIFIER RELATIVE TO is just a CHOICE of OBJECT IDENTIFIER and RELATIVE IDENTIFIER.
Re: SDER... yeah, so Heimdal's codec produces DER but accepts a subset of BER for interop with OpenSSL and others. If you really want streaming then you'll want a variant of OER with fixed-length lengths (which IMO OER should have had, dammit), which then looks a lot like XDR but with different alignment and more types.
I had kind of expected a subset of x.680.
Re: ASN1_KEY_VALUE_LIST, why not just do something like:
ValueList ::= SET OF KeyValue
KeyValue ::= SEQUENCE { key MyKeyType, value MyValueType }
MyKeyType ::= ... -- whatever you want here
MyValueType ::= ... -- ditto
?What are the numbers in parenthesis? UNIVERSAL tag values?
GeneralString relies on control characters to "load" character sets into the C0 and C1 registers. This is madness -- specifically it's pre-Unicode madness, but before Unicode it made sense.