Comment by packetlost

Comment by packetlost 10 months ago

8 replies

> One can dream of a future where perhaps it will be possible to write a function, library, or program in any language one chooses and call it from any other shell or language without swig or ceremony

I mean, there's a reason text was selected, it's the lowest common denominator. What you're asking for is universal FFI, which we already have that for the most part: C. The problem ultimately turns into runtime and resource management issues when crossing language boundaries, that is, unless you use OS primatives (pipes, sockets, shared memory, processes, etc.) and use message passing/pipelines exclusively. Then we're back to plaintext.

There's a ton of non-printable characters in ASCII that are useful as delimiters for non-keyboard interactive programs that are largely vestigial. One could at least consider reusing them for delimiters and special control sequences for message passing, but without some sort of standardization it's limited in practical use.

Brian_K_White 10 months ago

The problem with those ascii values is editors don't display them and there are no keyboard keys to type them.

The reason csv exists is because you can actually see and type all parts of it, on anything, any platform, any hardware, any software, any age, even a mechanical typewriter, even a pencil. That is not merely nice, it's essntially priceless, infinitely, incalculably valuable. The utility outweighs the problems as big as they absolutely are. Actually the same is true for json, yaml, xml...

  • packetlost 10 months ago

    > The problem with those ascii values is editors don't display them and there are no keyboard keys to type them.

    Did you read my post? Like, every word of it? Being able to type a control sequence is not particularly useful for ephemeral data in a message-passing between programs context. It's absolutely important for persistent, editable data. It seems like more of a feature to me to use these special characters for special contexts and not have escaping typeable characters be load-bearing.

    Further, my editor seems to print control characters just fine, though entering them would be a bit of a pain and the behavior is likely configurable.

WorldMaker 10 months ago

Instead of inventing some new delimited formats, we also already have nearly ubiquitous interchange formats like JSON. If you wanted to reduce the in memory footprint versus plaintext encoding the JSON there are good options like BSON and CBOR.

  • packetlost 10 months ago

    JSON isn't streamable. Neither is CBOR for that matter, though it's not quite as bad.

    • superb_dev 10 months ago

      Individual JSON records are streamable

      • packetlost 10 months ago

        JSON itself, no, that's not the case. If you accept an extension like JSON Lines, it can be. Many decoders will barf if you have multiple top level objects in the same file. If you want portable, non-ephemeral data JSON Lines is a poor choice.

superb_dev 10 months ago

Nitpick but I wouldn’t call C a universal FFI, it’s merely the de facto FFI. C can’t meaningfully represent any complex language semantics (not that it should or even could)

  • packetlost 10 months ago

    It's a near universal de facto FFI if we're being pedantic. My whole point was the reason that's the case is it has essentially no runtime expectations at all, which is a requirement for a truly universal FFI.