Comment by packetlost

Comment by packetlost 2 days ago

8 replies

> One can dream of a future where perhaps it will be possible to write a function, library, or program in any language one chooses and call it from any other shell or language without swig or ceremony

I mean, there's a reason text was selected, it's the lowest common denominator. What you're asking for is universal FFI, which we already have that for the most part: C. The problem ultimately turns into runtime and resource management issues when crossing language boundaries, that is, unless you use OS primatives (pipes, sockets, shared memory, processes, etc.) and use message passing/pipelines exclusively. Then we're back to plaintext.

There's a ton of non-printable characters in ASCII that are useful as delimiters for non-keyboard interactive programs that are largely vestigial. One could at least consider reusing them for delimiters and special control sequences for message passing, but without some sort of standardization it's limited in practical use.

Brian_K_White 2 days ago

The problem with those ascii values is editors don't display them and there are no keyboard keys to type them.

The reason csv exists is because you can actually see and type all parts of it, on anything, any platform, any hardware, any software, any age, even a mechanical typewriter, even a pencil. That is not merely nice, it's essntially priceless, infinitely, incalculably valuable. The utility outweighs the problems as big as they absolutely are. Actually the same is true for json, yaml, xml...

  • packetlost 2 days ago

    > The problem with those ascii values is editors don't display them and there are no keyboard keys to type them.

    Did you read my post? Like, every word of it? Being able to type a control sequence is not particularly useful for ephemeral data in a message-passing between programs context. It's absolutely important for persistent, editable data. It seems like more of a feature to me to use these special characters for special contexts and not have escaping typeable characters be load-bearing.

    Further, my editor seems to print control characters just fine, though entering them would be a bit of a pain and the behavior is likely configurable.

superb_dev a day ago

Nitpick but I wouldn’t call C a universal FFI, it’s merely the de facto FFI. C can’t meaningfully represent any complex language semantics (not that it should or even could)

  • packetlost 19 hours ago

    It's a near universal de facto FFI if we're being pedantic. My whole point was the reason that's the case is it has essentially no runtime expectations at all, which is a requirement for a truly universal FFI.

WorldMaker 2 days ago

Instead of inventing some new delimited formats, we also already have nearly ubiquitous interchange formats like JSON. If you wanted to reduce the in memory footprint versus plaintext encoding the JSON there are good options like BSON and CBOR.

  • packetlost a day ago

    JSON isn't streamable. Neither is CBOR for that matter, though it's not quite as bad.

    • superb_dev a day ago

      Individual JSON records are streamable

      • packetlost 19 hours ago

        JSON itself, no, that's not the case. If you accept an extension like JSON Lines, it can be. Many decoders will barf if you have multiple top level objects in the same file. If you want portable, non-ephemeral data JSON Lines is a poor choice.