Comment by SkiFire13 16 hours ago
The function coloring problem actually comes up when you implement the async part using stackless coroutines (e.g. in Rust) or callbacks (e.g. in Javascript).
Zig's new I/O does neither of those for now, so hence why it doesn't suffer from it, but at the same time it didn't "solve" the problem, it just sidestepped it by providing an implementation that has similar features but not exactly the same tradeoffs.
In the JS example, a synchronous function cannot poll the result of a Promise. This is meaningfully different when implementing loops and streams. Ex, game loop, an animation frame, polling a stream.
A great example is React Suspense. To suspend a component, the render function throws a Promise. To trigger a parent Error Boundary, the render function throws an error. To resume a component, the render function returns a result. React never made the suspense API public because it's a footgun.
If a JS Promise were inspectable, a synchronous render function could poll its result, and suspended components would not need to use throw to try and extend the language.
.NET has promises that you can poll synchronously. The problem with them is that if you have a single thread, then by definition while your synchronous code is running, none of the async callbacks can be running. So if you poll a Task and it's not complete yet, there's nothing you can do to wait for its completion.
Well, technically you can run a nested event loop, I guess. But that's such a heavy sync-wrapping-async solution that it's rarely used other than as a temporary hack in legacy code.
I see. I guess JS is the only language with the coloring problem, then, which is strange because it's one of the few with a built-in event loop.
This Io business is isomorphic to async/await in Rust or Python [1]. Go also has a built-in "event loop"-type thing, but decidedly does not have a coloring problem. I can't think of any languages besides JS that do.
It’s not the same situation because with async/await you end up with two versions of every function or library (see Rust’s std and crates like async_std, Node’s readFile and readFileSync). In Zig you always pass the “io” parameter to do I/O and you don’t have to duplicate everything.
In Rust, you can always create a new tokio runtime and use that to call an async function from a sync function. Ditto with Python: just create a new asyncio event loop and call `run`. That's actually exactly what an Io object in Zig is, but with a new name.
Looking back at the original function coloring post [1], it says:
> It is better. I will take async-await over bare callbacks or futures any day of the week. But we’re lying to ourselves if we think all of our troubles are gone. As soon as you start trying to write higher-order functions, or reuse code, you’re right back to realizing color is still there, bleeding all over your codebase.
So if this is isomorphic to async/await, it does not "solve" the coloring problem as originally stated, but I'm starting to think it's not much of a problem at all. Some functions just have different signatures from other functions. It was only a huge problem for JavaScript because the ecosystem at large decided to change the type signatures of some giant portion of all functions at once, migrating from callbacks to async.
[1]: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
It's sans-io at the language level, I like the concept.
So I did a bit of research into how this works in Zig under the hood, in terms of compilation.
First things first, Zig does compile async fns to a state machine: https://github.com/ziglang/zig/issues/23446
The compiler decides at compile time which color to compile the function as (potentially both). That's a neat idea, but... https://github.com/ziglang/zig/issues/23367
> It would be checked illegal behavior to make an indirect call through a pointer to a restricted function type when the value of that pointer is not in the set of possible callees that were analyzed during compilation.
That's... a pretty nasty trade-off. Object safety in Rust is really annoying for async, and this smells a lot like it. The main difference is that it's vaguely late-bound in a magical way; you might get an unexpected runtime error and - even worse - potentially not have the tools to force the compiler to add a fn to the set of callees.
I still think sans-io at the language level might be the future, but this isn't a complete solution. Maybe we should be simply compiling all fns to state machines (with the Rust polling implementation detail, a sans-io interface could be used to make such functions trivially sync - just do the syscall and return a completed future).
> I still think sans-io at the language level might be the future, but this isn't a complete solution. Maybe we should be simply compiling all fns to state machines (with the Rust polling implementation detail, a sans-io interface could be used to make such functions trivially sync - just do the syscall and return a completed future).
Can you be more specific what is missing in sans-io with explicit state machine for static and dynamic analysis would not be a complete solution? Serializing the state machine sounds excellent for static and dynamic analysis. I'd guess the debugging infrastructure for optimization passes and run-time debugging are missing or is there more?
How are the tradeoffs meaningfully different? Imagine that, instead of passing an `Io` object around, you just had to add an `async` keyword to the function, and that was simply syntactic sugar for an implied `Io` argument, and you could use an `await` keyword as syntactic sugar to pass whatever `Io` object the caller has to the callee.
I don't see how that's not the exact same situation.