Comment by flohofwoe

Comment by flohofwoe 20 hours ago

33 replies

If calling the same function with a different argument would be considered 'function coloring', every function in a program is 'colored' and the word loses its meaning ;)

Zig actually also had solved the coloring problem in the old and abandondend async-await solution because the compiler simply stamped out a sync- or async-version of the same function based on the calling context (this works because everything is a single compilation unit).

zarzavat 12 hours ago

In that case JS is not colored either because an async function is simply a normal function that returns a Promise.

As far as I understand, coloring refers to async and sync functions having the same calling syntax and interface, I.e.

    b = readFileAsync(p)
    b = readFileSync(p)
share the same calling syntax. Whereas

    b = await readFileAsync(p)
    readFileAsync(p).then(b => ...)
    
    b = readFileSync(b)
are different.

If you have to call async functions with a different syntax or interface, then it's colored.

  • flohofwoe 3 hours ago

    > In that case JS is not colored either because an async function is simply a normal function that returns a Promise.

    Exactly, IMHO at least, JS doesn't suffer from the coloring problem because you can call async functions from sync functions (because the JS Promise machinery allows to fall back to completion callbacks instead of using await). It's the 'virality' of await which causes the coloring problem, but in JS you can freely mix await and completion callbacks for async operations).

    • zarzavat 2 hours ago

      await isn't viral per se, it's a purely local transformation. The virality is from CPS/callbacks and Promise.

unscaled 6 hours ago

Let's revisit the original article[1]. It was not about arguments, but about the pain of writing callbacks and even async/await compared to writing the same code in Go. It had 5 well-defined claims about languages with colored functions:

1. Every function has a color.

This is true for the new zig approach: functions that deal with IO are red, functions that do not need to deal with IO are blue.

2. The way you call a function depends on its color.

This is also true for Zig: Red functions require an Io argument. Blue functions do not. Calling a red function means you need to have an Io argument.

3. You can only call a red function from within another red function.

You cannot call a function that requires an Io object in Zig without having an Io in context.

Yes, in theory you can use a global variable or initialize a new Io instance, but this is the same as the workarounds you can do for calling an async function from a non-async function For instance, in C# you can write 'Task.Run(() -> MyAsyncMethod()).Wait()'.

4. Red functions are more painful to call.

This is true in Zig again, since you have to pass down an Io instance.

You might say this is not a big nuisance and almost all functions require some argument or another... But by this measure, async/await is even less troublesome. Compare calling an async function in Javascript to an Io-colored function in Zig:

  function foo() {
    blueFunction(); // We don't add anything
  }

  async function bar() {
    await redFunction(); // We just add "await"
  }
And in Zig:

  fn foo() void {
    blueFunction()
  }

  fn bar(io: Io) void {
    redFunction(io); // We just add "io".
  }

Zig is more troublesome since you don't just add a fixed keyword: you need a add a variable that is passed along through somewhere.

5. Some core library functions are red.

This is also true in Zig: Some core library functions require an Io instance.

I'm not saying Zig has made the wrong choice here, but this is clearly not colorless I/O. And it's ok, since colorless I/O was always just hype.

---

[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

  • flohofwoe 2 hours ago

    > This is also true for Zig: Red functions require an Io argument. Blue functions do not. Calling a red function means you need to have an Io argument.

    I don't think that's necessarily true. Like with allocators, it should be possible to pass the IO pointer into a library's init function once, and then use that pointer in any library function that needs to do IO. The Zig stdlib doesn't use that approach anymore for allocators, but not because of technical restrictions but for 'transparency' (it's immediately obvious which function allocates under the hood and which doesn't).

    Now the question is, does an IO parameter in a library's init function color the entire library, or only the init function? ;P

    PS: you could even store the IO pointer in a public global making it visible to all code that needs to do IO, which makes the coloring question even murkier. It will be interesting though how the not-yet-implemented stackless coroutine (e.g. 'code-transform-async') IO system will deal with such situations.

  • ghosty141 4 hours ago

    In my opinion you must have function coloring, it's impossible to do async (in the common sense) without it. If you break it down one function has a dependency on the async execution engine, the other one doesn't, and that alone colors them. Most languages just change the way that dependency is expressed and that can have impacts on the ergonomics.

woodruffw 20 hours ago

> If calling the same function with a different argument would be considered 'function coloring', than every function in a program is 'colored' and the word loses its meaning ;)

Well, yes, but in this case the colors (= effects) are actually important. The implications of passing an effect through a system are nontrivial, which is why some languages choose to promote that effect to syntax (Rust) and others choose to make it a latent invariant (Java, with runtime exceptions). Zig chooses another path not unlike Haskell's IO.

SkiFire13 17 hours ago

> Zig actually also had solved the coloring problem in the old and abandondend async-await solution because the compiler simply stamped out a sync- or async-version of the same function based on the calling context (this works because everything is a single compilation unit).

AFAIK this still leaked through function pointers, which were still sync or async (and this was not visible in their type)

  • throwawaymaths 14 hours ago

    Pretty sure the Zig team is aware of this and has plans to fix it before they re-release async.

adamwk 19 hours ago

The subject of the function coloring article was callback APIs in Node, so an argument you need to pass to your IO functions is very much in the spirit of colored functions and has the same limitations.

  • jakelazaroff 19 hours ago

    In Zig's case you pass the argument whether or not it's asynchronous, though. The caller controls the behavior, not the function being called.

    • layer8 18 hours ago

      The coloring is not the concrete argument (Io implementation) that is passed, but whether the function has an Io parameter in the first place. Whether the implementation of a function performs IO is in principle an implementation detail that can change in the future. A function that doesn't take an Io argument but wants to call another function that requires an Io argument can't. So you end up adding Io parameters just in case, and in turn require all callers to do the same. This is very much like function coloring.

      In a language with objects or closures (which Zig doesn't have first-class support for), one flexibility benefit of the Io object approach is that you can move it to object/closure creation and keep the function/method signature free from it. Still, you have to pass it somewhere.

      • messe 16 hours ago

        > Whether the implementation of a function performs IO is in principle an implementation detail that can change in the future.

        I think that's where your perspective differs from Zig developers.

        Performing IO, in my opinion, is categorically not an implementation detail. In the same way that heap allocation is not an implementation detail in idiomatic Zig.

        I don't want to find out my math library is caching results on disk, or allocating megabytes to memoize. I want to know what functions I can use in a freestanding environment, or somewhere resource constrained.

      • derriz 16 hours ago

        > A function that doesn't take an Io argument but wants to call another function that requires an Io argument can't.

        Why? Can’t you just create an instance of an Io of whatever flavor you prefer and use that? Or keep one around for use repeatedly?

        The whole “hide a global event loop behind language syntax” is an example of a leaky abstraction which is also restrictive. The approach here is explicit and doesn’t bind functions to hidden global state.

        • layer8 16 hours ago

          You can, but then you’re denying your callers control over the Io. It’s not really different with async function coloring: https://news.ycombinator.com/item?id=46126310

          Scheduling of IO operations isn’t hidden global state. Or if it is, then so is thread scheduling by the OS.

      • [removed] 15 hours ago
        [deleted]
      • quantummagic 17 hours ago

        Is that a problem in practice though? Zig already has this same situation with its memory allocators; you can't allocate memory unless you take a parameter. Now you'll just have to take a memory allocator AND an additional io object. Doesn't sound very ergonomic to me, but if all Zig code conforms to this scheme, in practice there will only-one-way-to-do-it. So one of the colors will never be needed, or used.

jcranmer 20 hours ago

> If calling the same function with a different argument would be considered 'function coloring', than every function in a program is 'colored' and the word loses its meaning ;)

I mean, the concept of "function coloring" in the first place is itself an artificial distinction invented to complain about the incongruent methods of dealing with "do I/O immediately" versus "tell me when the I/O is done"--two methods of I/O that are so very different that it really requires very different designs of your application on top of those I/O methods: in a sync I/O case, I'm going to design my parser to output a DOM because there's little benefit to not doing so; in an async I/O case, I'm instead going to have a streaming API.

I'm still somewhat surprised that "function coloring" has become the default lens to understand the semantics of async, because it's a rather big misdirection from the fundamental tradeoffs of different implementation designs.

  • omnicognate 13 hours ago

    100% agree, but fortunately I don't think it is the "default lens". If it were nobody would be adding new async mechanisms to languages, because "what color is your function" was a self-described rant against async, in favour of lightweight threads. It does seem to have established itself as an unusually persistent meme, though.

  • conradev 8 hours ago

    My understanding of this design is that you can write the logic separately from the decision to "do I/O immediately" versus "tell me when the I/O is done"

    You can write a parser thats outputs a DOM and run it on a stream, or write a parser with a streaming API and run it synchronously on a buffer. You should pick the optimal tool for the situation, but there is no path dependence anymore.

    • thinkharderdev 40 minutes ago

      Honestly I don't see how that is different than how it works in Rust. Synchronous code is a proper subset of asynchronous code. If you have a streaming API then you can have an implementation that works in a synchronous way with no overhead if you want. For example, if you already have the whole buffer in memory sometimes then you can just use it and the stream will work exactly like a loop that you would write in the sync version.

  • zelphirkalt 12 hours ago

    Function coloring is the issue, that arises in practice, which is why people discuss, whether some approach solves it or does not.

    Why do you think it automatically follows, that with an async I/O you are going to have a streaming API? An async I/O can just like the sync I/O return a whole complete result, only that you are not waiting for that to happen, but the called async procedure will call you back once the result is calculated. I think a streaming API requires additional implementation effort, not merely async.

[removed] 19 hours ago
[deleted]
rowanG077 20 hours ago

If your functions suddenly requires (currently)unconstructable instance "Magic" which you now have to pass in from somewhere top level, that indeed suffers from the same issue as async/await. Aka function coloring.

But most functions don't. They require some POD or float, string or whatever that can be easily and cheaply constructed in place.

  • mk12 11 hours ago

    Colors for 2 ways of doing IO vs colors for doing IO or not are so different that it’s confusing to call both of them “function coloring problem”. Only the former leads to having to duplicate everything (sync version and async version). If only the latter was a thing, no one would have coined the term and written the blog post.

    • rowanG077 11 hours ago

      IMO the problem was never about it actually doing IO or an async actions or whatever. It's about not being able to a call a async function from a sync function. Because in my experience you almost never wholesale move from sync to async everywhere. In fact I would consider that an extremely dangerous practice.