Comment by norir

Comment by norir 16 hours ago

42 replies

View on Hacker News

I don't love a good deal of Lua's syntax, but I do think the authors had good reasons for their choices and have generally explained them. Even if you disagree, I think "without good reasons" is overly dismissive.

Personally though, I think the distinctive choices are a boon. You are never confused about what language you are writing because Lua code is so obviously Lua. There is value in this. Once you have written enough Lua, your mind easily switches in and out of Lua mode. Javascript, on the other hand, is filled with poor semantic decisions which for me, cancel out any benefits from syntactic familiarity.

More importantly, Lua has a crucial feature that Javascript lacks: tail call optimization. There are programs that I can easily write in Lua, in spite of its syntactic verbosity, that I cannot write in Javascript because of this limitation. Perhaps this particular JS implementation has tco, but I doubt it reading the release notes.

I have learned as much from Lua as I have Forth (SmallTalk doesn't interest me) and my programming skill has increased significantly since I switched to it as my primary language. Lua is the only lightweight language that I am aware of with TCO. In my programs, I have banned the use of loops. This is a liberation that is not possible in JS or even c, where TCO cannot be relied upon.

In particular, Lua is an exceptional language for writing compilers. Compilers are inherently recursive and thus languages lacking TCO are a poor fit (even if people have been valiantly forcing that square peg through a round hole for all this time).

Having said all that, perhaps as a scripting language for Redis, JS is a better fit. For me though Lua is clearly better than JS on many different dimensions and I don't appreciate the needless denigration of Lua, especially from someone as influential as you.

kbenson 12 hours ago

> For me though Lua is clearly better than JS on many different dimensions and I don't appreciate the needless denigration of Lua, especially from someone as influential as you.

Is it needless? It's useful specifically because he is someone influential, and someone might say "Lua was antirez's choice when making redis, and I trust and respect his engineering, so I'm going to keep Lua as a top contender for use in my project because of that" and him being clear on his choices and reasoning is useful in that respect. In any case where you think he has a responsibility to be careful what he says because of that influence, that can also be used in this case as a reason he should definitely explain his thoughts on it then and now.

Reply View 0 replies
ksec 6 hours ago

> I think the distinctive choices are a boon. You are never confused about what language you are writing because Lua code is so obviously Lua. There is value in this.

This. And not just Lua , but having different kind of syntax for scripting languages or very high level languages signal it is something entirely different, and not C as in system programming language.

The syntax is also easier for people who dont intend to make programming as their profession, but simply want something done. It used to be the case in the old days people would design simple PL for new beginners, ActionScript / Flash era and even Hypercard before that. Unfortunately the industry is no longer interested in it, and if anything intend to make every as complicated as possible.

Reply View 0 replies
bakkoting 14 hours ago

Formally JavaScript is specified as having TCO as of ES6, although for unfortunate and painful reasons this is spec fiction - Safari implements it, but Firefox and Chrome do not. Neither did QuickJS last I checked and I don't think this does either.

Reply View 1 reply
  • [removed] 2 hours ago
    [deleted]
    Reply View 0 replies
teo_zero 14 hours ago

> Lua has a crucial feature that Javascript lacks: tail call optimization.

I'm not familiar with Lua, but I expect tco to be a feature of the compiler, not of the language. Am I wrong?

Reply View 16 replies
  • mananaysiempre 12 hours ago

    You’re wrong in the way in which many people are wrong when they hear about a thing called “tail-call optimization”, which is why some people have been trying to get away from the term in favour of “proper tail calls” or something similar, at least as far as R5RS[1]:

    > A Scheme implementation is properly tail-recursive if it supports an unbounded number of active tail calls.

    The issue here is that, in every language that has a detailed enough specification, there is some provision saying that a program that makes an unbounded number of nested calls at runtime is not legal. Support for proper tail calls means that tail calls (a well-defined subgrammar of the language) do not ever count as nested, which expands the set of legal programs. That’s a language feature, not (merely) a compiler feature.

    [1] https://standards.scheme.org/corrected-r5rs/r5rs-Z-H-6.html#...

    Reply View 7 replies
    • teo_zero 11 hours ago

      Thank you for the precise answer.

      I still think that the language property (or requirement, or behavior as seen by within the language itself) that we're talking about in this case is "unbounded nested calls" and that the language specs doesn't (shouldn't) assume that such property will be satisfied in a specific way, e.g. switching the call to a branch, as TCO usually means.

      Reply View 1 reply
      • mananaysiempre 4 hours ago

        Unbounded nested calls as long as those calls are in tail position, which is a thing that needs to be defined—trivially, as `return EXPR(EXPR...)`, in Lua; while Scheme, being based around expressions, needs a more careful definition, see link above.

        Otherwise yes. For instance, Scheme implementations that translate the Scheme program into portable C code (not just into bytecode interpreted by C code) cannot assume that the C compiler will translate C-level tail calls into jumps and thus take special measures to make them work correctly, from trampolines to the very confusingly named “Cheney on the M.T.A.”[1], and people will, colloquially, say those implementations do TCO too. Whether that’s correct usage... I don’t think really matters here, other than to demonstrate why the term “TCO” as encountered in the wild is a confusing one.

        [1] https://www.plover.com/misc/hbaker-archive/CheneyMTA.html

        Reply View 0 replies
    • IgorPartola 9 hours ago

      I sort of see what you are getting at but I am still a bit confused:

      If I have a program that based on the input given to it runs some number of recursions of a function and two compilers of the language, can I compile the program using both of them if compiler A has PTC and compiler B does not no matter what the actual program is? As in, is the only difference that you won’t get a runtime error if you exceed the max stack size?

      Reply View 4 replies
      • mananaysiempre 3 hours ago

        That is correct, the difference is only visible at runtime. So is the difference between garbage collection (whether tracing or reference counting) and lack thereof: you can write a long-lived C program that calls malloc() throughout its lifetime but never free(), but you’re not going to have a good time executing it. Unless you compile it with Fil-C, in which case it will work (modulo the usual caveats regarding syntactic vs semantic garbage).

        Reply View 0 replies
      • Zacharias030 5 hours ago

        I think features of the language can make it much easier (read: possible) for the compiler to recognize when a function is tail call optimizable. Not every recursion will be, so it matters greatly what the actual program is.

        Reply View 1 reply
        • mananaysiempre 3 hours ago

          It is a feature of the language (with proper tail calls) that a certain class of calls defined in the spec must be TCOd, if you want to put things that way. It’s not just that it’s easier for the compiler to recognize them, it’s that it has to.

          (The usual caveats about TCO randomly not working are due to constraints imposed by preexisting ABIs or VMs; if you don’t need to care about those, then the whole thing is quite straightforward.)

          Reply View 0 replies
      • [removed] 7 hours ago
        [deleted]
        Reply View 0 replies
  • kerkeslager 12 hours ago

    I don't think you're wrong per se. This is a "correct" way of thinking of the situation, but it's not the only correct way and it's arguably not the most useful.

    A more useful way to understand the situation is that a language's major implementations are more important than the language itself. If the spec of the language says something, but nobody implements it, you can't write code against the spec. And on the flip side, if the major implementations of a language implement a feature that's not in the spec, you can write code that uses that feature.

    A minor historical example of this was Python dictionaries. Maybe a decade ago, the Python spec didn't specify that dictionary keys would be retrieved in insertion order, so in theory, implementations of the Python language could do something like:

      >>> abc = {}
  >>> abc['a'] = 1
  >>> abc['b'] = 2
  >>> abc['c'] = 3
  >>> abc.keys()
  dict_keys(['c', 'a', 'b'])
    But the CPython implementation did return all the keys in insertion order, and very few people were using anything other than the CPython implementation, so some codebases started depending on the keys being returned in insertion order without even knowing that they were depending on it. You could say that they weren't writing Python, but that seems a bit pedantic to me.

    In any case, Python later standardized that as a feature, so now the ambiguity is solved.

    It's all very tricky though, because for example, I wrote some code a decade that used GCC's compare-and-swap extensions, and at least at that time, it didn't compile on Clang. I think you'd have a stronger argument there that I wasn't writing C--not because what I wrote wasn't standard C, but because the code I wrote didn't compile on the most commonly used C compiler. The better approach to communication in this case, I think, is to simply use phrases that communicate what you're doing: instead of saying "C", say "ANSI C", "GCC C", "Portable C", etc.--phrases that communicate what implementations of the language you're supporting. Saying you're writing "C" isn't wrong, it's just not communicating a very important detail: what implementations of the compiler can compile your code. I'm much more interested in effectively communicating what compilers can compile a piece of code than pedantically gatekeeping what's C and what's not.

    Reply View 4 replies
    • beagle3 4 hours ago

      Python’s dicts for many years did not return keys in insertion order (since Tim Peters improved the hash in iirc 1.5 until Raymond Hettinger improved it further in iirc 3.6).

      After the 3.6 changed, they were returned in order. And people started relying on that - so at a later stage, this became part of the spec.

      Reply View 0 replies
    • heavyset_go 6 hours ago

      There actually was a time when Python dictionary keys weren't guaranteed to be in the order they were inserted, as implemented in CPython, and the order would not be preserved.

      You could not reliably depend on that implementation detail until much later, when optimizations were implemented in CPython that just so happened to preserve dictionary key insertion order. Once that was realized, it was PEP'd and made part of the spec.

      Reply View 0 replies
    • kristianp 10 hours ago

      Are you saying that Lua's TCO is an accidental feature due to the first implementation having it? How accurate is that?

      Reply View 1 reply
      • kerkeslager 9 hours ago

        What? No, I'm definitely not saying that.

        I'm saying it isn't very useful argue about whether a feature is a feature of the language or a feature of the implementation, because the language is pretty useless independent of its implementation(s).

        Reply View 0 replies
  • naasking 14 hours ago

    If the language spec requires TCO, I think you can reasonably call it part of the language.

    Reply View 2 replies
    • teo_zero 11 hours ago

      It wouldn't be the first time the specs have gone too far and beyond their perimeter.

      C's "register" variables used to have the same issue, and even "inline" has been downgraded to a mere hint for the compiler (which can ignore it and still be a C compiler).

      Reply View 1 reply
      • 201984 9 hours ago

        inline and register still have semantic requirements that are not just hints. Taking the address of a register variable is illegal, and inline allows a function to be defined in multiple .c files without errors.

        Reply View 0 replies
xonix 16 hours ago

Re: TCO

Does the language give any guarantee that TCO was applied? In other words can it give you an error that the recursion is not of tail call form? Because I imagine a probability of writing a recursion and relying on it being TCO-optimized, where it's not. I would prefer if a language had some form of explicit TCO modifier for a function. Is there any language that has this?

Reply View 6 replies
lioeters 16 hours ago

> as my primary language

I'd love to hear more how it is, the state of the library ecosystem, language evolution (wasn't there a new major version recently?), pros/cons, reasons to use it compared to other languages.

About tail-calls, in other languages I've found sometimes a conversion of recursive algorithm to a flat iterative loop with stack/queue to be effective. But it can be a pain, less elegant or intuitive than TCO.

Reply View 1 reply
  • alexdowad 15 hours ago

    Lua isn't my primary programming language now, but it was for a while. My personal experience on the library ecosystem was:

    It's definitely smaller than many languages, and this is something to consider before selecting Lua for a project. But, on the positive side: With some 'other' languages I might find 5 or 10 libraries all doing more or less the same thing, many of them bloated and over-engineered. But with Lua I would often find just one library available, and it would be small and clean enough that I could easily read through its source code and know exactly how it worked.

    Another nice thing about Lua when run on LuaJIT: extremely high CPU performance for a scripting language.

    In summary: A better choice than it might appear at first, but with trade-offs which need serious consideration.

    Reply View 0 replies
vanviegen 5 hours ago

I do not think your compiler argument in support of TCO is very convincing.

Do you really need to write compilers with limitless nesting? Or is nesting, say, 100.000 deep enough, perhaps?

Also, you'll usually want to allocate some data structure to create an AST for each level. So that means you'll have some finite limit anyway. And that limit is a lot easier to hit in the real world, as it applies not just to nesting depth, but to the entire size of your compilation unit.

Reply View 1 reply
  • lpribis 4 hours ago

    TCO is not just for parse trees or AST, but in imperative languages without TCO this is the only place you are "forced" to use recursion. You can transform any loop in you program to recursion if you prefer, which is what the author does.

    Reply View 0 replies
kerkeslager 12 hours ago

> More importantly, Lua has a crucial feature that Javascript lacks: tail call optimization. There are programs that I can easily write in Lua, in spite of its syntactic verbosity, that I cannot write in Javascript because of this limitation. Perhaps this particular JS implementation has tco, but I doubt it reading the release notes.

> [...] In my programs, I have banned the use of loops. This is a liberation that is not possible in JS or even c, where TCO cannot be relied upon.

This is not a great language feature, IMO. There are two ways to go here:

1. You can go the Python way, and have no TCO, not ever. Guido van Rossum's reasoning on this is outlined here[1] and here[2], but the high level summary is that TCO makes it impossible to provide acceptably-clear tracebacks.

2. You can go the Chicken Scheme way, and do TCO, and ALSO do CPS conversion, which makes EVERY call into a tail call, without language user having to restructure their code to make sure their recursion happens at the tail.

Either of these approaches has its upsides and downsides, but TCO WITHOUT CPS conversion gives you the worst of both worlds. The only upside is that you can write most of your loops as recursion, but as van Rossum points out, most cases that can be handled with tail recursion, can AND SHOULD be handled with higher-order functions. This is just a much cleaner way to do it in most cases.

And the downsides to TCO without CPS conversion are:

1. Poor tracebacks.

2. Having to restructure your code awkwardly to make recursive calls into tail calls.

3. Easy to make a tail call into not a tail call, resulting in stack overflows.

I'll also add that the main reason recursion is preferable to looping is that it enables all sorts of formal verification. There's some tooling around formal verification for Scheme, but the benefits to eliminating loops are felt most in static, strongly typed languages like Haskell or OCaml. As far as I know Lua has no mature tooling whatsoever that benefits from preferring recursion over looping. It may be that the author of the post I am responding to finds recursion more intuitive than looping, but my experience contains no evidence that recursion is inherently more intuitive than looping: which is more intuitive appears to me to be entirely a function of the programmer's past experience.

In short, treating TCO without CPS conversion as a killer feature seems to me to be a fetishization of functional programming without understanding why functional programming is effective, embracing the madness with none of the method.

EDIT: To point out a weakness to my own argument: there are a bunch of functional programming language implementations that implement TCO without CPS conversion. I'd counter by saying that this is a function of when they were implemented/standardized. Requiring CPS conversion in the Scheme standard would pretty clearly make Scheme an easier to use language, but it would be unreasonable in 2025 to require CPS conversion because so many Scheme implementations don't have it and don't have the resources to implement it.

EDIT 2: I didn't mean for this post to come across as negative on Lua: I love Lua, and in my hobby language interpreter I've been writing, I have spent countless hours implementing ideas I got from Lua. Lua has many strengths--TCO just isn't one of them. When I'm writing Scheme and can't use a higher-order function, I use TCO. When I'm writing Lua and can't use a higher order function, I use loops. And in both languages I'd prefer to use a higher order function.

[1] https://neopythonic.blogspot.com/2009/04/tail-recursion-elim...

[2] https://neopythonic.blogspot.com/2009/04/final-words-on-tail...

Reply View 1 reply
  • kerkeslager 6 hours ago

    EDIT 3: Looking at Lua's overall implementation, it seems to be focused on being fast and lightweight.

    I don't know why Lua implemented TCO, but if I had to guess, it's not because it enables you to replace loops with recursion, it's because it... optimizes tail calls. It causes tail calls to use less memory, and this is particularly effective in Lua's implementation because it reuses the stack memory that was just used by the parent call, meaning it uses memory which is already in the processor's cache.

    The thing is, a loop is still going to be slightly faster than TCOed recursion, because you don't need to move the arguments to the tail call function into the previous stack frame. In a loop your counters and whatnot are just always using the same memory location, no copying needed.

    Where TCO really shines is in all the tail calls that aren't replacements for loops: an optimized tail call is faster than a non-optimized tail call. And in real world applications, a lot of your calls are tail calls!

    I don't necessarily love the feature, for the reasons that I detailed in the previous post. But it's not a terrible problem, and I think it at makes sense as an optimization within the context of Lua's design goals of being lightweight and fast.

    Reply View 0 replies
unclad5968 15 hours ago

> I do think the authors had good reasons for their choices and have generally explained them

I'm fairly certain antirez is the author of redis

Reply View 2 replies
  • pedroza_alex 14 hours ago

    The word "authors" in that phrase refers to the authors of Lua, not Redis.

    Reply View 0 replies
  • rapind 14 hours ago

    Pretty sure he's talking about Lua's authors.

    Reply View 0 replies
znpy 4 hours ago

No offence but it seems that all of the people that are replying to this comment are essentially screaming in the void, if anything among each other.

I scrolled most of this sub thread and gp seem to not be replying to any of the replies they got.

Reply View 0 replies