Graphics livecoding in Common Lisp
(kevingal.com)201 points by adityaathalye a day ago
201 points by adityaathalye a day ago
Jack Rusher's recent interview is well worth reading too (the "stop writing dead programs" guy).
> On the need to sustain your creative drive in the face of technological change
> https://thecreativeindependent.com/people/multi-disciplinary...
nb. I recently submitted it here: https://news.ycombinator.com/item?id=43759204
That's a great submission! I put it in the second-chance pool (https://news.ycombinator.com/pool, explained at https://news.ycombinator.com/item?id=26998308), so it will get a random placement on HN's front page.
Thanks, dang!
More people need to read Jack's interview, especially in the context of the contemporary tension between generative AI v/s wetware creativity.
I've been plastering that interview in every discord / slack / zulip I lurk in, and been sliding it into everyone's Whatsapp DMs :D
Can't you just do live code reloading with a DLL? Have a small core which is the exe, and then all of the logic is a DLL which can be reloaded when the core detects changes to the dependent files...
Kind of, but there is a reason this is a whole business, see Live++.
Dynamic libraries for languages designed from scratch to support such features, have runtime support to handle all reloading use cases, like in Xerox PARC OSes, and inspirations thereof.
Traditional dynamic libraries for UNIX and Windows, are designed with C in mind, thus quite fragile to implement similar capabilities.
Oh wow, just had to log in and give you a high-five for livelove because this is the first I've heard of it and it sounds like the sort of thing I absolutely need to try out.
I remember giving Love2D a go a couple of years ago with Fennel and the lack of such a thing sent me grumbling back to Common Lisp. I'd never even have thought of building that functionality in Love/Lua myself - assuming it's something that the runtime just didn't support - and it absolutely would never have occurred to me to use LSP to set it up. I've not even used it yet and it's already doing things to my brain, so thanks!
Excited to spread the brain worm. Don't hesitate to join in the fun / log issues / contribute / share how you use it!
Well, for me it’s not enough because I need to get back to where I was, repeating the same actions so it gets to the same state. With live dev I don’t need this, or a history replay method. I only update the running code. Heck I could also update the in memory var too if so I want.
It’s good that it’s fast. Still no good enough!
Recompile and hot reload, maybe. 2 seconds if you're very lucky. Many setups are much slower. I've seen some really cool projects over the last few years- things like tcc + hot reload that have really good turn around time.
But "live" is a whole different thing. And most languages aren't built with the expectation that you'll be patching it while it's running - at least as standard operating procedure and without nuking state.
And that's a key part.
I think you should be able to build an app and develop a game without ever having to restart them.
And that would come with new challenges and the need for new tooling / constructs. E.g. When you change a type / structure, and there's existing state, what happens? (migration, deletion etc)
There are similar trends in music and sound art, which can be experienced with Glicol (https://glicol.org/) as well as many other languages here:
https://github.com/toplap/awesome-livecoding
also this live coding book is free to read!
Have you done some kind of a write-up on how to do this? Lua is popular enough that I suspect there might actually be a demand of a readymade library to use hot reloading.
I haven't done a write up- I could, for sure. The code is all open source and the Lua side is only a few files.
The short of it is, you need a communication medium between LSP and your program. When you edit a text file, it needs to tell the program it updated, along with the latest text.
To communicate the values of variables is more work- you need a way to keep track of all changes to all variables. I wrote auto instrumentation code that wraps every modification to every variable in the files it instruments (which for livelove is everything but the library files themselves). Obviously this isn't perfect, but it does pretty well- and to see the value of anything you just need to make sure it's assigned to something somewhere or modified (very likely already the case).
---
There are absolutely hot reload Lua libraries- there's even an IDE where this is a key feature!
I use Common Lisp (CL) for some of my small personal projects. A few publicly available examples I can share include my website [1] and a now-defunct mathematics pastebin [2]. My CL projects are usually text-oriented, not graphics-oriented. What keeps me coming back to CL is how convenient the live coding environment is.
When I am exploring ideas that are not fully concrete yet, I can begin by writing a small set of functions with very basic functionality. Then as the ideas evolve, I can refine existing functions or add new ones, then quickly "reload" them (with say, C-M-x in Emacs), and see the effects immediately. There is no separate compile or rebuild step. I don't have to restart any service or application. The effects are truly immediate -- what previously did X, now does Y.
In the Python or JavaScript ecosystems, similar live reloading capability is often provided by frameworks (e.g., FastAPI, React, etc.), which monitor file changes during development. In CL, it's just part of the language implementation itself.
Of course, at the end of the day, everything is committed and pushed to a version control system. Sometimes I restart the application too just to be sure it reflects the actual source, especially, after hours of live reloading. The stereotype of Lisp programmers making all of their modifications in an ephemeral image and then dumping it all to disk is not something I have actually seen in practice, at least not among the people I know.
So the rest of the software development practices happen to be typical. But during exploration, debugging, or troubleshooting, the live coding experience in Common Lisp is so seamless, it feels like programming at the speed of thought.
I tried using CL this year for a new version of my personal static site (blog) generator.
Yes, it's incredible and still an alien technology in some respects. Native AOT compilation at the level of functions with hot swapping - and without any ceremony, always available in the IDE - is one of them. As the OP writes at the end, only Smalltalk and Erlang come close.
But, when viewed objectively, in 2025 CL is quite behind in some aspects:
- Only one commercial IDE offers a true graphical debugger. Both SLIME and SLY (I'm not sure about stickers) fall behind JS or Python environments.
- Asynchronicity is based on explicit callbacks or callbacks wrapped in promises (and some macros). No async/await, no coroutines. Since writing async code is harder, people default to sequential code and a thread pool. For IO-bound apps, this unnecessarily adds synchronization problems that would be mostly avoided in single-threaded concurrency.
- Tiny ecosystem of libraries. Due to the stability (or stagnation) of the language, a lot of old code still works, but even with this, finding what you need on Quicklisp can be challenging.
- The stdlib is old, crufty, and quirky. It's also very small by today's "batteries included" standards. Initiatives like CL21 or CIEL try to alleviate this somewhat.
- Even in SBCL, the type system is limited compared to MyPy, TypeScript, or TypedRacket.
- Bolting packages (module system) on top of symbols leads to some problems in practice.
- The progress in CL development is severely hampered by the set-in-stone standard, small pool of users, and the need to reconcile 10 different implementations.
It's still a nice tool for many uses, and the interactive development is indeed comfortable and productive. Still, if nothing significant happens, I don't think CL will have a chance to gain popularity again. Up until now, a successor language was hard to justify - despite some shortcomings, CL was still a language from the future. Now that future is largely here already, so the shortcomings become more and more glaring. There's SICL, but it'll take another decade (if at all) before we can expect results from that.
I'm now looking at Jank and Gerbil with some expectations, though I think I'll stick with CL in my current project.
> fall behind JS or Python environments.
I dunno what you are using but we are a small team working on large codebases; one in js/ts mix, one in python and one in cl. Everyone cannot wait for it to be CL time; everything js and especially ts feels so incredibly clunky and half baked. The debuggers are awful and actually don't even stop on breakpoints a lot of the time for no apparent reason.
For most things in cl you have libraries (async etc) and for the rest you roll your own (much) faster than you can try out the tearinducing garbage on npm to figure out what half baked buggy thing might fit your purpose (before it gets hijacked by some north Korean hacker group of ofcourse). With LLMs this has become way easier.
Using the strong typesystem cl has and then adding on coalton when you need it, I prefer over typescript, if only for the speed of dev / no noticeable recompile.
> Bolting packages (module system) on top of symbols leads to some problems in practice.
Packages are namespaces, unrelated to modules. They are not used to organize dependencies but to remove ambiguity of naming and to provide encapsulation.
> The progress in CL development is severely hampered by the set-in-stone standard, small pool of users
The language is extensible, compilers do get new features added, sometimes in a coordinated manner. Standard being stable is not that significant an obstacle to moving forward; pool of users being small is much more noticeable. Yet more noticeable is that community is less healthy than the Emacs one. This comparison is valid because development in Emacs Lisp is attractive for the same reasons, unique to it and CL; they also happen to have extremely similar syntax. Elisp is moving forward steadily, and in recent years quite rapidly. CL would progress forward similarly, or better, had it had something like Emacs that provided that synergetic effect.
It's funny but I think your exact comment could have been written 10 years ago, substituting 2025 with 2015. Some things might be ±2 years. This observation could be seen as both a positive and a negative about both the language and people writing about it.
Things have changed since 2015, though, and in 2015 looking back at 2005 things had changed too. (Quicklisp was only on the scene in 2010, which is somewhat late compared to its equivalents in other languages.) Popularity-wise, I continue to be positive about CL's global future, even if nothing is done about your bullet points. Over the years I keep noticing new software, new implementation features, the occasional new book, new papers, new jobs, and new people writing stuff, I expect that to continue. Some long-standing companies and systems are still going, avoiding the death of a total rewrite or just better competition, some of them like managing train schedules are older than me. It looks like a living and (slowly) growing ecosystem. It won't ever again enjoy a top-10 language status by making technical changes, but I don't think many people care. (There's a similar level of not caring when it comes to non-GC languages and how their overall job market share is probably not going to shift from ~10% any time soon (barring the potential of forced mass early retirement due to AI).)
I was considering a more argumentative reply to your bullet points because I think they're not quite correct... There's a lot of nitpicking that could be done, and further elaborating the nuances might lead to a conclusion that, to the extent the points are true, they don't really matter -- in other words, my reaction to the list even in its strongest form (by my views anyway) is still kind of just a big shrug, so why bother trying to argue with it? I'll keep using Common Lisp. (Even despite my own pet complaints not on your list.) And given your final statement about sticking with it, I wonder if you also kind of shrug at the importance of those things.
> I wonder if you also kind of shrug at the importance of those things.
Yes, to an extent. I suspect I could nitpick on the points ("but X still works", "this isn't how you work with CL", "it's still better than alternatives", "you can work around that") just as easily as you. I still like working with Common Lisp, after all. Even the worst offender on my list - no native coroutines (or continuations) support - was true for both Python and JavaScript for most of their history. And in their cases, there was no way to sweeten the syntax with macros (as BlackBird does for CL). So yeah, these are all shruggable, even if ever more embarrassing as time passes and the rest of the world leaves some of the issues behind.
for folks looking for libraries: https://github.com/CodyReichert/awesome-cl/
(I don't know for popularity but my personal opinion is that CL is still unmatched: this level of interactive development + good debugging tools + excellent implementation that compiles to fast machine code + fast startup for binaries + self-contained binaries + stable yet improving language, implementations and ecosystem + connect from a running program + commercial vendors if you need + … such a unique and productive set)
(re. the type system: https://github.com/coalton-lang/coalton/ for a Haskell on top of CL now)
If you look at awesome-cl you may surmise that Common Lisp has a bit of a dearth of libraries, but a glance at OCICL's repositories shows 79 pages worth of libraries, and Ultralisp even claims to have 2,170 projects. So while it's not at the level of C, Java, or Perl; it's nothing to be sneezed at either.
The fact is that it's so easy to write your own code in CL, that the majority of glue code in NPM, Maven and the likes don't make sense to be written.
An example can be taken from a comparison between VS Code and Emacs. In the former, you have a whole browser engine and quite a bit of code to implement an editor on top of that. In the latter, you just have some primitive and the rest of it is LISP implementing multiple applications. LISP encourages less layers because the data structures and the language are so flexible.
My bad, I mean source-level debugger. Something that allows you to step through forms as they are in the source, unexpanded and not decompiled. It should allow jumping to a line or over an expression, continuing to a point, seamlessly stepping into subroutines, and stepping out of expressions. It would be nice to have single-activation and conditional breakpoints. Variables in the executed portion of the code should be annotated with their values. Sometimes watchers - forms auto-evalled in each frame - are helpful, though these are implemented in some CLs, IIRC. SLY's stickers are close, but they are gathered during execution and replayed later, so the break-edit-recompile-redo workflow doesn't work with them. My experience debugging CL using SBCL and SLIME reminds me of debugging CoffeeScript in a browser before source maps were a thing... It's still 100% doable, just less convenient.
That was such an unexpected claim, I had to look up on which platforms CMUCL currently runs. Seems it has been ported to MacOS (which I take you are referring to as there are perfectly fine working X11 servers available for M$ Windows). Kudos. If I'd be using MacOS, I'd have also a look at Clozure CL (CCL).
For clojure, you can use quil:
I’ve been using quil as I work through _The_Nature_of_Code_ by Daniel Shiffman:
Processing is what ignited my passion for programming and Quil has become my favorite way of writing it. It is amazing that you can re-evaluate the draw/update function in a running sketch and immediately see the changes, without having to reload the whole thing. And on top of it you have the beauty of the whole Clojure Stdlib with its immutable datastructures.
I just learned that there is now a tweak mode in Processing that lets you tweak certain parameters in the code (via draggable values, etc.) while the sketch is running, which is pretty awesome for experimenting with values. However, you still have to reload the whole sketch when you want to change other parts of the code, you can’t just eval a function in the editor and get immediate feedback like in Quil.
Are you just mentally switching over from their code examples to Quil, or?
If you mean translating from the JavaScript, yes. Quil wraps Processing. Quil did not wrap the vector library (P5.Vector) but most of vector operations are pretty easy to write in base clojure or you can use clojure.matrix. The fun(ctional)-mode lines up with passing initial state through the functions, which clojure makes easy.
This reminds me of the excellent CEPL library (https://github.com/cbaggers/cepl). You can write live shader like programs with cepl with an OpenGL backend.
The author also streamed hours and hours of content using it to make increasingly complicated programs:
https://www.youtube.com/watch?v=I0kWZP9L9Kc
Wonderful CL channel in general.
Among the plethora of strong suits that Common Lisp brings along, the live coding is hands down the best part of it. There is just no better way to experience programming than that.
Sure, the condition system is awesome, CLOS is next to nothing, but the live coding just wins al the way. All the other aspects are just icing on the cake for me.
Reminds me of something from like 2007ish(?)~ called (fluxus). It was a lisp text editor with a render target in the background and some nice standard library for making 3d objects appear or sfx play. Everything was constantly evaluating/hotloading in the background.
So much fun, I can't find any of the videos in a quick search, so maybe lost to time. Great performative lisping in them hills though.
EDIT: I did find the old page on the wayback machine. https://web.archive.org/web/20120810224932/http://www.pawfal...
I am not sure if this is what you mean, but the original UCBLogo (which I think is used in the Turtle Geometry book) is still alive and maintained[1] (not by the original authors, but Brian Harvey seems to chime in every now and then) and it does run well on modern computers.
Now that I think about it, Logo seems to be pretty much a livecoding environment (not surprising given that it is a Lisp, but with less parentheses). You can define and edit functions from the REPL while the program continues with the same state, the same canvas. You can even pause e.g. a running procedure that draws a polygon, rotate and move the turtle and then continue the polygon procedure with that new state (at least this is possible with UCBLogo).
slightly related, i've been following this game called replicube (https://store.steampowered.com/app/3401490/)
this is livecoding 3d voxel to solve puzzles. the demo was fun. looks like it'll be released soon.
The trailer for that is my nightmare - writing hyper specific if statements everywhere to handle a million edge cases.
This sounds like a great developer experience, but the path from live-coded LISP graphics to a game that can be made public on ie. Steam is a long and troublesome path, compared to using for example OpenGL and C++.
I wish more programming languages focused on quick compilation times, easy cross compilation and straightforward deployment in general.
Live development is still so under-explored, and is just so exciting to work with.
One of my favorite talks is "Stop Writing Dead Programs" (https://www.youtube.com/watch?v=8Ab3ArE8W3s) and touches on a lot of what could be in terms of live development.
Lisp is very well-suited to live development due to code being data, but live development doesn't need to be lispy.
I built a live development extension for Love2D which lets you do graphics livecoding (both lua and glsl) in real-time - every keystroke updating the output, (if a valid program).
https://github.com/jasonjmcghee/livelove
Here are some (early) demos of things you can do with it:
https://gist.github.com/jasonjmcghee/9701aacce85799e0f1c7304...
So many cool things once you break down the barrier between editor and running program.
I've also asked myself the question of, what would a language look like that was natively built for live development, and built out a prototype - though it's definitely a sandbox so haven't posted it anywhere yet, other than demos on mastadon.