Comment by zenkey

A corollary is that if at all possible try to solve problems without code or, failing that, with less code.

I agree, older code is evidence of survivorship bias. We don't see all of the code that was written with the older code that was removed or replaced (without a code repository).

"Since FORTRAN should virtually eliminate coding and debugging…" -- FORTRAN report, 1954 [1]

If, as you seem to imply and as others have stated, we should no longer even look at the "generated" code, then the LLM prompts are the programs / programming language.

I can't think of a worse programming language, and I am not the only one [2]

However, it does indicate that our current programming languages are way to low-level, too verbose. Maybe we should fix that?

[1] http://www.softwarepreservation.org/projects/FORTRAN/BackusE...

[2] https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...

[3] https://objective.st/

4GL and 5GL are already taken. So this is the 6GL.

https://en.wikipedia.org/wiki/Programming_language_generatio...

But speaking more seriously, how to get this deterministic?

> I think people are still fooling themselves about the relevance of 3GL languages in an AI dominated future.

I think, as happens in the AI summer before each AI winter, people are fooling themselves about both the shape and proximity of the “AI dominated future”.

Assemblers and compilers are (practically) deterministic. LLMs are not.

> It is a matter of time, maybe a decade who knows, until we can produce executables directly from AI systems.

They already can.

Yes I agree this is likely the direction we're heading. I suppose the "Python 4" I mentioned would just be an intermediate step along the way.

Wild thought: maybe coding is a thing of the past? Given that an llm can get fast&deterministic results if needed, maybe a backend for instance, can be a set of functions which are all textual specifications and by following them it can do actions (validations, calculations, etc), approach apis and connect to databases, then produce output? Then the llm can auto refine the specifications to avoid bugs and roll the changes in real time for the next calls? Like a brain which doesn't need predefined coding instructions to fulfill a task, but just understand its scope, how to approach it and learn from the past.

It should work if you feed the data yourself, or at the very least the documentation. I do this with niche languages and it seems to work more or less, but you will have to pay attention to your context length, and of course if you start a new chat, you are back to square one.

I don't know if that's a big blocker now we have abundant synthetic data from a RL training loop where language-specific things like syntax can be learned without any human examples. Human code may still be relevant for learning best practices, but even then it's not clear that can't happen via transfer learning from other languages, or it might even emerge naturally if the synthetic problems and rewards are designed well enough. It's still very early days (7-8 months since o1 preview) so to draw conclusions from current difficulties over a 2-year time frame would be questionable.

Consider a language designed only FOR an LLM, and a corresponding LLM designed only FOR that language. You'd imagine there'd be dedicated single tokens for common things like "class" or "def" or "import", which allows more efficient representation. There's a lot to think about ...

Welcome to the flight, this is your captain speaking. Just want to let you know our entire flight system was vibe coded to the strict standards you expect from our industry, iterated and refined in a virtual environment over twenty virtual-years, with no fallible human eyes reviewing it - even if it were possible to review the mountain of impenetrable machine-generated code. The pilot will be controlling the plane via a cutting-edge LLM interface, prompt-engineering our way to our overseas destination. Relax, get comfortable, and pray to the collective intelligence distilled from Reddit posts.

What percent of applications require that level of reliability?

Vibe coding will be normalized because the vast, vast majority of code is not life or death. That literally what “normal” means.

Exceptional cases like pacemakers and spaceflight will continue to be produced with rigor. Maybe even 1% of produced code will work that way!

While I go into another direction in a sibling comment, lifetimes does not imply not needing garbage collection.

On the contrary, having both allows the productivity of automatic resource management, while providing the necessary tooling to squeeze the ultimate performance when needed.

No need to worry about data structures not friendly to affine/linear types, Pin and Phantom types and so forth.

It is no accident that while Rust has been successful bringing modern lifetime type systems into mainstream, almost everyone else is researching how to combine linear/affine/effects/dependent types with classical automatic resource management approaches.

Rust lifetimes are generally fairly local and don’t impact refactoring too much unless you fundamentally change the ownership structure.

Also a reminder that Rc, Arc, and Box are garbage collection. Indeed rust is a garbage collected language unless you drop to unsafe. It’s best to clarify with tracing GC which is what I think you meant.

A lot of people want a garbage collected Rust without all the complexity caused by borrow checking rules. I guess it's because Rust is genuinely a great language even if you ignore that part of it.

Not only that: Rust is considerably faster and more reliable. Since you're not writing the code yourself, Rust would be an objectively better choice.

Who are we trying to fool?