Comment by pjmlp

Comment by pjmlp 2 months ago

32 replies

View on Hacker News

I think people are still fooling themselves about the relevance of 3GL languages in an AI dominated future.

It is similar to how Assembly developers thought about their relevance until optimising compilers backends turned that into a niche activity.

It is a matter of time, maybe a decade who knows, until we can produce executables directly from AI systems.

Most likely we will still need some kind of formalisation tools to tame natural language uncertainties, however most certainly they won't be Python/Rust like.

We are moving into another abstraction layer, closer to the 4GL, CASE tooling dreams.

mpweiher 2 months ago

"Since FORTRAN should virtually eliminate coding and debugging…" -- FORTRAN report, 1954 [1]

If, as you seem to imply and as others have stated, we should no longer even look at the "generated" code, then the LLM prompts are the programs / programming language.

I can't think of a worse programming language, and I am not the only one [2]

However, it does indicate that our current programming languages are way to low-level, too verbose. Maybe we should fix that?

[1] http://www.softwarepreservation.org/projects/FORTRAN/BackusE...

[2] https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...

[3] https://objective.st/

Reply View 0 replies
albertzeyer 2 months ago

4GL and 5GL are already taken. So this is the 6GL.

https://en.wikipedia.org/wiki/Programming_language_generatio...

But speaking more seriously, how to get this deterministic?

Reply View 2 replies
  • pjmlp 2 months ago

    Fair enough, should have taken a look, I stopped counting when computer magazines buzz about 4GLs faded away.

    Probably some kind of formal methods inspired approach, declarative maybe, and less imperative coding.

    We should take an Alan Kay and Bret Victor like point of view where AI based programming is going to be in a decade from now, not where it is today.

    Reply View 1 reply
    • codr7 2 months ago

      That future is far from inevitable, the first question we SHOULD ask is if it's a good idea to go down this path.

      Reply View 0 replies
dragonwriter 2 months ago

> I think people are still fooling themselves about the relevance of 3GL languages in an AI dominated future.

I think, as happens in the AI summer before each AI winter, people are fooling themselves about both the shape and proximity of the “AI dominated future”.

Reply View 1 reply
  • brookst 2 months ago

    It will be approximately the same shape and proximity as “the Internet-dominated future” was in 2005.

    Reply View 0 replies
Wowfunhappy 2 months ago

Assemblers and compilers are (practically) deterministic. LLMs are not.

Reply View 18 replies
  • chowells 2 months ago

    That's the wrong distinction, and bringing it up causes pointless arguments like are in the replies.

    The right distinction is that assemblers and compilers have semantics and an idea of correctness. If your input doesn't lead to a correct program, you can find the problem. You can examine the input and determine whether it is correct. If the input is wrong, it's theoretically possible to find the problem and fix it without ever running the assembler/compiler.

    Can you examine a prompt for an LLM and determine whether it's right or wrong without running it through the model? The idea is ludicrous. Prompts cannot be source code. LLMs are fundamentally different from programs that convert source code into machine code.

    This is something like "deterministic" in the colloquial sense, but not at all in the technical sense. And that's where these arguments come from. I think it's better to sidestep them and focus on the important part: compilers and assemblers are intended to be predictable in terms of semantics of code. And when they aren't, it's a compiler bug that needs to be fixed, not an input that you should try rephrasing. LLMs are not intended to be predictable at all.

    So focus on predictability, not determinism. It might forestall some of these arguments that get lost in the weeds and miss the point entirely.

    Reply View 0 replies
  • traverseda 2 months ago

    LLMs are deterministic. So far every vendor is giving them random noise in addition to your prompt though. They don't like have a free will or a soul or anything, you feed them exactly the same tokens exactly the same tokens will come out.

    Reply View 4 replies
    • mmoskal 2 months ago

      If you change one letter in the prompt, however insignificant you may think it is, it will change the results in unpredictable ways, even with temperature 0 etc. The same is not true of renaming a variable in a programming language, most refactorings etc.

      Reply View 0 replies
    • jnwatson 2 months ago

      Only if you set temperature to 0 or have some way to set the random seed.

      Reply View 1 reply
      • vlovich123 2 months ago

        Locally that’s possible but for multi tenant ones I think there’s other challenges related to batch processing (not in terms of the random seed necessarily but because of other non determinism sources).

        Reply View 0 replies
    • codr7 2 months ago

      That's not how they are being used though, is it?

      Reply View 0 replies
  • pjmlp 2 months ago

    Missed the part?

    > Most likely we will still need some kind of formalisation tools to tame natural language uncertainties, however most certainly they won't be Python/Rust like

    Reply View 11 replies
    • Wowfunhappy 2 months ago

      No, I didn't miss it. I think the fact that LLMs are non deterministic means we'll need a lot more than "some kind of formalization tools", we'll need real programming languages for some applications!

      Reply View 10 replies
sitkack 2 months ago

> It is a matter of time, maybe a decade who knows, until we can produce executables directly from AI systems.

They already can.

Reply View 1 reply
  • codr7 2 months ago

    If you're lucky.

    If you're even luckier it will perform the desired function fast enough.

    Why not spend the money on lottery tickets instead?

    Reply View 0 replies
zenkey 2 months ago

Yes I agree this is likely the direction we're heading. I suppose the "Python 4" I mentioned would just be an intermediate step along the way.

Reply View 1 reply
  • sanderjd 2 months ago

    I think the question is: What is the value of that intermediate step? It depends on how long the full path takes.

    If we're one year away from realizing a brave new world where everyone is going straight from natural language to machine code or something similar, then any work to make a "python 4" - or any other new programming languages / versions / features - is rearranging deck chairs on the Titanic. But if that's 50 years away, then it's the opposite.

    It's hard to know what to work on without being able to predict the future :)

    Reply View 0 replies
krembo 2 months ago

Wild thought: maybe coding is a thing of the past? Given that an llm can get fast&deterministic results if needed, maybe a backend for instance, can be a set of functions which are all textual specifications and by following them it can do actions (validations, calculations, etc), approach apis and connect to databases, then produce output? Then the llm can auto refine the specifications to avoid bugs and roll the changes in real time for the next calls? Like a brain which doesn't need predefined coding instructions to fulfill a task, but just understand its scope, how to approach it and learn from the past.

Reply View 2 replies
  • TechDebtDevin 2 months ago

    I really want to meet these people that are letting an LLM touch their db.

    Reply View 1 reply
    • krembo 2 months ago

      Fast forward to the near future, why wouldn't it with the correct restrictions? For instance, would you let it today run SELECT queries? as Hemingway once said "if it's about price we know who you are".

      Reply View 0 replies