Comment by jug

Comment by jug 5 days ago

17 replies

View on Hacker News

> there are some new capabilities that are big, but they are still fundamentally next-token predictors

Anthropic recently released research where they saw how when Claude attempted to compose poetry, it didn't simply predict token by token and "react" to when it thought it might need a rhyme and then looked at its context to think of something appropriate, but actually saw several tokens ahead and adjusted for where it'd likely end up, ahead of time.

Anthropic also says this adds to evidence seen elsewhere that language models seem to sometimes "plan ahead".

Please check out the section "Planning in poems" here; it's pretty interesting!

https://transformer-circuits.pub/2025/attribution-graphs/bio...

percentcer 5 days ago

Isn't this just a form of next token prediction? i.e. you'll keep your options open for a potential rhyme if you select words that have many associated rhyming pairs, and you'll further keep your options open if you focus on broad topics over niche

Reply View 15 replies
  • DennisP 5 days ago

    Assuming the task remains just generating tokens, what sort of reasoning or planning would say is the threshold, before it's no longer "just a form of next token prediction?"

    Reply View 5 replies
    • Vegenoid 5 days ago

      This is an interesting question, but it seems at least possible that as long as the fundamental operation is simply "generate tokens", that it can't go beyond being just a form of next-token prediction. I don't think people were thinking of human thought as a stream of tokens until LLMs came along. This isn't a very well-formed idea, but we may require an AI for which "generating tokens" is just one subsystem of a larger system, rather than the only form of output and interaction.

      Reply View 4 replies
      • DennisP 5 days ago

        But that means any AI that just talks to you can't be AI by definition. No matter how decisively the AI passes the Turing test, it doesn't matter. It could converse with the top expert in any field as an equal, solve any problem you ask it to solve in math or physics, write stunningly original philosophy papers, or gather evidence from a variety of sources, evaluate them, and reach defensible conclusions. It's all just generating tokens.

        Historically, a computer with these sorts of capabilities has always been considered true AI, going back to Alan Turing. Also of course including all sorts of science fiction, from recent movies like Her to older examples like Moon Is A Harsh Mistress.

        Reply View 3 replies
  • hnaccount_rng 5 days ago

    I'm not sure if this is a meaningful distinction: Fundamentally you can describe the world as a "next token predictor". Just treat the world als a simulator with a time step of some quantum of time.

    That _probably_ won't capture everything, but for all practical purposes it's non-distinguishable from reality (yes, yes, time is not some constant everywhere)

    Reply View 0 replies
  • fennecfoxy 2 days ago

    Yeah, I'd agree that for that model (certainly not AGI) it's just an extension/refinement of next token prediction.

    But when we get a big aggregated of all of these little rules and quirks and improvements and subsystems for triggering different behaviours and processes - isn't that all humans are?

    I don't think it'll happen for a long ass time, but I'm not one of those individuals who, for some reason, desperately want to believe that humans are special, that we're some magical thing that's unexplainable or can't be recreated.

    Reply View 0 replies
  • pertymcpert 5 days ago

    It doesn't really make explain it because then you'd expect lots of nonsensical lines trying to make a sentence that fits with the theme and rhymes at the same time.

    Reply View 0 replies
  • rcrsvpreordnmnt 5 days ago

    recursive predestination. LLM's algorithms imply 'self-sabotage' in order to 'learn the strings' of 'the' origin.

    Reply View 0 replies
  • throwuxiytayq 5 days ago

    In the same way that human brains are just predicting the next muscle contraction.

    Reply View 4 replies
    • fennecfoxy 2 days ago

      Potentially, but I'd say we're more reacting.

      I will feel and itch and subconsciously scratch it, especially if I'm concentrating on something. That's an subsystem independent of conscious thought.

      I suppose it does make sense - that our early evolution consisted of a bunch of small, specific background processes that enables an individual's life to continue; a single celled organism doesn't have neurons but exactly these processes - chemical reactions that keep it "alive".

      Then I imagine that some of these processes became complex enough that they needed to be represented by some form of logic, hence evolving neurons.

      Subsequently, organisms comprised of many thousands or more of such neuronal subsystems developed higher order subsystems to be able to control/trigger those subsystems based on more advanced stimuli or combinations thereof.

      And finally us. I imagine the next step, evolution found that consciousness/intelligence, an overall direction of the efforts of all of these subsystems (still not all consciously controlled) and therefore direction of an individual was much more effective; anticipation, planning and other behaviours of the highest order.

      I wouldn't be surprised if, given enough time and the right conditions, that sustained evolution would result in any or most creatures on this planet evolving a conscious brain - I suppose we were just lucky.

      Reply View 0 replies
    • alfalfasprout 5 days ago

      Except that's not how it works...

      Reply View 2 replies
childintime 5 days ago

LLM do exactly the same thing humans do: we read the text, raise flag, and flags on flags, on the various topics the text reminds us of, positive and negative, and then starts writing out a response that corresponds to those flags and likely attends to all of them. the planning ahead is just some flag that needs addressing, but it's learnt predictive behavior. nothing much to see here. experience gives you the flags. it's like applying massive pressure and diamonds will form.

Reply View 0 replies