https://pressbooks.openedmb.ca/wordandsentencestructures/cha...
I believe what the LLM replies with is in fact correct. From the standpoint of a programmer or any other category of people that are attuned to some kind of formal rigor? Absolutely not. But for any other kind of user who is more interested in the first two concepts instead, this is the thing to do.
Indeed, but consider this situation: You have a collection of documents and want to extract the first n words because you're interested in the semantic content of the beginning of each doc. You use a LLM because why not. The LLM processes the documents, and every now and then it returns a slightly longer or shorter list of words because it better captures the semantic content. I'd argue the LLM is in fact doing exactly the right thing.
Let me hammer that nail deeper: your boss asks you to establish the first words of each document because he needs this info in order to run a marketing campaign. If you get back to him with a google sheet document where the cells read like "We the" or "It is", he'll probably exclaim "this wasn't what I was asking for, obviously I need the first few words with actual semantic content, not glue words. And you may rail against your boss internally.
Now imagine you're consulting with a client prior to developing a digital platform to run marketing campaigns. If you take his words literally, he will certainly be disappointed by the result and arguing about the strict formal definition of "2 words" won't make him deviate from what he has to say.
LLMs have to navigate through pragmatics too because we make abundant use of it.
No, I am quite sure that if you asked a random person on the street how many words are in “We the people”, they would say three.