Comment by sjsdaiuasgdia

You still seem to be expecting some degree of thought and understanding from these tools.

They generate tokens. The output has a probabilistic relationship to the established context and prompts, plus whatever prompting is happening as you interact with the model.

There is no understanding of "don't do [thing]". Sometimes, you can get something closer to what you wanted by putting stuff like that in the prompt. But it's still probabilistic token generation. It's not interpreting that as a literal command to not do the thing. It has that command in its context now, and maybe that changes the output. Maybe it changes in a useful direction, maybe it doesn't. But it's not going to be treated as a literal command because the model does not have the capability to do so. Phrasing it differently doesn't change the fundamentals.