Honestly, i have been bitten so many times by LLM hallucinations when I work in parallel with the LLM, I wouldn't trust it autonomously running anything at all. If you have tried to use imaginary APIs, imaginary configuration and imaginary cli arguments, you know what I mean
I'm using it to orient me when tackling something new. For instance, the other day i was making a web driver client in shell and i asked it something apong the lines of "is there an http endpoint to the webdriver to get the class name of an element?"
These are the sort of questions i mostly do. "What is the best practice to read output from a device file in C", "Is there a cli tool to find dead typescript interface fields?"
I mainly use it as a typing assist. If it suggests ahead what I was thinking it saves time.
> If you have tried to use imaginary APIs, imaginary configuration and imaginary cli arguments, you know what I mean
I see this comment a lot but I can't help but feel it's 4 weeks out of date. The version of o1 released on 2024-12-17 so rarely hallucinates when asked code questions of basic to medium difficulty and provided with good context and a well written prompt, in my experience. If the context window is sub-10k tokens, I have very high confidence that the output will be correct. GPT-4o and o1-mini, on the other hand, hallucinates a lot and I have learned to put low trust in the output.