I used the Claude website and Mac desktop app for a relatively standard iOS SwiftUI project.
I used Claude Code with VS Code for some Godot stuff, and even there it sometimes gave outdated and outright made-up APIs (functions that seemed like they should exist but did not etc.)
They're terrible at anything new, including knowing about THEMSELVES and their latest versions.
This is me asking ChatGPT 5 about ChatGPT 5: https://i.imgur.com/aT8C3qs.png
Asking about Nintendo Switch 2: https://i.imgur.com/OqmB9jG.png
This could be solved and LLMs could be a lot more useful if they could be a wrapper around live web search: Just search for this shit, scrap the top few results, and summarize the info to me.
But that's a stillborn dream, crippled because Google won't let 3rd-party AIs use their search willy nilly and websites don't want to be scrapped :(
Don't get me wrong: I see the potential in AIs/LLMs and I think they could be amazing for everything, but like every awesome thing, they're hampered by corporate (and government) idiocy.
Claude Code has a neat fix for that - it knows to look at its own documentation if you ask it questions about itself: https://simonwillison.net/2025/Oct/24/claude-code-docs-map/
I've had great results from ChatGPT running the "GPT-5 Thinking" model since that almost always opts to run a search before it attempts to answer a question.
Here's what I got from that for your Switch 2 question: https://chatgpt.com/share/69089028-db8c-8006-b238-1d6946e791...
Screenshot of the searches it ran here: https://gist.github.com/simonw/048ffb895dd6b94419f0b4e066143...
A month ago when I asked Claude (on the website) about its privacy options and stuff, it always pointed me to the Antrhopic website to look it up myself.
Another annoying example: I thought Google's Gemini would be search-first since, well, they're Google.
I asked Gemini to search for Airbnb rooms in an area and give me a summarized list.
It told me it can't and I could do it myself.
I told it again.
Again it told me it can't, but here's how I could do it myself.
I told it it sucks and that ChatGPT etc. can do it for me.
Then it went and I don't know, scrapped Airbnb or used a previous search it must have had, to pull up rooms with an Airbnb link to each.
This could actually be THE absolute killer app for a lot of people, if AI could plan your trip from a single sentence: "I'm free next week. I'd like to go to A, B, or C for a couple days. What's a cheap flight and a room within this budget near X area?" and if it could also go and make a booking through your accounts it would be orgasmic. Finally we would have what people in the 1960s thought computers would be doing in 2000 :')
But as it is, in their current state you have to wade through quite a bit of dumbassery.
Unfortunately LLMs mostly suck at Swift and SwiftUI from what I've heard - they still change pretty often and as a result there aren't enough fresh examples in the training data.
As primarily a Python/JavaScript programmer I don't have that problem!