Comment by simonw

Comment by simonw a day ago

5 replies

Was this Claude Code or Claude.ai or some other tool that used Claude under the hood?

Different harnesses have different search capabilities.

If I'm doing something that benefits from search I tend to switch to ChatGPT because I know it has a really good search feature available to it. I don't trust Claude's as much.

Razengan a day ago

I used the Claude website and Mac desktop app for a relatively standard iOS SwiftUI project.

I used Claude Code with VS Code for some Godot stuff, and even there it sometimes gave outdated and outright made-up APIs (functions that seemed like they should exist but did not etc.)

  • simonw a day ago

    Unfortunately LLMs mostly suck at Swift and SwiftUI from what I've heard - they still change pretty often and as a result there aren't enough fresh examples in the training data.

    As primarily a Python/JavaScript programmer I don't have that problem!

    • Razengan 10 hours ago

      They're terrible at anything new, including knowing about THEMSELVES and their latest versions.

      This is me asking ChatGPT 5 about ChatGPT 5: https://i.imgur.com/aT8C3qs.png

      Asking about Nintendo Switch 2: https://i.imgur.com/OqmB9jG.png

      This could be solved and LLMs could be a lot more useful if they could be a wrapper around live web search: Just search for this shit, scrap the top few results, and summarize the info to me.

      But that's a stillborn dream, crippled because Google won't let 3rd-party AIs use their search willy nilly and websites don't want to be scrapped :(

      Don't get me wrong: I see the potential in AIs/LLMs and I think they could be amazing for everything, but like every awesome thing, they're hampered by corporate (and government) idiocy.

      • simonw 9 hours ago

        Claude Code has a neat fix for that - it knows to look at its own documentation if you ask it questions about itself: https://simonwillison.net/2025/Oct/24/claude-code-docs-map/

        I've had great results from ChatGPT running the "GPT-5 Thinking" model since that almost always opts to run a search before it attempts to answer a question.

        Here's what I got from that for your Switch 2 question: https://chatgpt.com/share/69089028-db8c-8006-b238-1d6946e791...

        Screenshot of the searches it ran here: https://gist.github.com/simonw/048ffb895dd6b94419f0b4e066143...

        • Razengan 9 hours ago

          A month ago when I asked Claude (on the website) about its privacy options and stuff, it always pointed me to the Antrhopic website to look it up myself.

          Another annoying example: I thought Google's Gemini would be search-first since, well, they're Google.

          I asked Gemini to search for Airbnb rooms in an area and give me a summarized list.

          It told me it can't and I could do it myself.

          I told it again.

          Again it told me it can't, but here's how I could do it myself.

          I told it it sucks and that ChatGPT etc. can do it for me.

          Then it went and I don't know, scrapped Airbnb or used a previous search it must have had, to pull up rooms with an Airbnb link to each.

          This could actually be THE absolute killer app for a lot of people, if AI could plan your trip from a single sentence: "I'm free next week. I'd like to go to A, B, or C for a couple days. What's a cheap flight and a room within this budget near X area?" and if it could also go and make a booking through your accounts it would be orgasmic. Finally we would have what people in the 1960s thought computers would be doing in 2000 :')

          But as it is, in their current state you have to wade through quite a bit of dumbassery.