Comment by lubesGordi

Comment by lubesGordi 3 days ago

Does the Unreal API change a bit over versions? I've noticed when asking to do a simple telnet server in Rust it was hallucinating like crazy but when I went to the documentation it was clear the api was changing a lot from version to version. I don't think they do well with API churn. That's my hypothesis anyway.

toshinoriyagi 3 days ago

I think the big thing with Unreal is the vast majority of games are closed source. It's already only used for games, as opposed to asking questions about general-purpose programming, but there is also less training data.

Reply View 0 replies

mac-mc 2 days ago

You see this dynamic even with swift which has a corpus of OSS source code out there, but not nearly as much as js or python and so has always been behind those languages.

Reply View 0 replies

Tiktaalik 3 days ago

There would be significant changes from 4 to 5, but sadly I haven’t had any improvement if clarifying version.

Reply View 1 reply

frankc 2 days ago

Clarifying can help but ultimately it was trained on older versions. When you are working with a changing api, it's really important that the llm can see examples of the new api and new api docs. Adding context7 as a tool is hugely helpful here. Include in your rules or prompt to consult context7 for docs. https://github.com/upstash/context7

Reply View | 0 replies