Comment by Fr0styMatt88
Comment by Fr0styMatt88 2 days ago
I primarily work in C# during the day but have been messing around with simple Android TV dev on occasion at night.
I’ve been blown away sometimes at what Copilot puts out in the context of C#, but using ChatGPT (paid) to get me started on an Android app - totally different experience.
Stuff like giving me code that’s using a mix of different APIs and sometimes just totally non-existent methods.
With Copilot I find sometimes it’s brilliant but it’s so random as to when that will be it seems.
> Stuff like giving me code that’s using a mix of different APIs and sometimes just totally non-existent methods.
That has been my experience as well. We can control the surprising pick of APIs with basic prompt files that clarify what and how to use in your project. However, when using less-than-popular tools whose source code is not available, the hallucinations are unbearable and a complete waste of time.
The lesson to be learned is that LLMs depend heavily on their training set, and in a simplistic way they at best only interpolate between the data they were fed. If a LLM is not trained with a corpus covering a specific domain them you can't expect usable results from it.
This brings up some unintended consequences. Companies like Microsoft will be able to create incentives to use their tech stack by training their LLMs with a very thorough and complete corpus on how to use their technologies. If Copilot does miracles outputting .NET whereas Java is unusable, developers have one more reason to adopt .NET to lower their cost of delivering and maintaining software.