mempko 19 hours ago

Great question! Don't use AI to process the data, especially when a computer can do the work :-). AI is good at taking unstructured data and structuring it. Computers are great at computing.

Here is an example of Google's AI failing

https://www.google.com/search?q=is+2026+next+year

Google screenshot: https://imgur.com/a/FOT4aDF

ChatGPT also fails: https://imgur.com/a/mb3rRgZ

and here is the ThetaEdge result: https://imgur.com/a/ZAZZgiR

  • IncreasePosts 16 hours ago

    Sure - I guess what I was asking is how to make sure everything is okay in the unstructured -> structured conversion.

    "My name is John and I'm 40 years old" -> {name:"John", age:40}

    How can you gain confidence that the AI doesn't spit out {name:"John", age:41}

    The only thing I do currently is have a massive test suite to gain some statistical confidence it works, but I worry about situations like a person having a rare unicode character in their name (not to even speak of people intentionally trying to trick the system)

    • mempko 14 hours ago

      Don't have the AI do the data parsing. Have the AI write a parser and have the parser do the parsing. Think about how a person would parse vasts amounts of data. They write a parser to do it. Devil is of course in the details.