Comment by skissane

I'm not sure how realistic it is to expect AIs to get detailed hands-on domain knowledge. A lot of this stuff humans learn by doing and by experience. AI models don't learn anything by doing and experience. A model vendor can't possibly encode all that experience into their training data, and even if they try, the problem is a lot of it will be vertical-specific, country/region-specific, and it is forever changing. SaaS firms have professional services and sales consulting teams who are constantly talking to customers about their actual business problems, and they feed that accumulated wisdom back to product management and data science, who in turn help engineering encode it into the product.

From what I've personally seen in SaaS AI agent development – if you try to build an AI agent to give customers advice in a particular business domain, you need to do a huge amount of work validating the answer quality with actual domain experts, and adjusting the prompts / RAG documents / tool design / etc to make sure it is giving genuinely useful advice. It is really easy to build a system which generates output which sounds superficially good, but an actual domain expert will consider wrong or worthless.

not if we don't tell it.