Comment by saaaaaam

> When you say BYOK tool do you mean effectively a GUI front end on the API?

Less that, and more focused tools like e.g. Aider (OSS Cursor from before Cursor was a thing).

I use TypingMind almost exclusively for any and all LLM chatting, and I do maintain a bunch of Claude-optimized prompts that specifically exploit the "XML tags" feature (some of them I also run through the Anthropic's prompt improver) -- but I don't expect the generic frontends to care about vendor-specific prompting tricks by default. Here, my only complaint is that I don't have control over how it injects attachments, and inlined text attachments in particular are something Anthropic docs recommend demarking with XML tags, which TypingMind almost certainly doesn't do. I'd also love for the UI to recognize XML tags in output and perhaps offer some structuring or folding on the UI side, e.g. to auto-collapse specified tags, such as "<thinking>" or "<therapeuticAnalysis>" or whatever I told the LLM to use.

(Oh, and another thing: Anthropic recently introduced a better form of PDF upload, in which the Anthropic side handles simultaneously OCR-ing and imaging the PDF and feeding it to the model, to exploit its multimodal capabilities. TypingMind, as far as I can tell, still can't take advantage of it, despite it boiling down to an explicit if/else on the model vendor.)

No, I first and foremost mean the more focused tools, that generalize across LLMs. Taking Aider as an example, as far as I can tell, it doesn't have any special handling for Anthropic, meaning it doesn't use XML tags to mark up the repo map structure, or demarcate file content or code snippets it says, or to let the LLM demarcate diffs in reply, etc. It does its own model-agnostic thing, which means that using Claude 3.5 Sonnet, I lose out on model performance boost it's not taking advantage of.

I singled out Aider, but there's plenty of tools and plugins out there that utilize some common LLM portability libraries, and end up treating every LLM the same way. The LLM portability libraries however are not the place to solve it - by their nature, they target the lowest common denominator. Those specialized tools should be doing it IMO, and it's not even much work - it's a bunch of model-based if/elses. Might not look pretty, but it's not a maintenance burden.