Comment by postalcoder

Comment by postalcoder 14 hours ago

I've been building agent type stuff for a couple years now and the best thing I did was build my own framework and abstractions that I know like the back of my hand.

I'd stay clear of any llm abstraction. There are so many companies with open source abstractions offering the panacea of a single interface that are crumbling under their own weight due to the sheer futility of supporting every permutation of every SDK evolution, all while the same companies try to build revenue generating businesses on top of them.

sathish316 13 hours ago

I agree with your analysis of building your own Agent framework to have some level of control and fewer abstractions. Agents at their core are about programmatically talking to an LLM and performing these basic operations: 1. Structured Input and String Interpolation in prompts 2. Structured Output and Unmarshalling String response to Structured output (This is getting easier now with LLMs supporting Structured output) 3. Tool registry/discovery (of MCP and Function tools), Tool calls and response looping 4. Composability of Tools 5. Some form of Agent to Agent delegation

I’ve had good luck with using PydanticAI which does these core operations well (due to the underlying Pydantic library), but still struggles with too many MCP servers/Tools and composability.

I’ve built an open-source Agent framework called OpusAgents, that makes the process of creating Agents, Subagents, Tools that are simpler than MCP servers without overloading the context. Check it out here and tutorials/demos to see how it’s more reliable than generic Agents with MCP servers in Cursor/ClaudeDesktop - https://github.com/sathish316/opus_agents

It’s built on top of PydanticAI and FastMCP, so that all non-core operations of Agents are accessible when I need them later.

Reply View 4 replies

drittich 12 hours ago

This sounds interesting. What about the agent behavior itself? How it decides how to come at a problem, what to show the user along the way, and how it decides when to stop? Are these things you have attempted to grapple with in your framework?

Reply View | 1 reply
- sathish316 10 hours ago
  
  The framework has the following capabilities:
  1. A way to create function tools
  2. A way to create specialised subagents that can use their own tool or their own model. The main agent can delegate to subagent exposed as a tool. Subagents don’t get confused because they have their own context window, tools and even models (mix and match Remote LLM with Local LLM if needed)
  3. Don’t use all tools of the MCP servers you’ve added. Filter out and select only the most relevant ones for the problem you’re trying to solve
  4. HigherOrderTool is a way to callMCPTool(toolName, input) in places where the Agent to MCP interface can be better suited for the problem than what’s exposed as a generic interface by the MCP provider - https://github.com/sathish316/opus_agents/blob/main/docs/GUI... . This is similar to Anthropic’s recent blogpost on Code tools being better than MCP - https://www.anthropic.com/engineering/code-execution-with-mc...
  5. MetaTool is a way to use ready made patterns like OpenAPI and not having to write a tool or add more MCP servers to solve a problem - https://github.com/sathish316/opus_agents/blob/main/docs/GUI... . This is similar to a recent HN post on Bash tools being better for context and accuracy than MCP - https://mariozechner.at/posts/2025-11-02-what-if-you-dont-ne...
  Other than AgentBuilder, CustomTool, HigherOrderTool, MetaTool, SubagentBuilder the framework does not try to control PydanticAI’s main agent behaviour. The high level approach is to use fewer, more relevant tools and let LLM orchestration and prompt tool references drive the rest. This approach has been more reliable and predictable for a given Agent based problem.
  
  Reply View | 0 replies
wizhi 5 hours ago

So you advice people to build their own framework, then advertise your own?

Reply View | 0 replies
spacecadet 12 hours ago

I also recommend this. I have tried all of the frameworks, and deploy some still for some clients- but for my personal agents, its my own custom framework that is dead simple and very easy to spin up, extend, etc.

Reply View | 0 replies

the_mitsuhiko 13 hours ago

Author here. I’m with you on the abstractions part. I dumped a lot of my though so this into a follow up post: https://lucumr.pocoo.org/2025/11/22/llm-apis/

Reply View 1 reply

thierrydamiba 13 hours ago

Excellent write up. I’ve been thinking a lot about caching and agents so this was right ilup my alley.
Have you experimented with using semantic cache on the chain of thought(what we get back from the providers anyways) and sending that to a dumb model for similar queries to “simulate” thinking?

Reply View | 0 replies

NitpickLawyer 13 hours ago

Yes, this is great advice. It also applies to interfaces. When we designed a support "chat bot", we went with a diferent architecture than what's out there already. We designed the system with "chat rooms" instead, and the frontend just dumps messages to a chatroom (with a session id). Then on the backend we can do lots of things, incrementally adding functionality, while the front end doesn't have to keep up. We can also do things like group messages, have "system" messages that other services can read, etc. It also feels more natural, as the client can type additional info while the server is working, etc.

If you have to use some of the client side SDKs, another good idea is to have a proxy where you can also add functionality without having to change the frontend.

Reply View 3 replies

postalcoder 13 hours ago

Creativity is an underrated hard part of building agents. The fun part of building right now is knowing how little of the design space for building agents has been explored.

Reply View | 1 reply
- spacecadet 12 hours ago
  
  This! I keep telling people that if tool use was not a an aha moment relative to AI agents for you, then you need to be more creative...
  
  Reply View | 0 replies
verdverm 7 hours ago

This is not so unlike the coding agent I'm building for vs code. One of the things I'm doing is keeping a snapshot of the current vs code state (files open, terminal history, etc) in the agent server. Similarly, I track the file changes without actually writing them until the user approves the diff, so there are some "filesystem" like things that need to be carefully managed on each side.
tl;dr, Both sides are broadcasting messages and listening for the ones they care about.

Reply View | 0 replies

_pdp_ 13 hours ago

This is a huge undertaking though. Yes it is quite simple to build some basic abstraction on top of openai.complete or similar but this like 1% of an agent need to do.

My bet is that agent frameworks and platform will become more like game engines. You can spin your own engine for sure and it is fun and rewarding. But AAA studios will most likely decide to use a ready to go platform with all the batteries included.

Reply View 1 reply

postalcoder 12 hours ago

In totality, yes. But you don't need every feature at once. You add to it once you hit boundaries. But I think the most important thing about this exercise is that you leave nothing to the imagination when building agents.
The non-deterministic nature of LLMs already makes the performance of agents so difficult to interpret. Building agents on top of code that you cannot mentally trace through leads to so much frustration when addressing model underperformance and failure.
It's hard to argue that after the dust settles, companies will default to batteries-included frameworks but, right now, a lot of people i've regretted adopting a large framework off the bat.

Reply View | 0 replies