Comment by chr15m

Comment by chr15m 3 days ago

I'm not sure if this is widely known but you can do a lot better even than AGENTS.md.

Create a folder called .context and symlink anything in there that is relevant to the project. For example READMEs and important docs from dependencies you're using. Then configure your tool to always read .context into context, just like it does for AGENTS.md.

This ensures the LLM has all the information it needs right in context from the get go. Much better performance, cheaper, and less mistakes.

gbnwl 3 days ago

Cheaper? Loading every bit of documentation into context every time, regardless of whether it’s relevant to the task the agent is working on? How? I’d much rather call out the location of relevant docs in Claude.md or Agents.md and tell the agent to read them only when needed.

Reply View 3 replies

chr15m 2 days ago

As they point out in the article, that approach is fragile.
Cheaper because it has the right context from the start instead of faffing about trying to find it, which uses tokens and ironically bloats context.
It doesn't have to be every bit of documentation, but putting the most salient bits in context makes LLMs perform much more efficiently and accurately in my experience. You can also use the trick of asking an LLM to extract the most useful parts from the documentation into a file, which you then re-use across projects.
https://github.com/chr15m/ai-context

Reply View | 2 replies
- gbnwl 2 days ago
  
  > Extracting the most useful parts of documentation into a file
  Yes, and this file becomes: also documentation. I didn’t mean throw entire unabridged docs at it, I should’ve been more clear. All of my docs for agents are written by agents themselves. Either way once the project becomes sufficiently complex it’s just not going to be feasible to add a useful level of detail of every part of it into context by default, the context window will remain fixed as your project grows. You will have to deal with this limit eventually.
  I DO include a broad overview of the project in Agents or Claude.md by default, but have supplemental docs I point the agent to when they’re working on a particular aspect of the project.
  
  Reply View | 1 reply
  
  chr15m 2 days ago
  
  > sufficiently complex
  Sounds like we are working on different types of projects. I avoid complexity at almost all cost and ruthlessly minimise LoC and infrastructure. I realise that's a privilege, and many programmers can't.
  
  Reply View | 0 replies

d3m0t3p 3 days ago

Yea but the goal it not to bloat the context space. Here you "waste" context by providing non usefull information. What they did instead is put an index of the documentation into the context, then the LLM can fetch the documentation. This is the same idea that skills but it apparently works better without the agentic part of the skills. Furthermore instead of having a nice index pointing to the doc, They compressed it.

Reply View 5 replies

chr15m 2 days ago

The minification is a great idea. Will try this.
Their approach is still agentic in the sense that the LLM must make a tool cool to load the particular doc in. The most efficient approach would be to know ahead of time which parts of the doc will be needed, and then give the LLM a compressed version of those docs specifically. That doesn't require an agentic tool call.
Of course, it's a tradeoff.

Reply View | 0 replies
bmitc 2 days ago

What does it mean to waste context?

Reply View | 3 replies
- therealpygon 2 days ago
  
  Context quite literally degrades performance of attention with size in non-needle-in-haystack lookups in almost every model to varying degrees. Thus to answer the question, the “waste” is making the model dumber unnecessarily in an attempt to make it smarter.
  
  Reply View | 0 replies
- bagels 2 days ago
  
  The context window is finite. You can easily fill it with documentation and have no room left for the code and question you want to work on. It also means more tokens sent with every request, increasing cost if you're paying by the token.
  
  Reply View | 0 replies
- PKop 2 days ago
  
  Think of context switching when you yourself are programming. You can only hold some finite amount of concepts in your head at one time. If you have distractions, or try to focus on too many things at once, your ability to reason about your immediate problem degrades. Think also of legacy search engines: often, a more limited and focused search query vs a query that has too many terms, more precisely maps to your intended goal.
  LLM's have always been at any time limited in the amount of tokens it can process at one time. This is increasing, but one problem is chat threads continually increase in size as you send messages back and forth because within any session or thread you are sending the full conversation to the LLM every message (aside from particular optimizations that compact or prune this). This also increases costs which are charged per token. Efficiency of cost and performance/precision/accuracy dictates using the context window judiciously.
  
  Reply View | 0 replies

epolanski 2 days ago

Docs of dependencies aren't that much of a game changer. Multiple frameworks and libraries have been releasing llm.txt compressed versions of their docs from ages, and it doesn't make that much of a difference (I mean, it does, but not crucial as LLMs can find the docs on their own even online if needed).

What's actually useful is to put the source code of your dependencies in the project.

I have a `_vendor` dir at the root, and inside it I put multiple git subtrees for the major dependencies and download the source code for the tag you're using.

That way the LLM has access to the source code and the tests, which is way more valuable than docs because the LLM can figure out how stuff works exactly by digging into it.

Reply View 0 replies

TeeWEE 2 days ago

This is quite a bad idea. You need to control the size and quality of your context by giving it one file that is optimized.

You don’t want to be burning tokens and large files will give diminishing returns as is mentioned in the Claude Code blog.

Reply View 1 reply

chr15m 2 days ago

It is not an "idea" but something I've been doing for months and it works very well. YMMV. Yes, you should avoid large files and control the size and quality of your context.

Reply View | 0 replies