Comment by 76SlashDolphin

Comment by 76SlashDolphin 11 hours ago

At least in its current state we just use an LLM to categorise each individual tool. We don't look at the data itself, although we have some ideas of how to improve things, as currently it is very "over-defensive". For example, if you have the filesystem MCP and a web search MCP, open-edison will block if you perform a filesystem read, a web search, and then a filesystem write. Still, if you rarely perform writes open-edison would still be useful for tracking things. The UX is such that after an initial block you can make an exception for the same flow the next time it occurs.

noddingham 10 hours ago

Thanks for the follow up. I can see the value in trying to look at the chained read - search - write or similar patterns to alert the user. Awareness of tool activity is definitely helpful.

Reply View 0 replies

daveguy 11 hours ago

Well, I guess 80-90% protective is better than nothing. Better might be a lock that requires positive confirmation by the user.

Reply View 1 reply

76SlashDolphin 10 hours ago

It is possible to configure it like that - when a trifecta is detected, it is possible for the gateway to wait for confirmation before allowing the last MCP call to proceed. The issue with that MCP clients are still in early stages and some of them don't like waiting for a long time until they get a response and act in weird or inconvenient ways if something times out (some of them sensibly disable the entire server if a single tool times out, which in our case disables the entire gateway and therefore all MCP tools). As it is, it's much better to default to returning a block message, and emit a web notification from the gateway dashboard to get the user to approve the usecase, then rerun their previous prompt.

Reply View | 0 replies