Comment by jasonjmcghee 2 days ago
I've used "structured output" (with supplied schema) on Google and openai, and function calling / tool use on those, anthropic and others- and afaict they are functionally the same (if you force a specific function / schema). Has someone had a different experience?
I meant between "structured output" and "function calling". Afaict one is outputting according to a schema and the other is outputting according to a schema... which will be used as the parameters to a function.
But they seem to be considered disparate concepts. So I'm trying to understand if there's some additional nuance I'm missing.
Ah ok, I misunderstood. As far as I've seen, structured outputs is essentially "json-mode" with some constraints (i.e. guided decoding over a known schema) - so the model effectively emits valid JSON that conforms to the schema. In function calling, the model is asked to emit "code" that conforms to some function parameter spec. You could use json-mode for function-calling, but probably not the other way around.
I've generally found json-mode to be more useful than function-calling, even though the latter is what everyone fixates on because of it's obvious use in agents.
I don't understand the difference based on your explanation (or the significance of "code") and have used function calling for outputting json according to a schema.
At least in the case of openai, you can set "strict" to "true" and function calling / tool use must / is enforced to follow the schema too.
The model might not use the tools every completion, depending on your setup.
They’re slightly nuanced - every model provider has a slightly different Pydantic /JSON schema compatibility (i.e for handling Literals, Unions, nested subtypes etc).
So you end up hitting roadblocks for seemingly simple Pydantic schemas.