Comment by jasonjmcghee

Comment by jasonjmcghee 2 days ago

I meant between "structured output" and "function calling". Afaict one is outputting according to a schema and the other is outputting according to a schema... which will be used as the parameters to a function.

But they seem to be considered disparate concepts. So I'm trying to understand if there's some additional nuance I'm missing.

fzysingularity 2 days ago

Ah ok, I misunderstood. As far as I've seen, structured outputs is essentially "json-mode" with some constraints (i.e. guided decoding over a known schema) - so the model effectively emits valid JSON that conforms to the schema. In function calling, the model is asked to emit "code" that conforms to some function parameter spec. You could use json-mode for function-calling, but probably not the other way around.

I've generally found json-mode to be more useful than function-calling, even though the latter is what everyone fixates on because of it's obvious use in agents.

Reply View 1 reply

jasonjmcghee 2 days ago

I don't understand the difference based on your explanation (or the significance of "code") and have used function calling for outputting json according to a schema.

Reply View | 0 replies

guntars 2 days ago

With function calls the model may or may not output something that matches the schema, with structured output the schema is enforced at the logit level.

Reply View 1 reply

jasonjmcghee 2 days ago

At least in the case of openai, you can set "strict" to "true" and function calling / tool use must / is enforced to follow the schema too.

Reply View | 0 replies