Comment by rokkamokka
Comment by rokkamokka 6 hours ago
For LLMs I recommend just doing NDJSON, that is, newline delimited json. It's much simpler to implement
Comment by rokkamokka 6 hours ago
For LLMs I recommend just doing NDJSON, that is, newline delimited json. It's much simpler to implement
Do any LLMs support constrained generation of newline delimited json? Or have you found that they're generally reliable enough that you don't need to do constrained sampling?