Comment by whimsicalism
Comment by whimsicalism 4 days ago
> believes Strawberry is mainly just CoT. I'm not saying they didn't fine tune a model too
You don't see the scaling with respect to token length with non-FT'd CoT like this, in my opinion.
Comment by whimsicalism 4 days ago
> believes Strawberry is mainly just CoT. I'm not saying they didn't fine tune a model too
You don't see the scaling with respect to token length with non-FT'd CoT like this, in my opinion.
I haven't even added Strawberry support to my app yet, and so haven't checked what it's context length is, but you're right that additional context length is a scaling factor that's totally independent of whether CoT is used or not.
I'm just saying whatever they did in their [new] model, I think they also added CoT on top of it, as the outer layer of the onion so to speak.