Comment by CamperBob2 3 days ago
The models, not the context. When it comes to weights, "quantity has a quality all its own" doesn't even begin to describe what happens.
Once you hit a billion or so parameters, rocks suddenly start to think.
That has traditionally been the problem with these types of models, but Genie is supposed to maintain coherence up to 60 seconds.
I've tried using it a couple of times, but can't get in. It is either down or hopelessly underprovisioned by Google. Do you have any links to videos showing that the quality degrades after only a few seconds?
Edit: no, it just doesn't work in Firefox. It works incredibly well, at least in Chrome, and it does not lose coherence to any great extent. The controls are terrible, though.
We're talking about context here though. The first couple seconds of Genie are great, but over time it degrades. It will always degrade, because it's hallucinating a world and needs to keep track of too many things.