Comment by orderone_ai
Comment by orderone_ai 3 days ago
Man, that is truly fascinating. Do you have ideas on how to expand the study to capture broader analysis like that...?
Comment by orderone_ai 3 days ago
Man, that is truly fascinating. Do you have ideas on how to expand the study to capture broader analysis like that...?
I was trying to solve AGI at the time this was just a side study I did to better understand how models forget the effect was not what I was looking for.
It could be expanded to better understand alignment.
But the resolution makes that cost prohibitive.
I did ~100 runs on different sizes but inferencing 100s of thousands of times made it computationally prohibitive. The key random statement is what allowed accurate measurements of the model.
The equivalent would be for every fine tuning data you train on run the entire evaluation dataset through it.