Comment by CamperBob2
Comment by CamperBob2 2 days ago
That's pretty good. Are you running the real 600B+ parameter R1, or a distill, though?
Comment by CamperBob2 2 days ago
That's pretty good. Are you running the real 600B+ parameter R1, or a distill, though?
The full thing, 671B. It loses some intelligence at 1.5 bit quantisation, but it's acceptable. I could actually go for around 3 bits if I max out my RAM, but I haven't done that yet.