Comment by H8crilA Comment by H8crilA 2 days ago 3 replies Copy Link View on Hacker News How do you run this kind of a model at home? On a CPU on a machine that has about 1TB of RAM?
Copy Link pixelpoet 2 days ago Next Collapse Comment - Wow, it's 690GB of downloaded data, so yeah, 1TB sounds about right. Not even my two Strix Halo machines paired can do this, damn. Reply View | 0 replies
Copy Link Gracana 2 days ago Prev Next Collapse Comment - You can do it slowly with ik_llama.cpp, lots of RAM, and one good GPU. Also regular llama.cpp, but the ik fork has some enhancements that make this sort of thing more tolerable. Reply View | 0 replies
Copy Link bertili 2 days ago Prev Collapse Comment - Two 512GB Mac Studios connected with thunderbolt 5. Reply View | 0 replies
Wow, it's 690GB of downloaded data, so yeah, 1TB sounds about right. Not even my two Strix Halo machines paired can do this, damn.