A hardware enthusiast ran Moonshot AI's Kimi K3, a 2.8-trillion-parameter model, on a CPU using discontinued Intel Optane memory without GPU acceleration.
Swipe through 3 outlets
Homelabber Runs Massive 2.8-Trillion-Parameter AI Model On Discontinued Intel Optane Memory, Without A GPU
Why Apple's HomeView relies so heavily on your iPhone
HomeView Launch Kicks Off as First Orders Reach Customers
The experiment used a third-generation Intel Xeon Scalable processor with 40 cores and second-generation Intel Optane Persistent Memory 200 Series modules.
Performance reached 0.85 tokens per second with 1-bit quantization, improving to 1.50 tokens per second when disabling Direct Access, though accuracy can be affected.
Written with AI from the sources above and checked against them before publishing. Send a correction.
We may earn a commission when you buy through these links.
As an Amazon Associate I earn from qualifying purchases.
1 story so far. This is the latest.
No replies yet.
Sign in to reply.
Get the weekly digest
The week’s stories in the categories you pick, in your inbox. Double opt-in, one-click unsubscribe.