EXO Labs has demonstrated that modern artificial intelligence can operate on much older hardware than commonly believed. Their team, led by Andrej Karpathy, succeeded in running a contemporary AI model on a Pentium II computer from 1997 equipped with just 128 MB of RAM.
The project began by acquiring a Windows 98 machine with a Pentium II processor for £118.88 on eBay. Despite its outdated specifications, the team confronted several challenges. The computer lacked USB ports, forcing the use of PS/2 peripherals that required the mouse to be plugged into port 1 and the keyboard into port 2 to function properly.
File transfer posed another obstacle due to incompatibility with modern USB drives. The solution was to establish an FTP connection between the Windows 98 machine and a MacBook Pro over Ethernet, facilitated by a USB-C to Ethernet adapter.
Compiling the AI code on the old system was difficult since the Pentium II processor did not support some modern instructions. After trying mingw unsuccessfully, the team used Borland C++ 5.02, a 26-year-old integrated development environment that still works on Windows 98.
Adjustments to the code, including variable type changes and simplified memory handling, enabled them to run the Llama 2 AI model. The 260K parameter version processed 39.31 tokens per second, while the larger 15M parameter model processed 1.03 tokens per second. Although slower than current high-end systems, running any modern AI model on this vintage hardware is remarkable.
The success stemmed from the simplicity of the BitNet architecture, which uses ternary weights (-1, 0, 1) to reduce computational demands. BitNet models require significantly less memory and power, with a 7B parameter version needing only 1.38 GB of storage, making it feasible to run on older machines.
EXO Labs envisions AI development focused on energy efficiency and accessibility, targeting older laptops, smartphones, and gaming consoles. By prioritizing CPU-based processing over GPUs, BitNet could democratize AI, making it available beyond modern, high-powered devices.