New chip architecture brings AI from the cloud to your device
Combining memory and processing in the same layer cuts energy consumption in half when running language models on phones.
A long-awaited step in the semiconductor industry has finally left the lab. A new architecture combines memory cells and processing units in the same layer, greatly reducing the energy spent moving data within the chip.
According to the manufacturer's tests, a medium-sized language model uses about half as much energy on this chip as on the previous generation. This opens the way for offline AI features on phones and laptops.
Why does it matter?
Most AI features today run in remote data centers. On-device models reduce latency and keep personal data on the device.