General purpose CPUs have been complemented with special purpose hardware accelerators since soon after they made an entrance on the consumer market.

The first PC in my childhood home had a math co-processor, and no gamer today would run a system without a dedicated graphics card with a GPU.

With the emergence of AI, the need for a new type of special purpose hardware accelerator is rising; an AI-chip specially designed to run deep neural nets.

Today, many neural nets run on GPUs, however, even though GPUs have had some enhancements towards running neural nets, more specialized chips have been designed by tech giants like Google and Microsoft, and startups focusing on this market are getting attention even before any products have been released.

Smartphones and augmented reality glasses are two cases where specialized AI chips can be extra important since the time to send images or sound to a dedicated server could take too long and take away from the augmented reality experience or make the phone user’s experience more like that of googling for answers than asking your phone in real time conversation.

AI chips also help keep the energy consumption lower (interesting both in large server farms and on your smartphone), and the computations needed for neural nets could potentially be faster with a dedicated chip.

It seems there’s even an AI-accelerator on a USB-stick if you are eager to try this out yourself (although I can’t vouch for what it will or won’t do… most likely not become self-aware and have a go at world-domination, though!)

Header image by Nick Youngson, CC BY-SA 3.0, Link