Artificial Intelligence and the Question of Processors
As transformative as artificial intelligence (AI) has already been in agriculture, medicine, finance, automotive, and elsewhere, there remain seemingly endless future benefits of AI. With no end in sight for the boom in the AI market, vendors of graphics processors (GPUs), general purpose processors (CPUs), neural processors (NPUs) and other specialised AI processors will be fighting each other for market share for years to come.
No single processor type is likely to emerge as a definitive winner, appropriate for all AI workloads, at least not in the foreseeable future. Each type of processor has advantages and drawbacks. As always in engineering, the tradeoffs will balance out differently for each use case. Avnet Silica can provide expert guidance as customers evaluate which processor type will be best for their specific AI workloads.
Training and GPUs
At the highest level, AI workloads are separated into two general categories: training and inference. Training is the process of analysing enormous troves of data. Inference is when a trained AI analyses new data. This dichotomy is consequential for silicon.
Training is a repetitive and iterative process. Training data is voluminous, and that data also tends to be structured, often formatted in matrices. Parallel processing, a particular strength of GPUs, is good at solving all of these problems. GPU vendor Nvidia has dominated the AI silicon market thus far.
Great effort has been expended training AIs, and that training is ongoing. One result is that the market for AI inference has been primed.
Inference and GPUs
GPUs are fully capable of running inference workloads, and as GPUs are what designers have been accustomed to using, GPUs are dominating the market in inference too. At least so far.
The natural advantages that GPUs have when it comes to training are not necessarily decisive when it comes to inference, however. Inference typically involves significantly less repetitive and iterative processing. In part, this is because inference applications typically handle unstructured data, and less of it.
Figure 2: An increasingly popular use for AI/ML is in embedded vision systems. AI’s can be built and trained to identify, categorise, evaluate and/or sort nearly anything, from people to machine parts on a factory conveyor belt to ripe fruit on a bush.
The inference segment is now growing rapidly, and that growth shows no sign of abating. CPU and NPU vendors see this as an opportunity, and are touting the advantages of their respective products.
Inference options: CPUs & NPUs
CPUs are fully capable of most AI inference workloads (though they might be slower for some, such as deep learning, for example). CPUs are less expensive than GPUs, and they also draw less power than GPUs. They tend to be more available, less expensive to acquire, and cheaper to run. CPU vendors believe these are compelling reasons to opt for CPUs rather than GPUs, especially when lower cost and lower power consumption are the priorities, and with an increasing number of workloads, they are.
The definition of NPUs is variable, depending in part on who is doing the defining. The most narrow view is that they are processors designed specifically for neural networks. Some of the processors that qualify as NPUs can be based on technologies that range from uncommon to exotic.
A wider definition includes any processor created for any type of AI workload. Under this definition, integrating standard CPU cores with peripherals optimised to support AI workloads could qualify.
Either way, NPUs are designed to run AI workloads more efficiently, faster and using less energy than GPUs and often even standard off-the-shelf CPUs.
The network center and the network edge
AI training tends to require significant investments in infrastructure, including banks of servers (usually based on GPUs), and significant storage capacity. This is why training is often conducted in data centers. Edge applications tend to run inference workloads on microprocessors or even microcontrollers.
But these are all generalities. Machine learning (ML) can be deployed just as readily in edge devices as it can in data centers or anywhere in between. Inference can be run in data centers, and increasingly often it is.
There is no one-size-fits-all when it comes to AI/ML, and as the technology progresses and silicon vendors jockey against each other, it is likely to stay true for the foreseeable future. Every engineer designing an AI/ML application will have to evaluate the tradeoffs.
Figure 2: Avnet Silica can recommend hardware, software, and algorithms that can be used to build AI/ML applications. That includes a range of developer kits from multiple vendors, such as the Nvidia Jetson Nano developer kit from Advantech.
silica content library/home/solutions/technologies/artificial-intelligence-machine-learning/ai-processors-hardware/artificial intelligence processors main body (lc)
Artificial Intelligence Processors main body (LC)
