Why AI must redefine processor architectures
Neural networks are redefining microprocessor architectures and instruction sets.
The demand for AI running on programmable platforms is a unifying application. Microprocessors and microcontrollers all share similarities, but the applications they target tend to vary. AI is different.
General purpose is essential to the industry. It offers economies of scale in manufacturing. Common architectures enable a common toolchain, which makes software development manageable. Similar is good, it creates competition and innovation. These are positives for customers in a global industry.
The concept of general purpose starts failing when talking about AI and neural networks. What the industry has today is a choice from highly efficient and very powerful general-purpose architectures. What startups believe the industry need are AI architectures that are application-specific and can scale from hundredths of a watt to many hundreds of watts.
Application-specific is not new. Manufacturers of high-volume electronic devices still employ application-specific ICs (ASICs). ASIC design is accessible thanks to advanced electronic design automation (EDA) tools and semiconductor foundries that can manufacture the ASICs. The non-recurring engineering (NRE) costs and commitment to high production numbers are the main financial barriers to using ASICs. Engineering skills to use the EDA tools are the technical barrier.
General-purpose devices provide the alternative to ASICs. Many semiconductor vendors will have families of devices that have a common core but offer a variety of peripherals. This creates a pathway from general-purpose to application-specific.
Some semiconductor manufacturers are now working on developing AI accelerators that fit this model. The accelerators integrate alongside the core, providing a boost in performance without sacrificing the benefits of a general-purpose family.
Most startups in the application area are proposing new architectures, designed to meet the needs of neural network processing.
Embedded designers are looking for more efficient ways to implement AI at the board level. New architectures target accelerating the complex math behind AI training and running machine learning.
Why do we need AI acceleration?
The demand for AI acceleration stems from the type of mathematical calculations needed to execute neural networks (NNs). A conventional scalar processor works on one instruction and one piece of data at a time. A superscalar uses parallelism to handle more than one instruction, while a multicore superscalar architecture can handle multiple instructions and multiple data (known as MIMD) in the same instruction cycle.
Parallel architectures work here, particularly in training. Graphics processors are highly parallel and a preferred option in training. They can be considered vector processors, operating on one-dimensional arrays. A trained neural network model is a multidimensional array. This puts higher demands on the processor’s ability to handle arrays. Matrix processors, like Google’s TPU (Tensor Processing Unit) are designed to accelerate matrix manipulation.
AI accelerators may surprise
The Arm® Cortex® family of cores still dominate the microcontroller application area. The Arm Cortex-M85 features Arm’s Helium M-Profile vector extensions. These extensions are designed to accelerate vector processing, exactly the kind of operations used by machine learning models at the edge.
Other technologies are less conventional. One approach gaining momentum is in-memory processing. This technology turns a memory array into a neural network by connecting the cells in the memory using variable resistors. The resistance acts as the weight between neural connections.
There are examples of microcontrollers using these technologies either available now or coming to market soon. Avnet is already working with customers developing AI and machine learning applications and is well positioned to offer advice and design support.
Startups targeting AI
Many startups believe accelerating AI computation requires matrix processors with new kinds of architectures. This is where they are offering differentiation.
One such company is UK-based Red Semiconductor. It is two to three years away from offering its first commercial product.
Red Semiconductor is the commercial realization of an open-source hardware project called Libre-SOC. The person behind Libre-SOC is Luke Leighton. Red Semiconductor’s CEO is James Lewis. Both have a lot of experience in the semiconductor industry. Lewis was co-founder of UK startup Oxford Semiconductor.
Lewis recognizes that general-purpose processors are good at most things but are compromised by this flexibility. Applying the right technology for a specific task, such as a GPU with a CPU and some hardware acceleration, results in a disconnected solution.
“Across your team of designers, you have different languages and different architectures, and you need different expertise to cover this,” Lewis said. “Bringing this together is a challenge, so the question was: ‘Is there an architecture that allows all of these functions to be carried out in a single architecture?’”
Lewis feels we are at a point analogous to moving from a complex instruction set (CISC) to a reduced instruction set (RISC). The solution that Red Semiconductor is putting forward is the VISC, which stands for Vector Instruction Set Computing. Based on Libre-SOC, Red Semiconductor’s VISC is aimed at executing matrix mathematics using a vectorized instruction set.
James Lewis, CEO, Red Semiconductor, believes his company’s VISC architecture will tackle the major challenges with AI.
It is interesting that the open-source core project of Libre-SOC is still open. Anyone can use the vectorisation concept for the basis of a commercial processor. This is similar to the way RISC-V is now being used in commercial companies.
Lewis explained that VISC has the capability to take a scalar instruction set, even that of RISC-V, and enable vectorisation of any or all the instructions as required for code efficiency. A VISC vectorized instruction is a loop system that executes a complex function a predetermined number of times. Lewis says that all the processing for that loop happens automatically in the VISC smart hardware and register set, eliminating or minimising reading or writing to off-chip memory, which incurs latency and power consumption penalties.
This avoidance of repetitive memory access delivers gains in speed and power. In many cases, memory access is the bottleneck for matrix calculations on traditional architectures. Real-time execution for time-critical AI tasks is one area where Lewis is aiming the VISC processor.
Conclusion
Executing AI algorithms and neural networks is highly parallel. Most legacy processor architectures are not optimized for matrix calculations. The industry is taking several approaches to addressing the performance requirements for AI and machine learning.
Hardware accelerators and vector extensions support legacy architectures and so offer a low-friction path to faster AI. Innovations like in-memory processing and new instruction sets are more disruptive but have the potential to bring massive gains.
As a technology supplier, Avnet is constantly monitoring these and other developments so we can support our customers. AI is here and is still evolving.
About Author
Philip Ling
Philip Ling is a senior technology writer with Avnet. He holds a post-graduate diploma in Advanced M...
silica content library/articles/why-ai-must-redefine-processor-architectures
Why AI must redefine processor architectures | Avnet Silica
