AI/ML and Location, Location, Location
In the last few years, huge leaps in artificial intelligence (AI) and machine learning (ML) technology have enabled the incorporation of ‘intelligence’ in a rapidly growing number of products in applications as diverse as GPT tools, IC layout, and autonomous navigation. The adoption of AI and ML has made such products more powerful, faster, accurate and easier to use.
These examples demonstrate more than just the range of applications that can be aided or enabled by AI/ML. Each also gives a use case of where AI/ML can be deployed – in the cloud, on premise, or at the edge, respectively.
Where an AI-enabled application is accessed, and where the processing actually takes place might be different, but the nature of the application dictates both. The location of AI/ML solutions has cost, performance and data security ramifications. Avnet Silica has expertise in deploying AI/ML for a wide range of applications situated and accessed in all three areas – cloud, on premise, or at the edge – and supports customers in building and configuring their AI/ML applications in the most cost-effective and appropriate way based on specific application parameters and needs.
AI/ML in the cloud
Users of AI-based GPT tools typically access them using their laptops, smartphones, or smart speakers. It is impractical, however, for the vendors of these and similar products to add AI/ML, especially when so many GPT tools already conveniently reside in the cloud. Data centers can economically and efficiently host the resources to handle thousands upon thousands of simultaneous requests. The same holds true for a growing number of other AI-enabled apps.
Communicating with distant data centers inevitably comes with time lags, but at present human users tend to find these delays negligible. In addition, the benefits of maximum processing power, accessibility and scalability in the cloud plus the resultant simplification and cost savings at the device or on the premises more than offset any downsides of remote AI processing.
Figure 1: A wide range of devices are capable of handling natural-language queries. Backing these devices with cloud-based AI/ML makes it possible to handle more complex queries and also to provide more thorough responses.
AI/ML on premise
Figure 2: Autonomous driving is an edge application that requires robust on-board AI/ML capabilities.
IC layout is emblematic of the type of AI/ML-based applications that require significant computing resources, but also involve information that is proprietary, private, or otherwise too valuable to risk exposing.
Data security can be ample justification for industrial companies, medical facilities, financial organisations and other operations with high-value data to operate private computer networks on premise, complete with the ability to run significant AI/ML workloads.
AI/ML at the edge
Autonomous vehicles exemplify the type of AI-enabled edge applications that must rely on local processing. While the lag associated with contacting a data center might be acceptable to somebody using a GPT tool to prepare an outline for a research project, that same lag is intolerable for a self-driving vehicle that must make a split-second decision regarding the safety of passengers, other drivers or pedestrians.
Running sophisticated AI-enabled applications at the edge can require substantial computational resources, and those resources are almost certain to have a hefty power budget. In edge applications where safety is paramount, such as autonomous driving, those are the costs of doing business. Those costs may be prohibitive for other edge applications, however.
Examples of AI at the Edge Solutions
AMD's Versal AI Edge Series
The AMD Versal AI Edge Series is a line of high-performance chips designed for artificial intelligence (AI) applications at the edge of networks. These chips are targeted for use in things like self-driving cars, smart factories, and medical devices.
What makes them special is their ability to handle the entire AI workflow, from receiving sensor data to making decisions in real-time, all with very low latency. They're also claimed to be energy efficient, delivering 4 times the AI performance per watt compared to leading GPUs.
The Versal AI Edge Series is built on a technology called an Adaptive SoCs (System on Chip). This means they combine multiple processing elements onto a single chip, including processors for running software, specialised AI engines for accelerating AI calculations, and programmable logic that can be customised for specific tasks.
This combination of features makes the Versal AI Edge Series a powerful and versatile platform for developing a wide range of intelligent edge devices.
NXP's MCX-N Series
NXP's MCX-N Series is a family of microcontrollers (MCUs) designed to strike a balance between high performance and low power consumption. These microcontrollers are ideal for industrial and internet of things (IoT) applications.
The MCX-N Series boasts several key features. First, it utilises dual Arm Cortex-M33 cores that can crank up to 150 MHz. This dual-core design allows for efficient multitasking and helps distribute workloads across the chip.
Secondly, select models within the MCX-N Series come equipped with NXP's eIQ® Neutron Neural Processing Unit (NPU). This NPU significantly accelerates machine learning tasks by up to 42 times compared to using CPU cores alone.
Thirdly, the MCX-N Series integrates various intelligent peripherals and accelerators. This not only improves overall system performance but also helps reduce power consumption. For example, the series includes a low-power cache and PowerQuad technology that boosts digital signal processing (DSP) voice processing by a factor of 8 or more.
Finally, the MCX-N Series offers a variety of connectivity options, including Ethernet, CAN protocols, USB, and FlexComm interfaces. This versatility allows for easy integration into various demanding industrial and IoT applications.
STMicroelectronics STM32MP2
STMicroelectronics' STM32MP2 is a series of microprocessors (MPUs) designed for industrial applications that require high performance, security, and artificial intelligence (AI) capabilities. It's essentially a powerful brain for machines in factories, smart cities, and other industrial settings.
The STM32MP2 boasts several features that make it ideal for these demanding tasks. First, it utilises a combination of processing cores: one or two high-performance Arm Cortex-A35 cores for tackling complex tasks, and a separate Arm Cortex-M33 core for handling more basic operations efficiently. This dual-core approach optimises power consumption while ensuring smooth operation.
Secondly, the STM32MP2 integrates a Neural Processing Unit (NPU) specifically designed for running AI applications. This NPU allows the chip to analyse data and make decisions directly on the device, without relying on the cloud for processing. This "edge AI" capability is crucial for real-time applications where speed and low latency are essential.
Security is another strong point of the STM32MP2. It's built with robust security features to protect sensitive data in industrial environments. Additionally, the series is designed for long-term operation, with a lifespan of up to 10 years, making it a reliable choice for industrial projects.
Overall, the STMicroelectronics STM32MP2 series offers a powerful and secure solution for developers building industrial applications that leverage AI and require high performance and real-time decision making.
Renesas RZ/V2
Renesas RZ/V2 is a family of microprocessors (MPUs) designed to bring artificial intelligence (AI) capabilities to various applications. There are two main variants within the RZ/V2 family: RZ/V2L and RZ/V2M.
RZ/V2L (General-Purpose): This is a good option for applications requiring a balance between performance and affordability. It features a dual-core Arm Cortex-A55 CPU running at 1.2 GHz, making it suitable for general-purpose tasks. The key highlight is Renesas' own AI accelerator, the DRP-AI. This built-in accelerator allows for real-time AI inference and image processing, without needing a separate chip. Additionally, the RZ/V2L includes a 3D graphics engine and a video codec engine, making it well-suited for tasks involving graphics or video processing.
RZ/V2M (Vision AI): This variant prioritizes high-performance AI for vision applications. It boasts a more powerful CPU with dual Arm Cortex-A53 cores up to 1.0 GHz. The star of the show here is the DRP-AI accelerator, reaching a performance of 1 TOPS/W (Tera Operations Per Second per Watt). This translates to efficient AI processing with low power consumption. The RZ/V2M also integrates a robust image signal processor (ISP) for handling high-resolution images from cameras. This combination makes it ideal for applications like facial recognition, object detection, and smart surveillance systems.
Overall, the Renesas RZ/V2 family offers a versatile solution for developers looking to integrate AI into their projects. Whether you need a balance between performance and cost or prioritise high-performance vision AI, there's an RZ/V2 variant to suit your needs.
The extreme edge and TinyML
AI-enabled applications are varied, but perhaps no more so than at the edge, where some use cases simply cannot justify the expense of sophisticated processing in each edge device, and where only the smallest of batteries are practical. A wide variety of embedded systems qualify, but examples include sensor networks set up to monitor phenomena like crop health, traffic, or fire.
Industry is meeting those challenges by developing ML technologies that can operate with minimal resources. An organisation called TinyML is coordinating the development of hardware, algorithms and software capable of performing on-device sensor data analytics on extremely low power budgets, making it possible to use AI/ML in an even wider range of applications.
