Why “Reliability” is the Real KPI in Edge AI: Moving Beyond TOPS, FPS, and Benchmark Hype
For years, the edge AI conversation has been shaped by performance numbers. Trillion operations per second (TOPS), floating-point operations per second (FLOPS), frames per second (FPS), model size, and benchmark scores have all become familiar reference points for comparing AI hardware. While these metrics matter, they do not tell the full story.
At the edge, AI systems are not operating in controlled lab conditions. They are installed in factories, warehouses, smart city infrastructure, surveillance systems, autonomous mobile robots (AMRs), kiosks, and embedded industrial devices. They may be exposed to dust, vibration, variable temperatures, unstable connectivity, limited power, and strict maintenance windows.
In these environments, peak performance is only useful if it can be sustained. Many edge AI systems are expected to run continuously, often 24 hours a day, seven days a week, without throttling, overheating, losing accuracy, or requiring frequent human intervention.
For engineers designing for the edge, the more important question is whether the application can deliver accurate, low-latency inference inside the real power, thermal, cost, and maintenance limits of the deployment.
The Edge Does Not Have Unlimited Headroom
In the data centre, power and cooling can be scaled around the compute infrastructure. At the edge, the compute architecture has to fit the environment.
A smart camera may be constrained by Power over Ethernet (PoE). A mobile robot may be constrained by battery capacity. A factory controller may need to sit inside a sealed cabinet. A smart city edge box may be mounted on a street pole and expected to function through changing weather, network instability, and limited physical access.
This creates a narrow operating envelope. In many edge AI deployments, latency must remain at or below 100 ms to support real-time perception, inspection, or control. Uptime expectations in critical infrastructure and high-performance applications can be as high as 99.999%. Power budgets are often limited to the 5-20 W range, particularly in PoE-powered, battery-powered, or fanless systems. Thermal limits are equally important, with junction temperatures generally needing to remain below 85°C to avoid throttling, premature ageing, or failure.
A processor that performs well on benchmarks but requires 40 W of power, active cooling, or frequent maintenance may be unsuitable for a sealed industrial enclosure or a compact robotic platform. Equally, cloud dependency may work in a demonstration but fail in a building interior, lift, warehouse, or factory floor where network conditions are inconsistent, or a model that runs quickly but loses accuracy after aggressive optimisation may not be reliable enough for inspection or safety-critical tasks. These are the types of constraints which define whether an AI system can be deployed in the first place.
Why Peak Metrics Can Mislead
TOPS and FPS are useful indicators of theoretical capability, but they say little about the physical operating conditions required to achieve them. A high peak figure often relies on power headroom, active cooling, and external bandwidth that sealed or remote edge systems simply cannot provide.
Closing this gap requires an architectural shift: prioritising sustained, deterministic performance over unconstrained burst power. This design philosophy is central to hardware such as DEEPX’s DX-M1 (distributed via Avnet Silica), a neural processing unit (NPU) architecture purpose-built to deliver real-time inference strictly within the physical limits of edge deployments.
Heat is a Reliability Problem
In constrained edge systems, heat affects more than comfort or enclosure design. As temperatures rise, processors may throttle, making inference less predictable while increasing stress on solder joints, dynamic random-access memory (DRAM), and other components. In sealed or dustproof industrial environments, removing excess heat is particularly challenging, especially when ambient temperatures can easily reach 55 °C.
This is where the gap between headline and sustained performance becomes clear. DEEPX’s DX-M1 delivers 25 TOPS of performance at just 2–5 W—achieving 20x the efficiency (FPS/W) of conventional general-purpose GPUs (GPGPUs) while maintaining equivalent AI precision.
In DEEPX benchmark data, the DX-M1 stayed at 61.9 °C during a YOLOv7 33 FPS workload, an object detection model family often used to test real-time vision performance, while a competing device reached 113.5 °C under the same conditions.
In a YOLOv9C maximum-load comparison, a newer and more demanding object detection workload, DX-M1 operated at 75.4 °C while delivering 59 FPS, whereas the competing device reached 114.3 °C at only 32 FPS, an 84% performance advantage while remaining 38.9 °C cooler.
The significance is not simply that one chip runs cooler than another. Staying within critical thresholds enables predictable performance, reduces or removes the need for active cooling, and supports more compact, robust designs. For edge AI, fans, vents, and larger thermal solutions increase cost, complexity, and the potential for failure points.
Accuracy is Part of Reliability
To ensure reliable physical AI interactions, an AI model must maintain its inference precision under real-world optimisation. This is especially important in autonomous or unmanned systems.
Edge systems need lower power and lower latency, but they cannot achieve this by stripping away too much of the model's capability. In many cases, the challenge is to preserve useful AI performance while fitting within constrained hardware.
DEEPX addresses this through its IQ8 quantisation technology, which converts 32-bit floating-point (FP32) models to 8-bit integer (INT8) models with less than 1% loss of accuracy across a large set of commonly used models. In benchmark testing, DEEPX found that its IQ8 quantisation maintained near-equivalent accuracy across multiple model types:
- YOLOv5N_640 (Lightweight Object Detection): Achieved 28.26% mAP on DEEPX IQ8 vs. 28.08% mAP on a reference GPGPU baseline.
- YOLOv8L_640 (High-Capacity Object Detection): Achieved 52.77% mAP on DEEPX IQ8 vs. 52.75% mAP on a reference GPGPU baseline.
- BiseNetV2_1024x2048 (Real-Time Semantic Segmentation): Achieved 75.00% mAP on DEEPX IQ8 vs. 74.95% mAP on a reference GPGPU baseline.
These results highlight how dedicated quantization techniques allow low-power edge NPUs to deliver deterministic efficiency without sacrificing task accuracy.
Hardware Reliability Must be Designed into the Data Path
Reliable edge AI is also shaped by how efficiently data moves through the system. Memory design, data movement, quantisation, and post-processing all influence whether an edge AI platform can sustain low-latency inference in the field.
In the DX-M1 architecture, DEEPX uses 6.75 MB of compact on-chip static random-access memory (SRAM) across three NPU cores, alongside local low-power double data rate 5 (LPDDR5) DRAM positioned close to the NPU. This supports running larger AI models locally without unnecessarily increasing chip size or forcing inefficient data transfers.
The reliability benefit is twofold. First, a smaller SRAM footprint helps reduce die area, leakage current, and heat, supporting more stable long-term operation. Second, local DRAM helps keep larger models available close to the processor. DEEPX testing has found that models such as YOLOv7 (~36 MB) can remain in local memory, allowing the DX-M1 to maintain over 100 FPS, whereas DRAM-less NPU designs drop below 25 FPS when a model exceeds their on-chip SRAM capacity, forcing external weight swapping.
Post-processing also matters. By executing hardware-accelerated post-processing on-chip - filtering and reducing raw output tensors from 8 MB to a 128 KB payload - the DX-M1 eliminates unnecessary bus traffic, lowering latency jitter and thermal stress under sustained workloads.
This is where hardware reliability and real-time performance meet. If the data path is inefficient, latency becomes harder to control, heat rises, and sustained performance becomes more difficult to guarantee.
Performance Per Watt is Closer to the Real Question
If operational reliability depends on sustained AI systems within fixed limits, performance per watt becomes more meaningful than peak compute alone.
DEEPX’s YOLOv7 benchmark data illustrates this point. In DEEPX testing, the DX-M1 measured 40 FPS/W, compared with 20 FPS/W for one competitor, 23 FPS/W for another, and 2 FPS/W for a traditional GPGPU-based approach. The same comparison shows the DX-M1 operating at an average of 2-3 W, while the GPGPU-based system operates at around 40 W.
For engineers, this gap becomes critical within many edge applications. A 40 W processor may be acceptable in a workstation or data centre environment, but it is harder to justify in a smart camera, a compact industrial PC, a robot, or an edge box. Higher power draw can exceed PoE budgets, reduce battery life, require larger power supplies, and generate heat that must then be managed mechanically.
DEEPX’s analysis suggests that, compared with traditional GPGPU-based systems, DX-M1 can reduce electricity and cooling costs by around 94% over five years. It also makes the stronger point that even if competing GPGPU hardware were free, the DX-M1’s operating costs could still make the lower-power architecture more economical over time.
Conclusion: The KPI that Matters is Sustained Intelligence
These requirements are already shaping real deployments. DEEPX technology is being applied by companies including POSCO DX, Hyundai-Kia Motor Company’s Robotics Lab, and LG U+ across smart factory control, robotics, surveillance, and smart city systems, where AI must operate close to the device, inside strict power, thermal, latency, and reliability limits.
As edge AI moves from experimentation into deployment, the metrics used to judge hardware need to mature. TOPS and FPS will remain useful, but they cannot indicate whether a system will continue to work under pressure.
Real-world AI systems must sustain low-latency inference, maintain accuracy, stay within thermal limits, fit available power budgets, avoid unnecessary cloud dependency, and remain economical to operate over time. This is why reliability is becoming the defining KPI for edge AI.
Avnet Silica recognises this shift, and the DEEPX’s DX-M1 directly reflects it. By combining a low-power NPU architecture, strong thermal behaviour, high-accuracy retention, and efficient local inference, it demonstrates how edge AI hardware can be designed around sustained deployment rather than peak benchmark claims.
