Streamlining the IoT using smart cameras for asset monitoring

The RSL10 Smart Shot Camera provides the perfect platform for developing a smart camera with automatic object recognition using cloud-based AI.

Developments in low-power image sensors and short-range wireless connectivity are enabling new applications to penetrate the IoT. As a sensing medium, every frame generated by an image sensor contains a huge amount of data ready to be exploited. Getting image data from the edge to the core of the network normally means a high-bandwidth connection like Ethernet or Wi-Fi, but these technologies can’t offer the ultra-low-power needed for endpoints expected to run for as much as five years on a single coin cell battery.

In this respect, Bluetooth is rapidly becoming the preferred wireless protocol for the IoT. Bluetooth has continued to evolve and has moved beyond being a simple alternative to cables. Since Version 4, the Bluetooth standard has provided greater support for IoT applications by offering low energy modes and other features aimed at machine-to-machine communication. With Version 5, the Bluetooth SIG extended this support to include longer range transmission at higher bit rates. This makes Bluetooth the ideal transport medium for data, even for image data generated by small, low power, smart cameras.

Image sensors are based on arrays comprising thousands or even millions of pixels. This generates much more data than, say, a temperature or humidity sensor. Even an integrated motion sensor detecting movement across six or nine axes doesn’t generate as much data as the smallest image sensor.

The large amount of data involved requires a correspondingly high processing effort. Using image sensors for machine vision is typically reserved for industrial equipment that has virtually unlimited resources, including offline power. Machine vision is also being used in robotics but, again, its use is relatively niche. Until recently, smart cameras haven’t been used for asset monitoring, beyond the typical closed-circuit security system.

That’s changing thanks to the development of smaller, more energy-efficient image sensors and the low power features of Bluetooth 5.

Bring your IoT solutions to life with Avnet Silica and onsemi. Learn more here.

Everything is an asset

When coupled with AI, a smart camera can be used to identify far more than defective widgets on a production line or suspicious movement in a parking lot. An asset is anything that has a value to its owner. Monitoring assets using smart cameras is a scalable approach to harnessing the power of the IoT.

Using machine vision at the network’s edge has huge benefits, but realizing its potential requires the optimal combination of image sensing, local image processing and efficient data transfer. It would be impractical to transfer all the data generated by an image sensor across a bandwidth-limited connection. It would also be difficult to put all the image processing needed to generate actionable insights into the sensor itself.

This points toward choosing a solution that has been developed with limited resources in mind. Using localized image processing along with the most efficient wireless data transfer methods allows machine vision to be applied to asset monitoring.

A smart camera platform optimized for the IoT

The RSL10 Smart Shot Camera platform from onsemi has been developed specifically for enabling machine vision at the edge of the IoT. At its heart is the RSL10 System-in-Package, one of the industry’s most energy-efficient Bluetooth 5 wireless solutions. This has been coupled with an onsemi Image Access System (IAS) based on the ARX3A0 CMOS image sensor, which features a 560 x 560 array of 2.2 µm rolling shutter pixels.

The platform also uses an image controller to provide flexible trigger modes and JPEG compression. This ensures that only the most important and relevant image data is sent across the Bluetooth connection to a gateway, from where it can be sent up to the cloud for further processing.

Using a cloud platform that offers image recognition using AI, such as the IoTConnect® Platform from Avnet, means your smart image sensor can be used to detect a huge range of objects and act on the information. Combining in-sensor image pre-processing with cloud-based image analysis provides the best mix of capability and efficiency. This also means that the image sensor in the field will get smarter over time as the AI capabilities in the cloud continue to improve.

Data driven asset management

Asset management is a data-intensive process. It requires information to be recorded, often manually, and can be as simple as counting the number of boxes on a warehouse shelf. The simplicity of this task belies its complexity. It involves object identification and classification, but it is also subjective. The asset may exhibit damage and if it has an expiry date it may be close or passed. Products may be in the wrong location so localization is also important.

Automating this using conventional sensors would be complex, but it is relatively simple for a smart camera that has been trained to recognize the products. The position of the camera would be recorded so the localization aspect is taken care of. The condition of the product would be almost impossible to detect using any other sensor modality.

The use of smart cameras integrated into an IoT platform that offers AI-driven object recognition delivers on all aspects of asset monitoring. Now, for the first time, it is technically and commercially possible to deploy smart cameras dedicated to asset monitoring it this way.

Conclusion

The IoT has unlimited scope to help streamline all aspects of industry and commerce. At the edge it relies almost entirely on sensors to collect and process data. Using smart cameras to capture information that can be processed in the cloud transcends other sensor modals. Through the introduction of Bluetooth-enabled low power image sensors, smart cameras are now being deployed in the IoT to automate, accelerate and improve asset monitoring.

Follow Avnet Silica on LinkedIn