Smart cameras, cloud platforms shorten long tail of the IoT

The ultra-low power RSL10 Smart Shot Camera platform can be used to monitor anything, making it easier to bring the benefits of the IoT to existing and new applications.

The idea of connecting devices to share data predates the internet. It was the introduction of ubiquitous connectivity that really created the Internet of Things (IoT). Decades later we are still to realize the Internet of Everything. There are too many “things” that could be connected to the internet but simply may never be. Analysts talk about tens of billions of connected devices but, in reality, that number could easily be trillions.

Evaluating the return on investment, or ROI, is part of the due diligence behind any IoT project and often involves adding sensors and wired/wireless connectivity to an otherwise “dumb” device. While sensors are not expensive and wireless connectivity modules are plentiful, the process isn’t free. As a minimum, it would demand resubmitting the project for compliance testing. It will require new processes for production, assembly and test. Aftersales support is also a huge consideration for connected devices.

Manufacturers are forced to choose whether to modify an existing product or wait until the next generation is developed. That could be years, meaning the benefits of the IoT may be out of reach for a long period of time. This delay is creating a long tail of devices waiting to be on-boarded to the IoT.

Bring your IoT solutions to life with Avnet Silica and onsemi. Learn more here.

Sensor fusion in the IoT

Sensor fusion takes multiple sensors and brings their data together to provide greater insights. This is the foundation of using AI in the IoT. It enables algorithms to infer by using multiple sources of data at the same time.

What most sensors have in common is that they only detect one property. The sensors must also be co-located or even deeply integrated into the application. Image sensors are fundamentally different from other sensors. An image sensor comprises hundreds, thousands or even millions of individual sensing elements, or pixels.

Cameras can capture data without being deeply integrated into the application. Because they can observe from a distance unobtrusively and independently, they can be deployed long after the real application goes into the field.

Ubiquitous connectivity will lead to ubiquitous intelligence, as AI becomes more integral to the IoT. This trend will make it easier and more cost-effective to bring many of the things in that long tail into the IoT. Smart cameras that use AI can monitor, identify, recognize objects and events, and automatically generate actions.

The RSL10 Smart Shot Camera platform from onsemi combines an ultra-low power camera module with the RSL10 Bluetooth System-in-Package (SiP). It integrates easily into the IoTConnect® Platform from Avnet to enable simple development of smart camera systems that use AI to identify objects automatically.

A smart camera can be used on its own or form part of a sensor fusion solution. For example, an industrial refrigerator may include a connected temperature sensor. There could be several causes for a reported temperature rise, the most likely being that someone has left the door open. Using a smart camera to monitor the door would provide immediate confirmation that the temperature rise isn’t due to a more serious fault.

Putting machine vision at the network’s edge

AI-based image recognition is a non-intrusive form of sensing. And because it uses image data, it isn’t tied to any single parameter. Using powerful AI in the cloud to recognize objects in the scene has virtually limitless potential. Some example applications are listed here.

• Asset tracking. With a distributed network of smart cameras in your building or warehouse, locating assets becomes easy. By training the AI service to recognize your assets, their location can be tracked automatically.
• Stock monitoring. Using a smart camera to monitor products on a shelf or in a vending machine can provide a simple way of managing resources such as delivery drivers and warehouse space. It can also be used to analyze consumer preferences by fusing the data from shelves/vending machines in several locations together to create metadata. Over a short period of time, this metadata could influence a vendor’s choice in which products to continue offering or discontinue. Developing a similar IoT solution using other types of sensing modalities could involve modifying the vending machines or adding complex proximity, weight, light or infrared sensors to shelving systems, with the corresponding control units.
• HMI monitoring. A smart camera can be positioned to observe a digital display or analog dial and send an image to the cloud platform for analysis. Using AI, the reading on a gauge can be recognized and compared against a set of parameters. Actions can be issued based on what the gauge reads. Achieving the same functionality using any other sensor would need the application to understand the property being measured. The value sent to the IoT platform would correspond to the property, which introduces the need for sensor calibration and signals. There may be safety issues if the property being measured is hazardous. Using a smart camera, all that additional effort is avoided.
• Hazard detection. Moveable hazards in work areas, walkways or other access points can cause accidents as well as become safety issues in emergencies. A smart camera can automatically detect objects in areas that should be clear, using AI to recognize what the objects are. This information could be used to alert the most appropriate person or, in some cases, activate an alarm. Monitoring the same spaces using other sensor modals would require considerably more complex solutions, particularly if it was important to minimize false positives and false negatives.
• Traffic monitoring. Both road and pedestrian traffic monitoring is now commonly used to minimize congestion. As road vehicles become smarter it will be even more necessary to share this information in a digital form. A smart camera can automatically identify types of traffic as well as speed, direction and time spent in congestion.
• Environmental monitoring. Many types of weather conditions can be easily seen and people are adept at identifying rain, wind, sunshine and snow. Each of those conditions would require a fleet of sensors to accurately detect. Using a smart camera makes it simple.
• Proximity detection. While proximity is one of the most common types of smart sensing modals used today, each example uses a different type of sensor. For example, proximity can be detected using ultrasound or infrared, but it could equally be sensed using a Hall-effect sensor or a reed switch and magnet. Resistive sensing is also often used. All these modalities could be replaced using a single smart camera without the need to accommodate a physical sensor in the design.

Cloud-based AI will power the IoT

AI is the right approach to scaling up the IoT. In the future, as billions of devices go into service, AI will be the only practical way to process the huge amounts of data being captured by all types of smart sensors. Some of that AI is being deployed at the edge, but cloud-based AI is and will remain a crucial part of the infrastructure.

The size, cost and power requirements of hardware like image sensors and wireless connectivity continue to be driven downward by developments in the semiconductor industry. This will lead to a massive expansion in smart cameras. When combined with continued improvements in cloud-based AI, smart cameras will provide an element of futureproofing in the IoT. Systems deployed today that use cloud-based AI will get smarter over time, without the need for any changes in the field. All the future gains will come from improvements in the cloud services. As long as they are connected, products in the field will always be up to date.

Conclusion

Smart cameras provide a scalable solution to reducing the long tail of the IoT. To achieve ubiquitous intelligence, more things need to be connected. Image sensors with localized processing and efficient short-range wireless connectivity like the RSL10 Smart Shot Camera – integrated to a comprehensive cloud-based platform with AI capabilities such as IoTConnect from Avnet – provide the ideal system solution.

Follow Avnet Silica on LinkedIn