Smart Sensors: Smart Eyes for IoT

Technicians have been reluctant to use cameras for IoT because they were considered too expensive. That is now history. Today, CMOS sensors with respectable resolution cost only a few dollars and chips that deliver high-definition images are available for a little more. Thanks to AI, new integrated image signal processors (ISPs) are taking the strain off transmission networks and edge computing.

Cameras provide lots of data and the more data, the better they are. Unfortunately, that clogs up the communications channels. In addition, the images have to be prepared and processed which puts a strain on the computers in the system. Often, the normal TV frame rate of around 25 frames per second is not enough, either. Manufacturing machines operate so fast that surveillance cameras have to deliver 60 or more images per second. Thanks to data reduction, the H.265 transmission format shrinks the amount of data and thus enables wireless operation – but H.265 is computationally expensive, which is a shortcoming for battery-powered devices.

CMOS chips also have peculiarities. Most sensors produce a certain basic noise level, even in absolute darkness. These chips also inevitably contain a few defective pixels. No problem – there are correction programmes for that. In addition, the distances between camera and object are often small in IoT applications. Also, no problem – that’s what wide-angle lenses are for, though they do tend to distort and suffer from vignette shadowing around the edges. If the image analysis determines whether labels on a bottle are correctly positioned, for example, image errors can confuse and deceive the analysis programmes. Here, too, there are proven remedies. However, correction programmes put a strain on the camera CPUs because the correction is best done before the image has been compressed in order to achieve optimum results. This, in turn, puts further strain on the communications channels. Usually, in cheaper systems, the black-and-white image of the sensor has to be transformed into a color image. For this purpose, the mosaic-like pattern of color filters over the camera sensor has to be back-calculated, so to speak, in order to obtain red/green/blue color separations (demosaicing). This also costs computing power.

A new generation of image signal processors (ISPs) now offers developers a solution to these problems. ISPs operate in the immediate vicinity of the camera module and handle all the pre-processing of the images. They are calibrated to the chip and its optics and transform the raw data into images that can be easily processed further.

In reality, no one is interested in the camera images, only the results count – thumbs up or thumbs down. In the labeling example, the operator wants to know if the label is positioned correctly: is it crooked or, in the case of pharmaceuticals, is it stuck together in such a way that it’s no longer legible? When cleaning bottles, no one is interested in the 100,000 bottles that have been cleaned well but only in those that need to be cleaned again or belong in the waste because they have a crack.

Today, AI can take over these relatively simple tasks and the new ISPs also have a neural network on board that runs an AI programme for image analysis. In this way, the engineers can let them make their own decisions without the need for cloud support or data flow gymnastics.

Qoitech: Fighting Power Hunger

Most IoT devices are battery powered but designers want to know how power consumption will fare over time. Finnish startup Qoitech has launched the Otii Arc, an inexpensive ‘little helper’ that makes it possible to precisely track current, voltage and power consumption before a device is deployed.

All batteries react the same way to load – the voltage output plummets. Unlike a flashlight, an IoT sensor does not load the battery evenly. Sometimes the sensor is ‘asleep’ and consumes no energy, sometimes it sends data and consumption skyrockets for a short time. Decisive for the load profile is the software, which may or may not be cleverly written, and the environment, especially the temperature. The Otii Arc measuring device connects directly to the sensor and a laptop. It acts as a power source and determines the load profile over the operating time until the dreaded ‘brown-out’ as the battery empties. Qoitech’s device allows small development offices to carry out their own measurements and tests at a high technical level. The target groups also include software developers because the more cleverly the software manages the energy reserves, the longer the sensor battery will last. Programmers can now conveniently and quickly determine the success of their efforts without having to rely on lab tests from engineering firms. Determining battery runtimes is not the easiest of tasks. Within a few months of the launch, Qoitech received a number of requests and suggestions from users, which it rapidly incorporated in a software update.

Fraunhofer: How to Bee a Sensor

What does a bee need to survive? Dramatically declining populations spurred the German government to have special sensors developed. In response, researchers at Fraunhofer IZM created ultralight wireless sensors that can be attached to a bee’s back, turning the insect into a flying IoT sensor. The RFID chip transmits important flight data as well as temperature and brightness to a base station. The system, which weighs only ten milligrams, is powered by a specially developed ultra-light battery which receives its energy from a microscopic solar module. In the medium term, the Sens4Bee project, based at Fraunhofer’s Helmholtz Centre for Environmental Research (UFZ) is to become a tool for all interested beekeepers. The aim is to clarify the relationship between bee mortality and the application of pesticides.

Learn more about the Smart Shot Camera solution from onsemi here

READ MORE ARTICLES