IoT and the Future of Retail: Done Dunning

Amazon has thrown down the gauntlet to the classic retail trade with its “Just Walk Out” high-street stores. This scared traditional traders but, while China is mobilizing billions to compete, Europe is relying on its culture of regionally fissured markets that has grown over centuries. It is a weakness and a strength at the same time.

In the first few months of 2020, the retail sector experienced an unprecedented surge in digitalization, forced upon them by the Covid-19 outbreak. As many European shops were closed by order of the authorities in all countries except Sweden, the companies affected had to find a second foothold by offering their goods online and then sending them by post or making them ready for socially distanced collection. This should lead to a permanent expansion of the distribution channels.

For the food retailers not affected by the closures, the pandemic led to a sudden increase in demand which could not be met as a result of panic-buying customers.

Interest in using service robots in the future for the provision of goods on the sales floor has now increased noticeably. Time is pressing, as China and the USA accelerate the pace. The Chinese Alibaba Group has been deliberately building a bridge between online and offline purchasing since 2016. Alibaba founder Jack Ma says that if 80 percent of retail sales are still being made in local retail outlets, his company must accommodate its customers. This is especially true of fresh products such as fish which are so popular in China. The customer can choose the fish and have it prepared in the shop, or have it delivered with-in three kilometers of the store.

If 80 percent of sales are still being made in local retail outlets, our company must accommodate its customers.
_{Jack Ma, Founder of Alibaba}

In Alibaba’s Freshippo (formerly known as Hema) supermarkets customers can scan the goods them-selves as all products have machine-readable labels, and then pay for their purchases by face recognition and Alipay using their smartphones. The superstores cover the full range of a supermarket, including sex toys. The giant marketer opened its first store in 2016, and there are now hundreds of them, with a five-year goal to establish 2,000 outlets.

Just Walk Out: Image analysis and deep learning combine to track purchases. The price is debited to a customer’s account so there is no need to stand in line at the cash register.

Pioneers and Latecomers

Without doubt, the technological pioneer is Amazon’s “Just Walk Out” store. The customer registers at the entrance with their smartphone using a QR code. The store is equipped with an extensive array of cameras, light barriers, and sensors for pressure, infrared, and volume changes, backed up by image analysis, sensor fusion, and deep learning. All this makes it possible to assign a selected product to a specific person or re-move it if they take a product out of their basket and put it back on the shelf. Audio monitoring also helps to detect when products are passed from one person to another, for ex-ample when someone asks a friend to pass a product to them. Once a customer has all the products on their shopping list, they can leave the store with their trolley without queuing to pay.

Take Your Pick: The smart fruit stand uses camera images to determine if a product is spoiled. AI informs the staff to refill the counter with fresh produce.

The smart infrastructure records everything and the purchase is automatically debited. Amazon’s test setup is not yet perfected and the presence of too many people confuses the cameras, so access is limited. Ironically, customers have become used to such restrictions during the corona virus pandemic, but analysts see this kind of future shop as problematic among privacy-conscious Europeans, especially in Germany. One does not wish to be permanently observed.

China and the USA may be ahead of the pack when it comes to big data and AI but Europe has not been idle. In the 2000s, for example, numerous research departments developed mobile service robots up to production readiness. They were intended to relieve the strain on employees, especially in monotonous and dangerous work, but commercial success was extremely moderate, partly because the robots had to be permanently controlled or monitored by humans. Today, thanks to advances in AI, their use is possible in a completely autonomous manner while, at the same time, much more powerful data networks are available to integrate the robots very closely into the retailers’ merchandise management systems. Europe is also countering by sponsoring projects, such as the EIT Digital program (www.eitdigital.eu), which are then supplemented by national research projects. Every-thing is not as homogeneously organized as in China or the US – but the Europeans have learned to live with it and still somehow keep up. They are relying on the strong medium-sized companies that manufacture shelves and shopping trolleys to future-proof their products for IoT through networking and sensors.

Europe’s Race to Catch Up

In contrast to Amazon, which is aiming for a shop without staff, the Europeans want to relieve their employees of less productive activities in order to increase the value-add. Consequently, shop work is shifting. Small but time-consuming tasks, such as invoice printout, become superfluous with the use of smart-phone apps, and the ability to use dynamic pricing ensures the timely sale of fresh produce. At the exit, self-checkout means only one cashier would be needed for supervision and information, allowing the others to devote time to other tasks. Near Saarbrücken, technicians are researching the European answer to Amazon and Alibaba in the 500 sqm Innovative Retail Laboratory operated by the DFKI research institute (the German Research Center for Artificial Intelligence).

AI is no better than a good employee, but it supports inex-perienced staff members.
_{Frederic Kerber, Innovative Retail Laborator}

The system is based on sensors on product shelves which are connected to the AI in the office and back office. With German thoroughness, the researchers are looking at the entire purchasing process: from the planning at home using a smartphone, the walk to the store, the actual shopping, and finally the payment process. Everything should become more modern without scar-ing the customer. Developing the system means working out the correct balance between technology and usability. “Retail is not only used by technology freaks, so as little as possible should change for the consumer to keep the learning curve flat,” explains Frederic Kerber, head of the Innovative Retail Laboratory. “We use technology that is not yet available commercially, so we are always a step ahead of the market.” AI plays an important role in this shop of the future. “Take the fruit counter. If AI uses camera images to determine that a product is being put back again and again, then it may be spoiled. In such a case, the AI informs the staff to refill with fresh produce. The AI is no better than a good employee here, but it supports inexperienced staff and is always present so that employees can focus on other aspects, such as customer service,” says Kerber.

AI and the Future of Retail: Outlook for AI

The DFKI project is financed by the German government and contributed to a recent report from the German Federal Ministry of Economics and Technology called “Perspektiven der Künstlichen Intelligenz für den Einzelhandel” (Prospects of AI in the Retail Trade). DFKI operates closely with other important EU states – after all, its base in the Saarland is located on the border between Germany, France, and Luxemburg, a stone’s throw away from Belgium or even the Netherlands. The researchers working on this answer to Amazon’s challenge are correspondingly international. “Small businesses are slow to respond to new technologies, but the Covid-19 pandemic has massively strengthened a trend among retailers that was already noticeable before. Even retailers who only ever sold over the counter are acquiring an additional online foothold and selling by post.

Due to covid, even retailers who only ever sold over the counter are now acquiring an online foothold.
_{Niels Will, DFKI}

During the shut-down, this was the only way to get money into the cash register at all in many industries,” says Niels Will, research assistant at DFKI and co-author of the study.

Small to medium-sized retailers, of-ten family-owned for generations and specialized in a limited range of goods, have a long tradition and significant market power in Europe. Such shops deal in locally produced food or tools and semi finished products for craftsmen or other trades-men. Most of them sell high-priced goods that require a certain degree of advice. Mutual trust is crucial for the long-standing relationships be-tween these traders and their customers. Such businesses have a number of employees, some branches, and a considerable customer base, all good prerequisites for making them fit for the future with a little extra know-how. “We wanted to give small and medium-sized businesses a guide on how to integrate AI into their business. The very small retailer, the famous ‘mom and pop store,’ is probably simply too small for such projects but shops with maybe two or three branches that have a certain amount of local market power are interesting,” says Will. Two thirds of the shop owners interviewed by the researchers consider AI an important new technology, but only one third have any experience of it.

Keeping Track: AI will bring a better understanding of processes to optimize the supply chain along the entire value chain – from the manufacturer to the wholesaler, the retailer, and finally to the end consumer.

While Amazon is planning a major innovation push, Europeans are as down-to-earth as ever and typically address the small problems of every-day life. In the retail trade, personnel costs make up the lion’s share of the operating expenses incurred, up to 75 percent in some cases. Employees should therefore work as productively as possible and not be distracted by having to check for incorrectly or unclosed refrigerator doors, to search for goods that customers have left in the wrong place, or to look for gaps on the shelves – all of which are unnecessarily time-consuming.

This is where additional sensors and a central data acquisition system could help. Antennas scan for RFID tags attached to the goods and specifically alert employees to incorrectly placed goods; intelligent sales shelves automatically detect gaps appearing and order replenishment to ensure they are refilled as quickly as possible.

Germany’s software giant SAP has jumped on the bandwagon and is getting its Leonardo Retail Innovation Kits middleware ready for IoT. “Sensors account for only a third of the effort, the rest is middleware and processes. These must be learned and mastered,” reports Kerber. The new applications also enable retail companies to carry out predictive maintenance of all systems. To do this, they bundle not only the information from refrigerated counters and chests, but also the data from baking machines and electrical empty-container collectors. Repairs can be optimally planned and unforeseen breakdowns due to defective parts can be prevented. The software also compares a company’s branches with each other to detect outliers, such as detecting highly excessive energy costs, which may indicate in-correctly installed or aging refrigeration units.

A further starting point for automation is the provisioning of goods in the store by AI-controlled autonomous vehicles. “Already today, there are automatically moving, low-level vehicles that can bring goods safely from the warehouse to the shelves. AI will optimize the routes and the positions of the goods in the storage area to ensure the fastest possible access,” says Will. The smart shop’s middleware independently ensures that goods needing to be replenished most quickly are located near the exit.

The DFKI researchers consider inadequate and undersized data sets to be one of the main obstacles to the successful introduction of AI and IoT in the retail trade, followed by insufficient competence in one’s own business. Large amounts of data are required to train an AI system and this causes a number of medium-sized companies problems. Not all of them have a complete data set of sufficient volume. In addition, the researchers have found that data records are supplemented with information based on the owner’s memory – often incorrectly. Despite the need for large amounts of data, the computing power of the existing IT is usually sufficient to run the system. “More than an average PC is not necessary,” explains Will.

End of the Queue: AI and the Future of Retail

The RFID tags widely used today are to be supplemented by NFC technology in the business of the future. Modern smartphones can read the RFID tags via NFC and this allows the customer to download additional information directly to their phone when deciding on a purchase. Buying will only require a command from the smartphone and this will make queuing at the checkout a thing of the past, much like at an Amazon store.

The researchers are thinking beyond this and want to put a digital twin of the store on the smartphone so that the customer can better plan and organize their next visit. “Then there is no need for long searches and important information about all the products appears on the customer’s smartphone. For example, the customer could see at a glance which allergenic substances a food product contains,” says Will.

Already implemented in some stores is digital screening of behavior on the shop floor. AI systems analyze the pose, dwell time, and viewing times of customers at specific shelves to optimize the sales area. These AI systems complement the evaluation of purchasing behavior by target groups and product category.

Right Connections: Smartphone apps of the customers or intelligent labels incorporating display screens on the shelves will provide references to products that are often bought with the customer’s current purchase.

In the future, the smartphone apps of the customers or intelligent labels incorporating display screens on the shelves will provide references to products that are often bought with the customer’s current purchase. This service could be sweetened for the customer by offering a discount, very similar to the bundles offered by mail-order companies.

The wholesaler Metro, one of the largest cash and carry operations in Europe, is experimenting with an app that provides information on prices, offers, and, above all, on the availability of goods. This is especially important in times of pandemics because wholesalers like Metro were unable to supply toilet paper for weeks and soap was available but subject to quotas. The researchers warned that customers often find unsuitable offers a nuisance. Personalized advertising is only accepted by customers if the retailer follows rules. People interested in buying cheese and wine expect offers in the same area and not information about office supplies.

Intelligent Dunning

Even payment reminders and order for payment procedures, called dunning, are to be made more efficient by AI. At present, these tasks are often still handled by mailing standardized letters. AI-based intelligent dunning procedures in the future will supplement the classic method and, for example, use digital channels to address debtors. The customer in question will be contacted by SMS, e-mail, or messaging apps in a pro-active way and problems or open claims can be identified and processed at an early stage before they become critical. In this way, people are also reached preventively, where unwillingness to pay or lack of liquidity is not the reason for a delay in payment. With the help of AI, a management process can be provided that decides when, through which channel, and with what tonality a person should be addressed. The system also optimizes and identifies the optimal time for the first contact, as well as the most effective channel for different customer groups. Later on, AI will further support the service center in an ongoing dunning process, until the final stage where lawyers become involved.

Another major step will be the use of mobile robots to support the employees. Apart from acting autonomously within buildings to help replenish the goods, they could also be used to ensure continuous monitoring of the building, acting as AI-controlled guards. Equipped suit-ably, systems could use their sensors to detect unusual activity and trigger an alarm if necessary. Researchers suggest pan-tilt cameras could be used as well as passive infrared and additional radar sensors. The latter would detect movements of people over long distances and through walls. In addition, the robots could be equipped with laser scanners. The digital security guard would then patrol autonomously within a self-generated map. With the possibility of map-based navigation, targeted or intelligent routing can be realized. In addition, the robots would then be able to safely approach a loading station and thus guarantee 24-hour security monitoring. Systems like this that enable seamless integration of the robots into existing safety systems and concepts have a clear advantage. Should they be implemented, Europeans would be one step ahead of the competition.

Moving Around: Mobile robots will work autonomously within buildings to help replenish stocks, but they could also ensure continuous monitor-ing of the building, in effect acting as AI-controlled watch-guards.

This advantage also applies to an-other future scenario. The researchers in Saarbrücken want to use AI to coordinate the purchasing behavior of customers, under various weather and road conditions, with transport capacities. In this way, notoriously congested motorways could be relieved and empty runs minimized. At the moment, it is estimated that every third truck is driven empty through Central Europe, so any re-duction would be an improvement.

Trading Future

Robots could also be used to auto-mate pick and place processes by independently handling product reshelving and sorting as well as for removing faulty or surplus goods and preparing them for return shipments. Such systems already work in the laboratory but they are still far too expensive and slow. The researchers’ wish list also includes drones that monitor room temperature and humidity in large warehouses, record inventory levels, and detect packaging damage Scientists are striving toward complete transparency of the process chains and automatic detection of weak points. This is where optimization takes place. To this end, the entire logistics area, pallets, packages, and containers of loose goods would be equipped with sensors connected to the Internet of Things and the merchandise management system. In this way, AI would bring a better understanding of processes to optimize the supply chain along the entire value chain – from the manufacturer to the wholesaler, the retailer, and finally to the end consumer. Forwarding agencies and parcel services providers should also be included, but that will only happen if an inter-national standard for data exchange between the various trades can be agreed and implemented.

READ MORE ARTICLES