Practical technologies for Smart Shops 2

Part 1 of this article introduced the technologies of the MIMEX project, focusing on the use of computer vision (CV). As mentioned there, CV allows MIMEX to recognize both people and objects without the need to add explicit, wearable/mobile tags. While this offers clear advantages, including ease of installation and use, by its nature, CV suffers from object occlusion (when objects are outside or hidden from the view of the cameras), poor lighting conditions (in which objects in the resulting images cannot be distinguished from one another) and cost (as one of the mitigation techniques for occlusion is to add multiple cameras, even in a small space).

 

While MIMEX is actively working on these CV challenges, we are also exploring "sensor fusion" -adding non-vision-based sensors to the Smart Shop and using their data to enhance and supplement the information gathered by the vision system, improving the robustness and accuracy of the MIMEX Smart Shop. These additional technologies are the focus of this second
article.

 

First, we consider two technologies that can be used to track the products of the Smart Store: smart shelves and object tagging.

An intelligent shelf should know the products on top of it, raising events when they are moved  and/or removed. The MIMEX smart shelf relies on strain-gauge based sensors that are essentially thin wires embedded in the horizontal platform of the shelf. When an object is placed on the shelf, the tiny deformations in the wires are detected as variations in the electrical resistance of the wires, revealing the presence (or lack of) products on the shelf. The weight sensors used in MIMEX are a load cell based on a strain-gauge sensor selected for their cost-effectiveness and precision. Because the shelves are stocked with known items, MIMEX knows which item has been picked up and will likely be purchased in the near future.

 

Alternately, directly tagging products allows them to be tracked from the shelf to the exit. Examples of such tagging and tracking technologies include RFID and UWB. Notably, in a Smart Shop without an on-site attendant, the tag's size, flexibility, and cost must be considered as they cannot be removed from the products at purchase time. RFID (Radio-frequency identification) tags are already common in many shopping environments.

Their low cost makes it reasonable to attach them directly to individual products. RFID readers using electromagnetic fields can detect nearby products based on their attached tags. When embedded in an exit passageway, the sensed product identities can be merged with data from other sensors collected during the shopper's visit and applied to confirm the purchased products.

Ultra-WideBand (UWB) offers an alternate tagging and tracking system using a novel, low-power, high bandwidth radio technology that allows a set of fixed anchor devices, e.g., affixed in the corners of a room, to track objects tagged with active UWB devices with centimetre-level accuracy. While this could be used to track products, the cost of the tags is relatively high.

​

MIMEX also explores radio-based solutions for detecting and tracking people inside the Smart Store. They can be used to measure the distance between a person and a product or shelf, supplementing the other sensors to refine the possible products a shopper has chosen or to analyze the shopping experience to identify future improvements to the MIMEX Shop itself.

​

While UWB is prohibitively expensive for individual objects, MIMEX explores the possibility of attaching a tag to an optional shopping basket and tracking it throughout the store as a proxy for the shopper. Alternatively, a Bluetooth-based system can detect the presence of a shopper in the store by overhearing BLE transmissions from their smartphone. BLE receivers placed throughout the shop can also provide an estimate of the shopper's location. The challenge for the MIMEX Smart Shop is to balance the benefits of each technology against the costs, both in economic terms for the installation as well as in the acceptability of the technology to the potential shoppers.

 

The resulting multi-sensor approach, in which each technology fills in sensing gaps inherent to the other technologies, will result in a reliable, welcoming Smart Shop.

The (new) end :-)