Smile as you buy your holiday goods in a store – you are probably being watched, tracked and analyzed

Amazon may have started out by selling books, but it is now getting heavily into the surveillance market. There are four main sectors where it is already working on solutions that pose risks to privacy. Three of them have been covered extensively in this blog before. They are: the home, with its Alexa devices; the street, where its Ring doorbell cameras are creating massive community surveillance systems; and police forces, with its cloud-based facial recognition system called “Rekognition”. The fourth sector where Amazon is using surveillance as the basis of its offerings is in retail. Although the idea of what came to be called Amazon Go was first revealed three years ago, the roll-out has been relatively sedate. Today there are a couple of dozen stores, with more planned. Amazon’s explanation of how these stores differ from conventional outlets is as follows:

Our checkout-free shopping experience is made possible by the same types of technologies used in self-driving cars: computer vision, sensor fusion, and deep learning. Our Just Walk Out Technology automatically detects when products are taken from or returned to the shelves and keeps track of them in a virtual cart. When you’re done shopping, you can just leave the store. A little later, we’ll send you a receipt and charge your Amazon account.

The central idea is that shoppers are monitored by cameras and computer system at all times, with their actions recorded and analyzed in order to calculate the amount to be charged seamlessly to their Amazon account. As computer vision and AI continue to make rapid advances, this kind of approach to retailing is attractive for companies looking to remain competitive and cut costs. Even the current leaders in the grocery sector are starting experimental deployments. The grocery giant Walmart is using advanced digital technologies for a number of purposes, including monitoring bananas:

When a banana starts to bruise, the cameras send an alert to a worker. Normally, that task would have relied on the subjective assessment of a human, who likely doesn’t have time to inspect every piece of fruit.

Welcome to Walmart’s Intelligent Retail Lab — the retail giant’s biggest attempt to digitize the physical store.

Other, more conventional, applications of the technology include tracking when shelves need to be restocked, or if shopping carts are running low. The system can also detect spills on the shop floor and if long lines are forming, so that more cash registers can be opened up. Those kind of uses will be welcomed by most people. If high-resolution CCTV cameras are hooked up to AI systems, managers will naturally also see them as ways to monitor customers as they move through the shop. One use is to identify shoplifters, or to stop “shrinkage”. Another is to spot known thieves as they enter the store. Again, many might feel those are justified, even welcome, uses. But once AI is applied to high-resolution camera data in order to guess your age, gender and even mood, things are moving into a different area. Some stores are using those guesses to show real-time micro-targeted ads on in-store screens.

This raises some troubling possibilities. For example, older customers who seem more affluent might be charged higher prices, especially if automatic, invisible billing is in operation that masks differential pricing. Poorer-looking people might be offered discounts on lower-quality goods on the assumption they couldn’t buy more expensive ones. People looking depressed might be offered medication to make them feel better – or alcohol.

The other problem with these technologies is that once companies deploy them, an arms race will begin as rivals try to leapfrog each other, using more and more tracking technology. New startups are already appearing that aim to take this kind of in-store surveillance to the next level. For example, AiFi works in the same way as Amazon Go, but is designed to scale: the company claims its Autonomous Store Platform can track up to 500 people, and tens of thousands of products. TechCrunch explains:

it can track shoppers’ behavior in the store, including things like if they’re shopping in groups, what items they’re picking up and putting back, their gait, their body poses, where they go in the store, and even identify if they’re doing something abnormal, like shoplifting.

The system today is designed to use a combination of technologies, including facial recognition, to identify shoppers during their session, but in the future this could be connected to their past sales history to make personalized recommendations.

Significantly, AiFi is also looking to expand to other retail contexts, such as airports and train stations. And where there is the Autonomous Store Platform or similar, there is also pervasive tracking. The trend is for any environment in which things can be sold to move to automated sales using discreet but powerful surveillance systems.

What’s most troubling about this is that soon there will be few spaces where people can be sure they are not being tracked or subject to some form of surveillance. So-called smart speakers and other smart devices are constantly listening to what we say. Digital doorbell systems like Ring are creating a dense web of surveillance even in leafy suburbs. Within cities, CCTV is deployed ever-more widely. Schools too are installing cameras systems, usually on the grounds of safety. That last tendency normalizes surveillance for young people growing up with them. It makes the transition to an adult world where everything you do online and offline is tracked, stored and analyzed, all-too smooth.

Featured image by Ingolfson.