Is it true that Amazon's checkout-less smart stores employed a thousand Indians instead of artificial intelligence?

In April 2024, the news that in Amazon’s offline stores, instead of the declared artificial intelligence, the work was carried out by hundreds of Indians who watched customers using video cameras caused widespread resonance. We checked if this is true.

Publications that Amazon is allegedly closing its stores with AI technology Just Walk Out (“Take it and go"), began to appear first in the American press, and then in Russian-language media. April 3 edition RBC wrote that instead of AI, the products were scanned by 1000 Indians, and the publication on the website NTV published under the headline “The work of the neural network in Amazon smart stores was carried out by thousands of Indians.” Portal "CityChe” described what happened like this: “The biggest scam in the field of AI. 1000 Indians replaced by artificial intelligence. "Amazon has admitted to a major scam." The news was also commented on in the State Duma of the Russian Federation - according to opinion Deputy Alexey Govyrin, “Russia should not get involved with Amazon,” because “instead of hiring ten cashiers and a security guard, the company exploited a thousand Indian remote workers.” A similar statement was published by the publication “Cold": "Amazon passed off the work of thousands of Indians as artificial intelligence," but in fact, "the cashiers were simply transferred to another place." Messages about the scam also spread on Telegram - for example, the channel “Ax 18+”(3 million views at the time of writing this analysis) noted: “It turned out that there was no artificial intelligence,” and “VPS” (500,000 views) wrote that “[Amazon’s] technology turned out to be fake.” “Shoppers were watched through cameras by a thousand low-paid Indians - they looked at what goods you took and made a purchase,” the publication says.

Technology Just Walk Out is a development by Amazon that allows offline stores to serve customers without cash registers and sellers. There are turnstiles at the entrance to the store; to enter you need to scan the QR code in the Amazon Go app. On the sales floor, the buyer can take an item from the shelf and put it in his bag or pocket - then the product is added to the preliminary shopping list (if the item is returned to its place, it will be removed from the list). When a person leaves the store through the same turnstiles, the application records the final shopping list, and after a couple of minutes sends an electronic receipt and debits the payment from the Amazon account.

From a technical point of view, such smart stores work something like this: at the moment the buyer scans the QR code, video cameras read and remember his physical characteristics, and motion sensors and microphones begin to track the person’s movement around the sales floor. A system of infrared sensors and weight sensors recognizes which products the visitor took from the shelf and put in the bag, and which ones he returned to its place. By information on Amazon's website, Just Walk Out combines the same technologies as self-driving cars: computer vision (a field of AI related to image and video analysis), sensory data fusion (building a single picture based on information received from different sensors) and deep learning (a type of machine learning that uses artificial neural networks to enable digital systems to learn and make decisions based on unstructured data).

Amazon started development of Just Walk Out technology in 2012. Four years later, the company opened the first test store for its employees, and two years later launched the first offline store for the general public. Later system implemented in Amazon Go stores, Amazon Fresh stores, supermarkets of other major retailers and convenience stores at stadiums, airports and other establishments in the US, UK, Canada and Australia. In 2018, Amazon founder Jeff Bezos in his annual letter to shareholders wrote that Just Walk Out technology is a way to “eliminate the worst part of physical retail: the checkout line.”

So, did Just Walk Out's technology really rely not on AI, but on thousands of people in India remotely observing shoppers and manually entering their purchases into receipts? For the first time, mention of a thousand Indians employed in the project appeared in the publication The Information in May 2023, but how noted Almost a year later, the authors themselves remained without much attention. They suggested that this could be due to "a lack of skepticism towards any technology related to artificial intelligence." The Information reported that the Just Walk Out project employs more than a thousand employees, mainly based in India. At the same time, an Amazon representative confirmedthat the company hired people in this country to watch the videos, but did not specify how many there were or what exactly they were doing. April 2, 2024, Amazon Vice President Tony Hoggett in an interview with The Information stated to stop using Just Walk Out in Amazon Fresh supermarkets. And when, in an article with an interview, the publication again mentioned the team from India in the context of some of the difficulties faced by the developers of Just Walk Out, then the detail about a thousand Indians received so much attention in the media.

However, in the publications accusing Amazon of deception, there is a large inaccuracy regarding why the Just Walk Out technology, in addition to AI, needed so many employees. AI system requires constant adjustments, and even the simplest neural network must first be trained on a huge number of examples with already labeled data. In the case of Just Walk Out, this could be, for example, pre-prepared footage of customers performing a certain action, and notes about what exactly the person is doing. This method of training a neural network is called “training with a teacher" That is, before AI can independently and qualitatively analyze data, a person must prepare thousands of samples with explanations for it. Speaking to Quartz, an unnamed former Amazon employee who worked with Just Walk Out said explained, that his Indian colleagues were engaged in labeling data for training an AI model and that this is a common practice in machine learning. Also a team from India rechecked videos of purchases marked by the neural network itself as “events with a low degree of confidence,” that is, the results of its analysis in which errors are likely. In such cases, people watched the videos and, if necessary, recalculated the amount in the buyer's check.

Originally Amazon assumed, that as the AI ​​model improves, no more than 5% of purchases will have to be double-checked. At the time of Just Walk Out's launch in 2016, human consultation was required in almost all cases, by mid-2022 the number of inspections had decreased to 70%, and in April 2024, by words former Amazon developer, dropped to 20–50%, which was still many times higher than the company’s original plan. At the same time, the Amazon press service in a commentary to USA Today stated, that “employees check only a small proportion of purchases” and that “from year to year, the number of double-checks by people is declining, despite the scaling of the project." Amazon spokesperson Sarmishta Ramesh responded to accusations that AI was being replaced by workers from India. objected, that these employees are only annotating video images in order to improve the model underlying the Just Walk Out technology.

Such support for artificial intelligence by humans is used everywhere, especially at the stage of training AI models. For example, OpenAI - creator of ChatGPT - in 2023, paid thousands of freelancers $15 per hour to write suggested answers to questions from chatbot users, which were then used to train AI. The company offered about the same amount Invisible for writing training conversations in which the employee alternately acted as a user and a neural network. For labeling data for self-driving cars, some freelancers received $1 per hour. Amazon even has a special platform for hiring such employees - Amazon Mechanical Turk, named after chess machine XVIII century, in which a living chess player was actually hiding, passing off his moves as the game of a smart machine.

Human labor is widely used not only to train AI, but also to back up and correct mistakes it makes. For example, Cruise's self-driving taxis are at the testing stage demanded remote intervention by a human operator every 7–8 km when “robot drivers” found themselves in a difficult situation on the road. And ChatGPT freelancers corrected chatbot responses flagged by users as low quality. Despite the rapid development of AI, it still relies heavily on vast human resources.

However, Amazon really decided to stop using Just Walk Out in its Fresh supermarkets for a number of reasons. According to a former employee of the company whom Quartz spoke with, management had been planning to do this for almost a year - the technology worked well in small stores with a small selection of goods and a flow of people, but it turned out to be unsuitable for large retail outlets. Weight sensors for tracking purchases, which bypassed cameras in reliability, were too expensive to manufacture ($350 each), and the rate of human cross-checks, although decreasing, was still significantly higher than the planned 5%.

In many publications about the closure of Amazon stores with Just Walk Out technology, not only the role of employees from India in the project was distorted, but also other inaccuracies were made. First, this technology is only leaving Amazon Fresh supermarkets in the US, but will still be available apply in Amazon Go stores, select UK Amazon Fresh locations and more 130 third party stores at airports, educational institutions and cafes. Secondly, Amazon Fresh stores in the US don't close, it’s just that Just Walk Out technology will be replaced by a new gadget - smart carts Dash Cart, which scan the goods they contain. Some Amazon Fresh supermarkets have been around for several years use both technologies simultaneously in addition to the traditional checkout method for non-Amazon customers.

Thus, the claims that the Just Walk Out technology turned out to be fake and that there was no artificial intelligence, but a thousand Indians worked instead of it, are not true. The employees (it’s not entirely clear how many there were, but most of them were actually in India) carried out the usual work for AI projects: marking up data, training the model, and correcting errors it made. The decision to discontinue Just Walk Out in Amazon Fresh stores was due to many inaccuracies in the AI ​​and the high cost of equipment. At the same time, the model has successfully proven itself in small stores, so it will continue to work in dozens of locations in the USA, Great Britain, Canada and Australia.

Cover photo: Wikimedia Commons

Half-truth

What do our verdicts mean?