Amazon closing its Fresh and Go stores
(finance.yahoo.com)312 points by trenning 5 days ago
312 points by trenning 5 days ago
I left the following comment some months ago, duplicating it here:
[Disclaimer: Former Amazon employee and not involved with Go since 2016.]
I worked on the first iteration of Amazon Go in 2015/16 and can provide some context on the human oversight aspects.
The system incorporated human review in two primary capacities:
1. Low-confidence event resolution: A subset of customer interactions resulted in low-confidence classifications that were routed to human reviewers for verification. These events typically involved edge cases that were challenging for the automated systems to resolve definitively. The proportion of these events was expected to decrease over time as the models improved. This was my experience during my time with Go.
2. Training data generation: Human annotators played a significant role in labeling interactions for model training-- particularly when introducing new store fixtures or customer behaviors. For instance, when new equipment like coffee machines were added, the system would initially flag all related interactions for human annotation to build training datasets for those specific use cases. Of course, that results in a surge of humans needed for annotation while the data is collected.
Scaling from smaller grab-and-go formats to larger retail environments (Fresh, Whole Foods) would require expanded annotation efforts due to the increased complexity and variety of customer interactions in those settings.
This approach represents a fairly standard machine learning deployment pattern where human oversight serves both quality assurance and continuous improvement.
The news story is entertaining but it implies there was no working tech behind Amazon Go which just isn't true.
The go tech is amazing in 2 places: airport and stadium beverage tunnels. There's a premium price and high volume in those areas. The go tech has basically revolutionized the speed of getting a beer and a dog at the stadium here in Seattle. I can be back in my seat in 4 minutes including the bathroom now which for NFL means I can literally be back in a commercial break sometimes.
no idea how much they make on it, but it's a game changer in that small area.
One wonders just how much technology is needed to dispense a beer and a hot dog.
Couldn't you just use vending/automat machines in these scenarios? Beers in particular are... not complicated. I believe the go tech makes the existing situation better but if you were to reimagine it from ground up I can't help but imagine you could do better.
I get being proud of the work done but if they scrapped the project after 10 years because of feasibility I don't think the tech rolled out at the start was "working" as intended.
The first iteration of the tech reached the accuracy needed to support just-walk-out for a small-format store. It did achieve that goal. I left the project before it went further.
I imagined, at the time, future goals would be to scale store size and product variety while reducing the cost of the technology, but I have no insight into how that progressed. I am sorry to learn it's been shut down.
Could it be improved by requiring the customers to use a "smart" shopping basket that can read RFC codes from the product packaging? In combination with vision tech it should give a relatively higher accuracy.
If so, is the reason why it is not used related to cost?
As soon as you get to ~99% accuracy, you probably don't need to go further.
If the customer is accidentally billed for an orange instead of a tangerine 1% of the time, the consumer probably won't notice or care, and as long as the errors aren't biased in favour of the shop, regulators and the taxman probably won't care either.
With that in mind, I suspect Amazon Go wasn't profitable due to poor execution not an inherently bad idea.
Actually, discount grocers operate on razor-thin margins of 2-4%. If your inaccuracy is geared to the benefit of your customer (because otherwise you'll be out of business due to the regulatory bodies) and thus removes just one percent of that, you suddenly lose a quarter to half of your earnings! And that goes ON TOP of the additional cost incurred with all that computer vision tech.
In addition to that, you'll have the problem of inventory differences, which is often cited as being an even bigger problem with store theft than the loss of valued product. If the inventory numbers on your books differ too much from the inventory actually on the shelves, all your replenishment processes will suffer, eventually causing out of stock situations and thus loss of revenue. You may be able to eventually counter that by estimating losses to billing inaccuracies, but that's another complexity that's not going to be free to tackle, so the 1% inaccuracy is going to cost you money on the inventory difference front, no matter what.
And to add to that, it's not a neutral environment. If there's 1% of scenarios that are incorrect, people will figure out they haven't been billed for something, figure out why, and then tell their friends. Before you know it every teenager is walking into Amazon Fresh standing on one foot, taking a bag of Doritos, hopping over to the Coca Cola stand, putting the Doritos down, spinning 3 times, picking it up again and walking out of store, safe in the knowledge that the AI system has annotated the entire event as a seagull getting into the shop.
I don't have insight into what ultimately transpired at Amazon Go so take the following as speculation on my part.
It is unlikely the tech would be frozen when an acceptable accuracy threshold is reached:
1. There is a strong incentive to reduce operational costs by simplifying the hardware infrastructure and improving the underlying vision tech to maintain acceptable accuracy. You can save money if you can reduce the number and quality of cameras, eliminate additional signal assistance from other inputs (e.g., shelves with load cells), and generally simplify overall system complexity.
2. There is business pressure to add product types and fixtures which almost always result in new customer behaviors. I mentioned coffee in my prior post. Consider what it would mean to add support for open-top produce bins and the challenge of complex customer rummaging. It would take a lot of high-quality annotated data and probably some entirely new algorithms, as well.
Both of those require maintaining a well-staffed annotation team working continuously for an extended time. And those were just the first two things that come to mind. There are likely more reasons that aren't immediately apparent.
It's great that they faced essentially no consequences for this. A sure sign that we have a functional and sane market.
Why would they face consequences? Every store has video surveillance that can be reviewed.
They trusted their tech enough to accept the false-positive rate, then worked to determine / validate their false positive rate with manual review, and iterate their models with the data.
From a consumer perspective the point is that you can "just walk out". They delivered that.
If the stock price goes down, I won’t be surprised if there’s a shareholder lawsuit claiming that they misrepresented their level of AI achievement and that lead to this write-off by keeping operating costs and error rates high. The whole business model really assumed that they could undercut competitors by lower staffing.
Their initial advertising claimed near full automation by their "AI" system when, in reality, they had people manually handling around 70% of the transactions.
I get that this is a message board for YC, so lying about your company's tech is considered almost a virtue but that is an unreasonably big lie to tell without getting your hand-slapped by some regulatory body or investor backlash.
It's also pretty par for the course from Amazon automation initiatives. Like Glacier being marketed as robotic tape drive loaders, where in reality it is mostly just regular old S3 running on the outdated server clusters.
Isn't the consequence that that they're shutting the stores down?
It only takes 1 employee to staff 20 self checkouts for comparison.
For a full fat grocery store. With zero change or adjustment to the rest of the grocery store. And customers weirdly like self checkouts even when they are a dramatically worse outcome (compared to the highish bar of well trained cashiers)
If it's a publicly traded company, everything is securities fraud.
AI is not unique in this regard. We just saw the same thing with the crypto/blockchain nonsense.
Regulation lags so far behind that you can get away with bad behavior long enough that, by the time regulation catches up, you can buy your way out of consequences.
This was proven to be false on the WAN show. Only 20% of transactions were low confidence and handled by mechanical turk.
One transaction per minute is nothing at all when the transaction can be as simple as "did the person put that back on the shelf" with a 5 seconds clip.
Proven "false." I've noticed that if one admits the truth with a dismissive or offended tone, you can just continue to claim the lie and through sheer force of will people will still go with it.
I think people just think that they must be misunderstanding something; that nobody could claim one thing while offering evidence of its opposite. 1/5 of purchases lose their significance.
Nothing has been "proven". The original story was The Information (paywalled article) reshared by Business Insider [1] and claimed that 70% of the transactions were reviewed by an indian. The source was an anonymous source.
Business Insider also reached out to Amazon at the time and a spokesperson denied that actually reviewed any transactions.
This "proven false" thing is just another anonymous source claiming that actually it was only 20%.
So you actually have no proof of anything, you just have three persons claiming three different things (0%, 20% and 70%).
[1] https://www.businessinsider.com/amazons-just-walk-out-actual...
Transactions or grabs? Cuz I grab >5 things every time..so it stands to reason Indians always reviewed me.
I’m skeptical of this scoop.
It’s reasonable to expect a system like Amazon’s to use human feedback in training, and to quote the article linked on Wikipedia:
> Amazon said that the India-based team only assisted in training the model [and validating] a small minority of shopping visits.
I went to Lidl UKs first walk out shop a few weeks ago. You get the bill and receipts about 40 minutes after you've left.
It certainly felt like it could have been sent off to a lower paid country for a human to tot up.
Also consider you're in the store for what, 10 mins - that's a lot of video processing presumably using state of the art CV models. It's quite possibly cheaper to pay a human than rent the H100 to do it.
Why did "outsourced workers get (relatively) much more expensive after"?
Essentially the thinking went. If everyone is remote, why not hire remote workers from countries that are a lot cheaper. Suddenly you had a hard time finding contractors and FTEs from those countries because everyone was hiring them. At the same time it got really hard for entry level developers in the USA to find work.
The supply/demand curve shifted and now those workers are becoming more expensive while domestic workers are becoming cheaper.
India specifically is in the middle of a massive years-long labor movement that is changing the terms of work there and I believe shifting the degree of alignment with western corporate outsourcing though I'm not very informed about the details.
Scale is beyond comprehension though, there were 250 million people on strike one day last summer. This is not ever really covered in western media or mentioned on HN for reasons that are surely not interesting or worth pondering at all.
Beyond just the usual inflation?
I'm not an economist either, but I also assume that as the country attracts more local talent for local companies, the competition for outsourcing becomes harder. (i.e, you now have to pay more than the local companies).
All just speculation on my part though, I really have no clue either.
Another case where AI = "actually Indians". It's funny how often this has happened.
Maybe. I'd really want to know what percent of items (not transactions) needed review. 1,000 people to oversee how much revenue?
Theoretically if it was 99% computer and 1% human, that's enough to mess up the economics but it's not a bait and switch like some companies have done.
I remember this case the one who put "Actually Indians" in my mind. What other instances do you know?
(Not to refute your point, of course, I am just curious)
Their fate seemed sealed when it was revealed a bit back that the “just walk out” technology was more hype than substance. Just lots of people watching what you’re doing on camera vs an actual AI that worked well at mass deployment scale. A good idea, poorly executed.
Reports said the “AI” was largely 1000+ people in India watching the cameras.
If Amazon actually managed to build AI that worked well at a decent cost point it would have been great since nobody likes those silly self checkout machines.
What’s amusing about all of this is that before it got leaked that it was basically a bunch of people in India watching cameras Amazon folks spoke about the tech like there was some super secret AI they developed. Since that story broke nobody there seems to want to talk about “just walk out” anymore.
Even that didn't work well, when I was at an airport recently I had investigated 4-5 items as I had some time to kill. When I was walking out it wanted to bill me for 70 dollars even though I only had a bottle of water and a candy bar.
I have little trust that a corporate behemoth will do right by me and refund the discrepancy at an unspecified later time as it says it will on checkout.
This keeps me away from these sorts of stores if I can avoid them, which is pretty much always (so far, anyway). I would be absolutely shocked if the error rate was comparable to a normal checkout process and I don't want to waste the cognitive overhead of either wondering how much I'm getting ripped off by a corporation or having to go back and review and try to resolve overcharges.
Yeah, we had one near us, close to the metro exit, and it was genuinely great when you needed to grab something for dinner on the way home. Once you knew where things were, you could be in and out in 20 seconds. That said, it never seemed busy compared to other grocery shops in the area, so I think a lot of people were put off by it feeling "weird" to shop without checking-out.
You can use the Apple Store app to purchase physical items at Apple retail locations (smaller items like cables or cases). I've used it a couple of times, and I feel very awkward using it, so much so that I'll walk out kinda waving the receipt/acknowledgement screen around so that staff/security can hopefully see I'm not nicking something.
IIRC the Fresh near my old job required you to have a Prime membership, otherwise it was just a normal grocery store. I only went in there a few times, but I don't have a Prime membership, so there wasn't much of a point.
AI: Actually Individuals¹
¹ Individuals manning a labyrinthine system of cameras and sensor fusion, like hawks, logging the precise moment you plop a Twix into your basket! Praise Bezos!
There is no difference from the customer perspective so the store failed for reasons that have nothing to do with the "just walk out" technology or lack thereof. Why spend lots of money doing R&D only to find out that the concept doesn't sell? Wait for the product to be successful before spending the money to scale it up. Same as anything else.
"Do things that don't scale."
I think the idea could work well but the execution in the field was consistently very poor. There were a few of these at airports with just an intimidating gate and generally non-engaging human standing there.
It was as if they expected everyone to know what to do, but when I’d watch 99% of people just sort of looked at the store, saw the odd gate things, and then just shrugged and walked off. The stores were almost always completely empty amidst a busy concourse.
Even if the tech worked (reports say it didn’t work well) they completely missed the boat on creating a clear customer experience that navigated the new tech.
I agree, it needed a better hook to get people in the 'gates' so to speak. I don't think I've ever waited behind like maybe a single transaction at an airport convenience store, so it's not like having to fiddle with my phone to get in beats tapping a card or phone or watch at checkout. Either way most people are buying 1-3 things so it's not like it saved time scanning.
As for the big Amazon Fresh grocery stores, I only have one out of my way so I only visited once or twice, but the big things I noticed were that it had a small selection and very average prices. Not that surprising because even after buying Whole Foods, Amazon itself has terrible prices on dry goods (meaning supermarket items besides fresh food), and relies heavily on random third-party sellers with big markups for a ton of it.
If they really wanted to get people to buy into Amazon Fresh it would have taken a lot more money (and thus pretty unprofitable for a long while): Probably one way to do that would have been making it as attractive as Costco for Prime members.
I’m a bit surprised a publicly traded company is allowed to make materially false claims about their products and capabilities without getting into a major lawsuit for defrauding shareholders. Maybe Amazon is just above such trifling things such as law.
Do people really have problems with self-checkout? I use it all the time in box stores like Kroger, Walmart, Home Depot, etc. It seems to work just fine for me and doesn't add more than a minute or two vs just walking out of the store.
Here in Germany, the newer generation self checkout terminals are fine. I use them all the time. No issues. The first generation ones were terrible.
The issue with the first generation was that they were too strict with bag placement, weight sensors, etc. They were impossible to use without having to call a very grumpy shop attendant to unlock them. Sometimes multiple times. They were grumpy because all this was technically user error but when a largish percentage of users run into the same issues over and over again, it gets really annoying to deal with.
They fixed most of the glaring UX issues with the newer generation. No weight sensors. No prompts to put the item in the bag it is already in, etc. The new ones basically only need people to unlock things like alcohol purchases, but are otherwise fine. The first generation was over engineered and had way too many failure modes. They still have them in some super markets but they are getting replaced with better ones.
Anyway, it's getting harder and harder to hire staff for supermarkets. These are low wage jobs and most of these people can get better paying jobs. Self checkout creates some opportunities for shop lifting of course. But that is offset by the wage savings. They compensate with security, cameras, etc.
The bag weight sensors was something I was very happy to see go away. I hated self checkouts for years because it was a miserable experience of it freezing every third item and requiring someone to come get it working again. I only realized this tech had changed when COVID forced me to try self checkout again and it was suddenly a very pleasant experience, though one I have to imagine causes a lot of shrink for stores.
Self checkout is fine, if the happy path works. If everything scans once and doesn't accidentally scan a second time, if everything scans at the price you thought it was posted for, if you don't have any controlled substances requiring approval, if the weight sensor doesn't freak out incorrectly or from putting your bags on it, if it accepts any coupons you have, if it accepts and processes your payment method correctly.
If everything goes fine, self checkout is fine. But the exception handling process for any of those is thoroughly aggravating, as you wait and try to get the attention of the one overworked attendant dealing with a dozen of these machines constantly throwing exceptions, as the computer screams at you for whatever it thought you were doing wrong.
Yeah I agree that it can potentially go wrong but in my experience here in east TN the machines have gotten better to the point that hasn't happened for me in the past few years. Also it seems like the "just walk out" process would have more potential error modes but I never visited one.
If you find value in it, that's fine. I not only find value with interaction with the lovely checkout people, I dislike the cost of scanning and managing the items during checkout being my problem so a huge company can save money. If they were to implement a discount as a way to say "we'll pay you for your work to give us your money" I would consider it.
That's not to say the value of the convenience is never worth it. I exclusively use Sam's Club scan-and-go because the time I save is much larger than the publix/walmart/ect.
Yeah true. I do enjoy visiting with the cashiers but I don't love waiting in line.
> Since that story broke nobody there seems to want to talk about “just walk out” anymore.
Optimus and Robotaxi are just as fake and Elon Musk never shuts up about them.
I guess Amazon never learned the important lesson that the OP meta for modern technology companies is just to consistently and blatantly lie.
> On April 4, 2024, it was revealed that Amazon's "Just Walk Out" technology was supported by approximately 1,000 Indian workers who manually reviewed transactions. Despite claims of being fully automated through computer vision, a significant portion of transactions required this manual verification. ( https://en.wikipedia.org/wiki/Amazon_Go )
Wonder how much of this is due to economics since computer vision tech never reached the expected performance + outsourced workers got (relatively) much more expensive after COVID.