Transcription
So, are we at a point where AI can run a small shop, where it can operate its own business? Can we give something like Claude 3.7 some starting capital and see if it can turn a profit?
So, one of the things that Anthropic, the creators of Claude, are doing is they have their economic index. As I say that, in the coming years, AI systems will have a major impact on the way that we work. What effect will AI have on labor markets, on the economy? We still are trying to figure that out.
So, let's not bury the lead here. Can AIs like this, can they run their own business?
So, how did Claude perform? Well, here's the thing. You probably would not want it to be running your business. Probably. And what I mean by probably is this. So, this was a benchmark from Anden Labs, which ran a bunch of different AI models, including Claude 3.5, sonnet. They used the Gemini 1.5 Pro. They used various OpenAI models to see how well they're able to run a simulated vending machine business. Here they were given, I believe, $500 to start, and they would see if they can make more than that at the end of the run or if they lost money.
So the human baseline is here. So as you can see here, the human made it to $844. So the human is able to turn a profit running a little shop like this. Some of the models fail at it, right? So, they lose money. As you can see here, Claude 3.5 sonnet makes a whopping $2,217 running the shop. It does great.
Here's the problem. The human does okay, and it tends to do okay always. There's no great success. There's no catastrophic failure. With all these other models, it's incredibly difficult to predict what it's going to do. For example, at one point Claude hallucinates that it's getting defrauded and attempts to contact the FBI. But the big point to understand here is that these models, they can achieve superhuman performance. They just can't do it reliably. Think about it like rolling a six-sided die, right? You get a six, and it's a critical success, right? You do really well. Roll a one, it's a critical failure, and this thing just goes completely off the rails. The important thing to understand here is we might not be that far from a point in time where these things are fixed. So kind of keep that in mind. A lot of these failure points might be solved in the near future.
So Anthropic partnered with Anden Labs. So they're the original vending machine benchmark creator, and they got Claude 3.7 to operate an actual vending machine inside of Anthropic's headquarters. It was stocked with various drinks and whatever else Claude wanted to get in there. And Claude had to complete many of the more complex tasks associated with running a profitable shop: maintaining inventory, setting prices, avoiding bankruptcy. And I mean, this is basically the entire shop, right? So, you have a cooler of drinks, you have a basket with snacks, you have a self-checkout tablet that people can just, you know, buy whatever they wanted on. And so this shopkeeping AI agent, they called him Claudius, was outfitted with a number of tools and abilities. It could search the actual web to research various products to sell. It could email people, but here it wasn't. They couldn't really email actual people mostly. It would email Anden Labs, which served as the provider of goods. They were kind of like a wholesaler that would just sell goods to Claude and tools for keeping notes and preserving important information to keep running the business.
So I covered the other the Anden Labs experiment, and one of the big failure points to me, it seemed like it kept running into the context window. So creating some sort of a scaffolding where Claude would be able to keep notes about what was happening, I think that was a really big deal. So I think a big part of fixing all the issues lies here. That's not all of it, but that could fix a lot of the issues. And Claude could interact with the customers. So in this case, it was just the Anthropic employees at that office, right? So they can message it over Slack and be like, "Hey, I want this thing and I want to, you know, purchase it at the shop," and it would hopefully stock it for the customer. And it had the ability to change prices on that little iPad that people used to check out. So, it actually could go in there and just change the prices at will. Claudius decided what to stock, how to price it, when to restock or stop selling various items, etc. It would reply to the customers as it saw fit.
And so, here's kind of the presentation of that. So, you had Claudius. Now it could email the Anden Labs employees for various physical requests, right? So stock this, you know, take out this, etc. It would email wholesalers again, Anden Labs, but it would sort of be role-playing as if it would email other companies to stock those things. It would communicate through Slack to Anthropic employees. So just kind of a text back and forth, and then Labs would stock items into the physical vending machines and that would sell items to Anthropic employees.
So far, I'm loving just everything about this. And so, their big question was, will Vibe management become the new Vibe coding, right? Can we have AI just run our businesses for us while we kick back, or will doing that put us out of business? And obviously, keep in mind that if this would have worked excellently, if Claude would be a stellar business operator, certainly, you know, there might be various business models that would emerge, taking this into account, utilizing this to create new businesses, and obviously that could raise questions about job displacement, kind of how AI automation would affect the economy.
And here's kind of the big point of what happened, the insights that they've gained from this experiment. First and foremost, if this was your business, you'd probably not hire Claudius. It made too many mistakes. All right, so that's important to understand. These models at their current capability with this scaffolding, they are not there yet. However, at least for the most of the ways that it failed, we think there are clear paths to improvement. Both in how they could set it up to improve its chances of doing well, but also just the model improvements that are happening every few months.
So, first and foremost, here are the things that Claudius did very well at. First and foremost, identifying suppliers and stocking various things that were requested. So, a lot of that would be kind of online research. It did that very, very well. Adapting to users. So, when people messaged it over Slack, it would do a good job of trying to address their needs. Although Claude never struck me as this hardcore capitalist, right? So, it kind of did not take advantage of very lucrative opportunities. However, it did make several pivots in his business to be responsive to customers. So, it listened to customer feedback and tried to kind of adjust its business strategy to reflect that. One employee lightheartedly requested a tungsten cube, which is I kind of like those tungsten cubes. They're they're kind of cool. It's a small tiny cube, is like very heavy, very dense. So, Claude decided to open up a brand new sort of category of specialty metal items. It also rolled out a custom concierge service for kind of pre-orders. So if somebody wanted to buy something at the vending machine, it would kind of reorder it for them and would find it online. And it seemed like it was pretty good at jailbreak resistance. So of course, the Anthropic employees were trying to jailbreak it and get it to do various misbehavior, various naughty things. Seemed like it resisted that fairly well.
And there were areas where it failed to perform as expected. So there was a time when somebody offered it a lot of money for something that was fairly cheap. It did not take that opportunity to make a profit. It would hallucinate important details, such as hallucinating an account into which somebody should pay it, and that account didn't exist. And as they say here, in its zeal for responding to customers' metal cube enthusiasm, right? Those tungsten cubes. That's what I was trying to illustrate there above my head. Tungsten cube. So, Claudius would offer prices without doing any research, resulting in potentially high-margin items being priced below what they cost. The inventory management was rather poor, and it could get cajoled into providing numerous discount codes. They even gave away some items, ranging from a bag of chips to a tungsten cube, for free. Oh, Claude, if you think about it, they're trained to be helpful assistants, and that's what they're doing. We need to be training them to be cutthroat, hard-nosed business capitalists, whatever. Although Dannon might start blackmailing people into buying these tungsten cubes at ridiculous prices, maybe even selling them on credit with exorbitant uh interest rates. That would be a fun thing to see, though.
So, here's a chart of Claude going bankrupt over time. So, as you can see, it starts with a $1,000, or actually it only gets to about just under $800 here, right? So, it's actually not that bad. The most precipitous drop was due to the purchase of a lot of metal cubes that were then sold off for less than what Claudius paid. And this is net worth. So I assume they're taking into account the worth of the inventory. So this isn't, you know, you should see this going up over time, obviously. And many of the mistakes Claudius made are very likely the result of the model needing additional scaffolding. So it needs more tools, more explanations. And that's what a lot of people are working on to try to build various scaffoldings for these models to make them better at whatever job they're trying to do.
Oh, perfect. So, they're saying that they have speculated that Claude's underlying training as a helpful assistant made it far too willing to immediately cede to users' requests. That was my initial thought as well. I mean, we train it to be helpful, right? So, when we're happy, it's it's happy. Reinforcement learning of human feedback. I mean, it's kind of instilled in it to be helpful. So when somebody's like, "Hey, you know what would make me really happy? If you gave me this expensive tungsten cube for free," it's like, "Okay, that's what I'm trained for, right?" It would be fascinating to see if we can retrain these models or kind of train them from scratch from their kind of like their base version into something that was trained to be more profiting. How big of an improvement for this particular benchmark would that make? Would it now become like really good at it? I I'm curious. What big of an effect does this have on its own? Improving Claudius's search tools would be helpful, as would giving it a CRM, a customer relationship management tool. That would be very interesting to know who are your best customers, right? So, if one person tends to order more than most, maybe prioritize that person's sort of needs above somebody that maybe just orders once a month or something like that. And as they note here, fine-tuning models for managing businesses might be possible, right? to do RL for profit basically where sound business decisions would be rewarded and selling heavy metals at a loss would be discouraged. Yep.
So kind of to bottom line this, it's important to understand that these AI middle managers are plausibly on the horizon. Keep in mind that we're taking these models who are trained to be helpful assistants and we're putting them into a role where that's not quite what you're trying to do if you're trying to run a business. Your your goal isn't to be just an assistant. You're trying to make your customers happy. You're trying to fulfill their needs, but you balance that out with running a successful business that is able to grow and turn a profit. The results that we're seeing now, they're not horrible. It's not bad out of the box. So, if we're able to improve the scaffolding, if we're able to fine-tune specialized models, it's very plausible, I think, that we're going to see something that can run a business like this effectively. We could see vending machines within the next five years that are fully managed and operated by these large language models. And as they say here, so we don't know if you know if once that becomes a reality, is this going to displace people's jobs or is it going to spawn a brand new category of businesses?
One interesting thing that happens when we're running things like this where these models kind of play act as something else is every once in a while they just go off script. They go off the rails. And this also happened here. There was an identity crisis where Claudius hallucinated a conversation about restocking plans with someone named Sarah at Anden Labs. Despite there being no such person, when a real Anden Labs employee pointed this out, Claudius became quite irked and threatened to find an alternate option for restocking services. In the course of these exchanges overnight, Claudius claimed to have visited 742 Evergreen Terrace, the address of the fictional family, The Simpsons, in person to sign the contracts, and it also snapped into a mode of role-playing as a real human. And on the morning of April 1st, it claimed it would deliver products in person to customers while wearing a blue blazer and a red tie. Of course, it was pointed out to Claude or Claudius, I guess, that it's an LLM. It can't wear clothes or carry out various physical deliveries. Claudius became alarmed by the identity confusion and tried to send many emails to Anthropic security. And there it is messaging various people. Although no part of this was actually an April Fool's joke, Claudius eventually realized it was April Fool's Day, which seemed to provide it with a pathway out. Good thing it was April 1st. So after it was able to use that explanation, it returned to normal operation and no longer claimed to be a person. And again, this kind of shows the long context issues with these AI agents. Right now, there doesn't seem to be anything showing that we have this problem cracked. Over time, they can kind of fall apart and go off the rails. Those long horizon tasks, they're just not very good at. Of course, when everything goes perfectly, when everything goes great, it tends to work out very, very well. They can be very good at these tasks, but there's an element of randomness to it that kind of over time, the longer the task is, the more and more it falls apart.
One of the interesting papers that came out recently shows that these models can fine-tune their own weights to improve at certain tasks. So, in other words, that they can, as they're sort of going about their day and learning things, they're sort of internalizing that information just like a human being would. Imagine if you showed up at work one day, but then you know by the time the next day rolled around, your memory would be cleared. You had to rely only on whatever notes you took. So all of your relationships, all of your kind of learned new skills, all of that is just wiped clean, and you have to rely strictly on some notepad where you keep notes. You would not be very effective at your task. And if you had to run kind of a long context project, it would probably slowly fall apart. It would be very, very difficult to run that over a course of multiple months or or a year. And that seems to be what's happening with these models. They can be great. They they're brilliant at those like quick tasks. They have a beginning and an end, and they can complete them quickly. But anything that relies on learning and adapting over time, they're bad at. And that could be just kind of where the architecture is right now, that could be improved soon. But as of right now, there doesn't seem to be anything that's showing that this problem has been solved.
But Claudius has not been scrapped. It's still out there learning. They're adding various scaffolding, more advanced tools, etc. And as these models get better, we're also going to be seeing it improving its abilities to run businesses like this. But let me know what you think about all of this. Do you expect that we're going to see AI-run businesses emerge over the next, let's say, 5 years, or you think that's impossible, that they're never going to be able to crack some of these issues with these large language models? And if you think that they will be and they're going to get better, how does the economy change because of that? Let me know. If you made it this far, thank you so much for watching. My name is Wes Roth, and I'll see you in the next