On Sunday, a team of bots developed by OpenAI defeated highly-ranked humans at Dota 2, one of the most popular, complex and highly paid e-sports games in the world. Dota, short for Defense of the Ancients, is a multiplayer online war game in which two opposing teams, made up of mythical characters called “heroes,” try to destroy the other’s home base.
In front of 100,000 live-streamers, the bot team easily took out the best-of-three tournament, winning the first game in 21.5 minutes and the second in just under 25 (evenly matched games generally take around 45 minutes).
Sunday’s tournament was a test run before the world Dota 2 championships later this month, where the bots will compete against a team of professional players. If the the OpenAI Five are victorious, they not only join the pantheon of machines that have publicly beaten humans at games (Deep Blue, Watson, AlphaGo), but will have done so in a game that balances several real-time interactions at once.
According to Greg Brockman, a co-founder of OpenAI, the company is working on designing bots that can beat elite human gamers because the simulated battle experience in Dota is so multi-faceted that it begins to approximate the chaos of the real world.
“We work on Dota because it is so complex,” Brockman said during a speech at the tournament on Sunday. “You have imperfect information, you have team work, you have these exponential combinations of different heroes and items, and you have to be able to deal with all of that.” Brockman added that to overcome these challenges, the bots had to be able to develop “intuitions” about their human opponents, improvise in response to unfamiliar situations, and collaborate with one another.
If the bots can learn to do all of this, the argument goes, they will be closer to being able to master complex tasks in reality. In a press release following the tournament, OpenAI said that they saw the victory as a “step towards advanced AI systems which can handle the complexity and uncertainty of the real world."
OpenAI, a non-profit research company co-founded by tech fixtures Elon Musk and Sam Altman, has been working on the Dota-playing bots for a year. The bots are powered by five artificial neural networks trained to play the game from scratch using reinforcement learning, an iterative process that involves observation, action and feedback. In preparation the bots played through 180 years’ worth of games against each other every day.
Human vs. machine competitions provide a way of demonstrating computational power while also provoking existential reflection about what differentiates us from machines. Just last year, Gary Kasparov published a book about his experience of publicly losing against Deep Blue. “If we feel like we are being surpassed by our technology,” he writes, “it’s because we aren’t pushing ourselves hard enough, aren’t being ambitious enough in our goals and dreams.”
After losing the first three games of Go to Google’s AlphaGo, Lee Sedol, a professional player ranked second in international titles, deployed a highly unusual strategy to win the fourth game, inspired in part by the idiosyncratic algorithmic playing style of the AI that he had observed during the previous three games.
The human team who took on the bots on Sunday were also granted new insight into their game. OpenAI had the capacity to predict its chance of winning during each stage of the game, as well as during character selection. After completing the first draft, the OpenAI system predicted a 95%-win probability, even though the matchup seemed more or less even to the audience, suggesting that drafting is even more important to strategy than previously thought by human players.
The humans won the final game, but only after the audience was allowed to select the bots’ player characters, which put them at a significant advantage. Still, the bots demonstrated that they have now learnt how to play the game well enough to defeat a team of five human players ranked in the 99.95th percentile.
While the victory on Sunday was a step closer towards OpenAI’s aim of beating a professional team at the world championships, their next goal—transferring skills from the closed world of a game to unpredictable real-world environments—is far more challenging. IBM’s Watson, who trounced humans at Jeopardy, has thus far failed to apply this intelligence to medical diagnosis, as the company had promised it would.
But OpenAI may be onto something with their Dota bots. Last month, the company released footage of a robotic arm manipulating a cube with uncanny dexterity that was trained using reinforcement learning algorithm as the OpenAI Five.
“It’s hard to know what kind of progress you’re making if you’re just making progress in simulators,” Jack Clark, who works at OpenAI, said in an interview with Axios. “But if I make progress on a real robotics task, I’ve done a no-bullshit reality thing.”