AI has reached a milestone in the strategising department.
Just last July, a poker bot developed by researchers from Facebook’s AI lab and Carnegie Mellon University beat some of the world’s top players in a series of no-limit Texas Hold‘em poker games.
In a 12-day session with over 10,000 hands played, the AI system, called Pluribus, emerged victorious against 12 professional poker players in two different settings. First, Pluribus played alongside five human players; and in the other, five AI bots played against one human player. Take note that the bots were not able to collaborate with each other in this scenario. In the end, Pluribus won an average of $5 (£4) per hand, winning around $1,000 (£803) every hour. In total, the bot won a virtual $48,000 (£38,500).
Pluribus’ achievements are significant as it is the first to beat not just top professionals, but win in a multiplayer setting no-limit Texas Hold‘em game, which is considered the elite form of poker. Aside from five copies beating top players such as Darren Elias and Chris Ferguson in a 5,000 hand game, a paper published in Science describes how a single copy of Pluribus took on five human professionals for 10,000 hands, and won.
Pluribus learned how to play poker (and play well) by playing against copies of itself, a common technique in AI training known as self-play. It only took 8 days on a single powerful server with 64 processor cores equipped with less than 512GB of RAM for Pluribus to master the game. In fact, the AI bot has developed a “blueprint strategy” that it uses for the first round of betting. The bot projects the potential outcomes from particular points in the game, looking only a few moves ahead at a time instead of all the infinite possibilities, as it may take a more powerful computer to determine all the outcomes of a six-player game.
Winning against multiple humans counts as a milestone for AI, as no computer program has ever achieved this — something that the two researchers, Noam Brown and Tuomas Sandholm, can gladly take pride in. Pluribus’ forerunner, Libratus, was able to win against poker players, but only in a one-on-one situation. AI systems have already been developed in other games; The Guardian’s report on AlphaZero reveals that game-playing AI has beaten the world’s best chess player after teaching itself how to play in just under four hours, while another Google-invented system can play against Go players.
These, however, are just 2-player games, and given that all the information is available to the players on the board, it’s easier to see the possibilities and risks, in order to act accordingly. In poker, however, players are only given partial information, with the possibility of others bluffing — making it a much tougher challenge for both human and AI players.
So, given all this, what does a successful poker-playing AI mean for greater society?
Well, poker has many similarities with real-world situations, which is why it has taken so long for researchers to pull off such an achievement. Unlike chess, PartyPoker’s guide to Texas Hold’em explains that players must play without knowing what kind of cards their opponents hold, as is the case in politics, business, and even war. And given that the system uses less computing power than one that tries to compute every single possible outcome at different points in time, Pluribus is cheaper and more efficient to run, without losing its accuracy or how effective it is.
In fact, Brown says that the AI can be developed on a cloud computing service for just $150 (£122), making it easily applicable to other domains. This proves something that we’ve outlined on how ‘AI-Based Finance is the Future We Should All be Prepared For’: AI’s primary benefits involve an affordable business model, eliminating the need for thousands of employees for a task AI can complete in seconds.
In the long term, Brown and Sandholm are hoping that the methods they’ve demonstrated with Pluribus can be applied in domains such as cybersecurity, fraud prevention, and financial negotiations. “Even something like helping navigate traffic with self-driving cars,” says Brown, will be a great application.
