Eric Jackson is a computer scientist with an interest in poker AI. The following is the third and final part of our interview, which has covered both the theoretical and practical aspects of poker-playing bots. The previous parts are available here: Part 1, Part 2.
Alex: My experience with AI in general is that computers’ approach to games is distinctly different from the way a human thinks. Playing poker with other humans, we tend to rely on psychology a lot. Do you think part of the difficulty people have against your No-Limit bots is just lack of familiarity with the way a computer “thinks”?
Eric: That’s probably part of it. For one thing, the bots often employ a much larger range of bet sizes than a typical human player. They’re capable of making both very large bets and very small bets.
I don’t think that’s all it is, though. Even if they’re theoretically exploitable, they are very strong. My best guess is that even with lots of practice most humans would still have trouble against the latest generation of No-Limit bots. Of course, this is largely speculation on my part since we still haven’t had a top player step forward to challenge one of these bots or play enough hands with them to prove anything one way or the other.
Alex: Even for humans, poker is radically different now than it was ten years ago. A ton of mathematical analysis has been done on the game, and a modern professional makes moves that would have been unheard of in the past, for example light 3-, 4- and 5-betting preflop, etc. Do you think bots’ play resembles theirs more than old-school tight-aggressive poker, or is it a third thing entirely?
Eric: Possibly, although it would be hard to do a direct comparison without a lot of data. Certainly bots are capable of making large bluffs or semi-bluffs in spots where old-school players would typically not be bluffing very much, which sounds like the sort of thing you’re talking about.
Alex: That being the case, do you think the bots’ approach to the game is something humans could learn from – for instance the wider spectrum of bet sizes you mentioned?
Eric: Yes, I think so, although you’d have to be careful how you went about it. It’s not really practical to memorize what the bot does, except maybe for a few straightforward preflop situations. In most situations, the bot has a very mixed strategy, taking one of several possible actions with varying probability, rather than always doing the same thing. So you couldn’t just copy what it did one time, and the number of options it chooses from in each situation makes its overall strategy very difficult to memorize.
One could certainly imagine practicing against a bot. If nothing else, you should learn how to cope with unusual bet sizes and maybe how to integrate them into your own game.
You could also learn something from aggregate statistics. For instance, maybe you notice that the bot check-calls the river about 60% of the time in a given type of situation. If that’s the case, then maybe you should likewise be check-calling about 60% of your range in that kind of situation, even if you don’t know the bot’s exact probability of check-calling there with one specific hand or another.
Alex: As I understand it, this has so far been an academic interest for you. Do you have any commercial plans for your bots?
Eric: Not in the immediate future, no.
Alex: But do you see any practical (and legal) applications for this technology?
Eric: I think it would be cool if people had the option of playing online against bots. To be clear, what I’m envisioning is that the bots would be clearly labeled as such, not passed off as human opponents of course.
I’ve heard that in Las Vegas, they already have a machine that lets you play rake-free heads-up Limit Hold’em against an AI. I think it would be neat if we could advance the No-Limit bot to the point that you could have that as an option as well.
People have also talked a lot about using bots as personal trainers. Hone your game by playing against a bot for play money. That seems like a promising avenue as well.
Alex: Of course, the big question on many reader’s minds is whether you have any thoughts about illegal bots. Based on what you know, how big of a threat do you think they are currently, and in the future?
Eric: I know of course that there are people out there trying to hook their bots up to online sites and have them pose as humans. I don’t know how many people are doing that or how good the sites are at catching them.
Just based on what you read on various internet forums, it sounds like the more established sites like PokerStars are pretty vigilant about fighting bots, while the more fly-by-night sites are kind of indifferent.
In terms of the strength of the AI, I suspect the bots are going to continue getting better and it won’t be too long before there is no game at which they can’t beat a human. Sites may have to introduce counter-measures like Captchas, but then you still have the possibility of a human operating the poker client but taking instructions from a bot.
So, I don’t know, it seems like the sites have a difficult uphill battle in the future. But just looking around, online poker is definitely not dead yet. So whatever the sites are doing seems to be keeping the problem in check for the time being.
Alex: You said that your bots’ play is unusual and doesn’t resemble the way most humans play. Do you think sites could defend against bots – even those with a human middleman – by analyzing hand histories for signs of bot-like decision making?
Eric: I think that’s probably a fruitful countermeasure, but of course the bot makers could try to disguise their tracks. Right now my bots employ unusual bet sizes, and I think most other bots do too. But if that’s what you were looking for, the bot operator could easily put an upper and lower limit on its bet sizes and it would still play pretty well.
With enough hand histories you could probably find subtler patterns in bot play. Maybe in a certain situation it bets pot with exactly these seven hands and no others. That sort of thing would be harder for the bot-maker to disguise, especially if he doesn’t know what you’re looking for. But you would need a large sample of hand histories before you could start to identify that kind of pattern.
Adaptation is another thing, or rather lack thereof. As I said earlier, the best bots we have today don’t adapt. So you could potentially catch a bot by identifying situations that almost always cause a human to adjust, and then looking for players who are not adjusting to those situations.
Alex: Can you think of anything else that would help?
Eric: There are other sorts of things sites are probably doing already. Looking for suspiciously even timing, for example. Or I think that some sites also track your mouse movements to see if they look natural, or whether you’re always clicking the button in the same place.
I think these kinds of things are helpful in combination with other methods of bot detection, but of course they don’t help if you have a human middleman taking instructions from a bot.
Alex: I think that about wraps it up. Thanks for your time.
Alex Weldon (@benefactumgames) is a freelance writer, game designer and semipro poker player from Montreal, Quebec, Canada.