Last week we have peaked “Behind the Scenes of the US Poker Business”, which was an excellent “trailer” into this week’s special guest post, focused at: “a set of algorithms to quantify the effects of skill and luck in online poker”… how sexy does that sounds? WE KNOW! (clarification for Thomas: you can stop panicking and climb down from your desk – it’s algorithms, not alligators!)
So without further delays, here is Dave Thornton of “Skill in Games” and their approach:
—————————————————————————————————————————————
Analysis
We analyzed the first hundred hands played by ~1.1 million players in our database, and for each player, recorded his/her outcome, skill, and luck.
Outcome refers to money won or lost; skill quantifies how the player performed relative to the average, given the situations he encountered; and luck captures how much the cards helped or hurt. Mathematically, $outcome = $skill + $luck.
We used these 100-hand experiences to create basic cohorts, and calculated the increase in the average number of future hands played within each cohort (click on the table to enlarge):
We found these results both intuitive and intriguing. They’re intuitive because they show patterns you’d expect to see, e.g. better players play more. They’re intriguing because they hint at the possibility of better-constructed cohorts, and new categories of promotions and messaging.
Suggestions for possible testing
Higher-yielding reactivation cohorts. Dormant players almost invariably lost money in their last experience at the tables. Thus, reactivation is partially an effort in identifying which types of past losers are the best to bring back. This table shows that players who played well but got very unlucky played a lot more hands than other losing players. It follows that reactivation campaigns targeting this cohort should have meaningfully higher yields.
Tighter cohort construction. Whether you’d prefer to retain or reactivate high or low-volume players, it’s clear that cohorting by skill (or lack thereof) will do a significantly better job targeting the players you want than cohorting by money won (or lost).
New marketing messages. It’s become industry gospel that first experiences matter. Taking a bad beat or encountering a cold deck (e.g. say, running your KK into AA) within 100 hands of your initial deposit is, undoubtedly, a poor first. Why not turn a demoralizing time at the tables into a loyalty generator by offering new players a bad luck bonus? More generally, why not explore ways to recognize a player’s actual experience on the virtual felt? A pat on the back after solid play, or consolation after a bad run of cards, could go a long way towards helping a player feel appreciated.
We are just beginning to explore whether separating skill and luck can be valuable in the CRM context. If you’d like to continue exploring with us, we welcome further dialogue in the comments section.
About the author:
Dave Thornton is the founder and Managing Member of Skill in Games (SiG), a data vendor to the online poker industry. SiG takes a hole-card-up hand history, and breaks each player’s outcome into action-by-action skill measurements, and street-by-street luck measurements.
—————————————————————————————————————————————
PS: Meet us in Vegas! Or, meet us in Barcelona!! OR, even better – get a life!!!
- G2E, 24-26 September (Las-Vegas)
- EiG, 8-10 October (Barcelona) <- 10% discount if you register through us!
iGaming CRM 
Fascinating stuff – I’m convinced that a more “common sense” approach (ie. looking at customers’ emotional reactions based on their experiences) to tackling classic problems such as churn prevention will bear more fruit than just chucking a load of data at a clever statistician. Not just with Poker, but with casino games (trying to find “unlucky” customers based on RTP variances, burning through cash quickly, losing streaks) and also Sportsbook and Bingo.
As an industry I think the psychology of the player, and emotional triggers are often either not considered at all, or vastly undervalued. When combined with some reasonably standard analytics (OK, maybe some clever SQL!) the results have the potential to drive some pretty clever CRM.
I’m only just touching the surface of this in my current role but would love to chat more with you on it Shahar, maybe in Barca! Dave – will you be in Barcelona for EiG too? Keen to know more about how you derive your skill / luck metrics, and what services you may be able to offer.
Fascinating stuff – I’m convinced that a more “common sense” approach (ie. looking at customers’ emotional reactions based on their experiences) to tackling classic problems such as churn prevention will bear more fruit than just chucking a load of data at a clever statistician. Not just with Poker, but with casino games (trying to find “unlucky” customers based on RTP variances, burning through cash quickly, losing streaks) and also Sportsbook and Bingo.
As an industry I think the psychology of the player, and emotional triggers are often either not considered at all, or vastly undervalued. When combined with some reasonably standard analytics (OK, maybe some clever SQL!) the results have the potential to drive some pretty clever CRM.
I’m only just touching the surface of this in my current role but would love to chat more with you on it Shahar, maybe in Barca! Dave – will you be in Barcelona for EiG too? Keen to know more about how you derive your skill / luck metrics, and what services you may be able to offer.
Hadn’t made a decision yet. But if Shahar’s buying enough beer, that would certainly help the case for going.
There’s NEVER enough beer
Beer talk distracted me from responding substantively to your post. Sorry!
On your first point, we wholeheartedly agree. As we sift through the data, it’s increasingly clear that how players *experience* luck (and skill) has big ramifications for players’ ultimate longevity. We’d love to continue brainstorming that topic.
In terms of how we derive the measurements, and what services we can offer – I’ll give the tip-of-the-iceberg description here, further detail is probably best offline (I can be reached at info@SkillinGames.com):
Measurements: our algorithms require a hole-card-up hand history, and return, for each player in the hand, action-by-action skill measurements, and two types of street-by-street luck measurements. For a given player, in a given hand, these measurements have the relationship: [the sum of the player’s skill measurements] + [the sum of the player’s luck measurements] = [the amount of money the player won or lost].
Services: this one’s certainly open to some creative brainstorming. But for now, targeted marketing/CRM, new types of leaderboards and associated promotions, ecosystem management, educational systems more conducive to individualized feedback, and cheater detection. (We’re essentially a specialty data vendor that offers supplementary consulting and product development help.)
Sure thing mate – beer is on me :) Shahar
Now you’re talking!
Cheers Dave, sounds fascinating. I will give you a shout in the next few weeks and hope to meet up in Barcelona for that beer if you’re around.. Wayne