Ready to Bang My Head Against a Poll

            How many of you are sick of reading about polls?  Presidential polls, Senate polls, debate polls, local question polls.  This column is NOT political.  But since all the numbers you hear about polls are about math and statistics, many of the same types of numbers that are used in gaming, I thought I might give a quick lesson on interpreting some of what you hear.  The idea for this week's column came when a friend of a friend of mine remarked about two recent polls.  One showed Candidate A leading by 11 points and the other by 4 points.  Both had a margin of error of 3%.  He declared that one of the polls clearly had to be wrong.  WRONG!

            In order to understand any of these numbers, we need to define what some of these numbers mean.  First of all, what is a 'margin of error'.  Many people think this means that if we say candidate A is leading 49% to 45% with a margin of error of 3%, that Candidate A is leading by 1-7%  (4% +/- 3%)?  In the case of a poll like this where there are only two choices, this means that the 3% belongs on both sides.  So, 49-45 might mean 46-48 or 52-42.  That's a BIG gap.  But, this isn't really what margin of error means anyhow.

            Missing from the equation is a very, very important other number called 'confidence interval'.  From what I've read, most political polls use a 95% confidence interval.  But, unless they specifically say they do, we can't even be sure of this.  So, what is a confidence interval?  It means that if the poll were repeated, how likely will we get results within the margin of error.  So, if we go back to that poll that said 49-45% with a 3% margin of error, this means that if we were to do the poll again, we have a 95% probability of having it end somewhere between 52-42 and 46-48%. 

            So, that still leaves us with a pretty big gap and quite frankly, 'only' a 95% chance of it fitting in this range.  The other 5% of the time, it may go outside the range.  How far?  We don't really know for sure.  Is it likely to be 80-20%?  Probably not.  There is math that gives us some idea of the range, but these numbers are not told to the general public.  But, back to the post from a friend of a friend.  If one of these 2 polls that were being cited happened to have one of the 'other 5%' occur during this sampling than the poll is not 'wrong', it simply had one of the outliers occur during this sample. 

            Of course, all of this ignores any particular bias done during a political poll.  Political polls are far more complex than calculating payback.  Payback doesn't change until the paytable changes.  Political opinions can change as the wind changes.  In the case of those two polls, if they were done just a few hours apart after some news had been released that would be further cause to believe neither is 'wrong', but rather reported after additional information had been factored in.

            As a gaming mathematician, my world is generally more static than that.  Yes, there are countless games out there, but each has its own model.  It doesn't change from day to day unless something known changes about the game (i.e. the paytable or the rules).  Of course, gaming math can be a bit more challenging in that the decision is generally not binary.  You don't pick Candidate A or Candidate B.  You don't pick Yes or No on a referendum.  For example, in video poker, there are 9 possible outcomes - Royal Flush, Quads, Full House, Flush, Straight, Trips, Two Pair, High Pair, Nothing.  Unlike an election where it is one person, one vote, the same is not true for gaming results. 

            But the math terms are still in play.  When we say that a game has a payback of 99%, we are talking the long term.  But how long is the long term?  This depends on the game.  How likely are we to achieve the payback in that long term?  What about short term results?  These are all great questions which could be answered with numbers like our polls.

            If you play video poker for an hour (800 hands), I might be able to say that the payback will be 98% with a 95% confidence interval and a margin of error of 2.2% (I just made up all these numbers for illustrative purposes).  Because the Royal accounts for 2% of our payback, these numbers might be absurd.  The 800 hand sessions that have that Royal will have HUGE paybacks.  All those that don't will be adjusted downward relatively speaking.  There are methods for reporting these numbers, ranging from math models to simulations.  The more hands used in our sample the tighter the range becomes.  That is that the margin of error decreases and the confidence interval goes up.

            For example, if you get $5 free play and play jacks or better 25 cents at a time (hence 20 hands), the margin of error is going to be HUGE relatively to our 800 hand example.  The confidence interval and margin of error work together.  I can lower the confidence interval and then keep the margin of error smaller.  Or I can raise the confidence interval and the margin of error will increase.  Or, I can increase my sample size and the margin of error can decrease while the confidence interval stays the same.

            For example, if you would like to know the stats after playing 100,000 hands of jacks or better, then the payback I give you will be closer to the long term theoretical payback and the margin of error will decrease.  After all, if you were to look at multiple samples of 100,000 hands, you'll find that the real paybacks are in a much more narrow range than if you compare samples of only 1000 hands.

            In the political polls, these margins of errors would get smaller if the polling companies were to poll thousands of people instead of hundreds as most do.  Of course, that would probably cost them that much more, so they (and the public) seem content with these polls with what seem to be rather large margins of error.  But that doesn't make any of them wrong.