You see them on the internet all the time. No, not cat videos. I’m talking about online polls. And they look a lot like the one above from Entertainment Weekly. So what’s so special about online polls, you ask? Answer: We’re doing them wrong.
Take a look at the Entertainment Weekly poll. The first thing we see is that it uses radio buttons. This means the respondent is required to choose only one selection. Choose another option and you switch the selection. What could be wrong with limiting people to select one movie here? When you force people to choose only one, support artificially divides between options, particularly among similar options. The product is a polling result that can be practically meaningless.
Using online polling wrong skews the data
Let’s play this poll out. I like comedies, so there are a number of good options. And although it’s not a stereotypical Christmas movie, I also like Die Hard (unlike the newer Die Hard atrocities). But if I was forced to pick one option, I’d have to go with Scrooged. Hey, it’s Bill Murray. Now for the results: A Christmas Story wins, with Love Actually just behind. My personal favorite, Scrooged, came in … good grief … last with just over 3%.
Perhaps it makes sense that A Christmas Story won, although It’s a Wonderful Life beats it by over half a point on Internet Movie Database (IMDB). But why did Love Actually come so close to winning? After all, Love Actually’s IMDB score is a full point lower than It’s a Wonderful Life, which it beat.
Aside from Die Hard, Love Actually, a romance comedy, is the most dissimilar to the other options. Any support it got was less likely to be divided. Conversely, there were three classic movies that could divide support, and there were several other traditional comedies (naturally explaining why Bill Murray’s film got such little support!).
This artificial division of support among options—what we call vote splitting—is a trademark of choose-one voting and polling. It can flat out give you the wrong answer, and it reliably distorts the results of the remaining options, especially when the options are remotely similar.
There is an exception when it’s okay to force one selection in a poll. You should limit the respondent’s selection when the options are mutually exclusive, i.e. when it’s only feasibly possible for the respondent to have one answer. An example might be, “How many times have you eaten out this month?” If the options are 0-2, 3-8, and 9+, then it makes no sense for someone to have eaten out both between both 0-2 as well as 9+ times. It’s okay to limit the respondent to selecting one option here. Polls with only two options are also an acceptable use of choose-one polling.
I’m not picking on Entertainment Weekly here with their poll. This style of polling is ubiquitous on the Internet. You’ll unfortunately find nearly everyone using it from Mashable to even Lifehacker. Now how do we avoid this mistake? Many polling services allow you to pick multiple options, using checkboxes instead of radio buttons. This switch from a single-selection to an unlimited selection technically switches us from a plurality voting method to approval voting — an enormous improvement.
When we do this, we wind up with what we see to the right: a poll with checkboxes. Permitting the respondent to choose multiple options sidesteps the vote-splitting issue. Like similar options? Choose each one. 
With this approach, we can be more confident in the winner, and alternatives that don’t top the leaderboard still get their fair share of support.
If you’re feeling extra feisty, you can take this to the next level and allow respondents to score their options. You can see an example of this below. Technically, this is called score voting. This permits maximum expression.
So please, internet, stop doing choose-one polls, and use approval polls! It hardly ever makes sense to limit respondents to one selection. Use those checkboxes or allow for ratings so we can get online polling results that actually make sense.
Richard Fobes says
Great article Aaron! It explains a very important concept that more people need to understand.
If we used Approval voting in primary elections then we, the voters, could elect the problem-solving leaders we want — instead of getting stuck with special-interest puppets.
As you say, let’s start by teaching online polls administrators how to eliminate vote-splitting.
SteveC says
This annual Gallup poll for “America’s Most Important Problem” seems also to suffer from vote splitting:
https://www.gallup.com/poll/1675/Most-Important-Problem.aspx
Aaron Hamlin says
You’re totally right. Ugh!
Michael Ossipoff says
Hi, Aaron–
Thnks for trhe article.,
Let me point out the other humungous fault of political polls: They ask “Which candidate would you vote for if the election were today?”
…so people, in the poll, cast the same lesser-of-2-evils strategic vote that they’ll cast on election-day.
O course our public elections have a problem that’s a lot worse than the voting system: It makes not the slightest difference what the voting system is, as long as the vote-count result is unverifiable. The first thing that everyone must demand is verifiable vote-counting.
Without it, the elections will remain illegitimate.
And, with illegitmate elections, we have no democracy. I mean _zero_ democracy.
Yes, as I was saying in my first article at Democracy Chronicles, Approval would be a tremendous improvement over Plurality. The combination of Approval and verifiable vote-counting would be completrely sufficient for genuine democracy.
And Approval has the advantage of easy and inexpensive implementation.
In fact, Approval is supported by many or most website polling services or facilities. Lots of them give the poll-starter the option of allowing people to give a point to as many options as they want.
So, of course, whenever I’ve done a poll at such a website, I’ve made it an Approval poll.
But, as my favorite, I prefer to do my polling at the Condorcet Internet Voting Service (CIVS), because it supports the best rank-balloting voting-systems.
Although it’s practically impossible to improve on Approval for our political elections under current conditions (dishonest, disinformational media, the best rank-balloting methods could greatly improve on Approval under more favorable conditions. (Check my articles “Voting Systems for the Green Scenario” and “Kinds of Voting Conditions, with Recommendations”).
I don’t believe that any improvement or reform is possible under current conditions. So political voting systems for current conditions is a pointless subject. I suggest that a more relevant topic is, voting systems for when a better party, a democracy-supporting party, is already in office, and the media are open and honest, and the public are non-gullible (as they’d have to be, to elect such a party via Plurality). I call that “the Green scenario”, because I used to think that the Greens were the most feasible progressive party (but I won’t go there today).
Fortunately, the Greens, along with every party that offers voting-system reform, offers, in its platforml, a voting system that would be excellent for the Green scenario: Instant Runoff (IRV).
(I _don’t_ recommend IRV for current conditions–As I said, for current conditions, it would be difficult to improve on A;prov al or Score–though the rank-balloting method ICT would be some improvement).
IRV meets the Mutual Majority Criterion and the Chicken-Dilemma Crition (it has no chicken dilemma). That’s a very powerful combination of properties
Benham’s method and Woodall’s method bring even more improvement, because they also meet the Condorceet Criterion. In fact, they meet the stronger Smith Criterion.
For current conditions: Approval or Score, or ICT if you can get it. (but voting reform under current conditions is impossible anyway.
For the Green scenario, IRV, Benham’s method, or Woodall’s method.
For ideal majoritarian conditions (where there’s no chicken dilemma), the best methods are MAM and Beatpath (Schulze). MAM is the better of those two, for reasons that I’ve discussed in my articles.
For my polling at CIVS, I use MAM, because poll results show that there’s no chicken dilemma in CIVS polls.
But if your electorate is contentious or inimical, then I recommend that you do your CIVS polling by Condorcet-IRV (Benham’s method).
P.S. I can’t stand Bill Murray, because of his blatant egotism.
Michael Ossipoff
Aaron Hamlin says
No doubt this problem is pervasive and faces a tough battle for reform.
Really, not a Bill Murray fan? Oh jeeze.
Michael Ossipoff says
No matter what I judge to be the chance of success, I still greatly appreciate the people who are making the effort, to make things better.
——————————–
Bill Murray is/was ok later in his life, when he was older. …like in that movie in which he’s searching for his son or daughter. I only meant to criticize his earlier roles.
What I couldn’t stand was his earlier roles, like in Ghostbusters, or on Saturday Night Live. He just seemed to be putting too much of his own personality into the roles, being so similar in all the roles.
Michael Ossipoff says
Just a few additional comments about voting on movies:
I like Approval for voting on movies–Academy-Awards, or small groups choosing a movie to go to or play. Voters should be instructed to approve only those movies that they consider to be excellent.
What could be better than choosing the movie considered excellent by the most people.
Of course MAM lets you also choose _among_ the excellent options that you’re voting over all the others–But, if a group of people vote on the choice of a movie, then a MAM hand-count could likely be unfeasiblly time-consuming, as could setting up a private online poll. And most people won’t have on hand a machine already programmed for that count.
And, for something like the Academy Awards, with infinitely many ways to count rankings, arguing for one particular best way would be quite a taskk.
So Approval is probably the most feasible way.
But Approval, like most voting systems, becomes tie-prone when there are very few voters. Not a problem. If Approval returns a tie, then use Score to choose from among the tied options. It seems to me that sincerely-voted 0-100 Score is the most tie-free voting-system, when there are very few voters.
Of course, decisions and choices in households, and among other small groups, tend to be intuitively be done by what effectively amounts to Approval and Score, even when no voting system is explicitly used.
Something is chosen that’s acceptable to or approved by everyone. When there’s more than one such, then it matters how much the various people want what they advocate.
Only once have I done a formal vote among movies. I was in a group of four people who’d decided to go to a movie that evening. That was a long time ago, and I used the simple Condorcet rank-method known as MinMax (My name for it in those days was “Plain Condorcet”)..
Now I doubt that MinMax is any better than, or even as good as, Approval for that purpose. Now, in that same situation, I’d use Approval, with Score as a backup tiebreaker.
Anyway, when we voted for what movie to got to, there was a Condorcet winner. In fact, one movie was the voted favorite of a 75% majority: “A Man and a Woman”. It was about a married couple in which the wife’s alcoholism puts a strain on the marriage. The husband is an airline pilot.
I’d voted “The Mask”, with Jim Carey, in 1st place. Someone pointed out that “A Man and a Woman” was my _last_ choice, and that my favorite, “The Mask”, was popular as a 2nd choice. That person suggested that, in fairness to me, “The Mask’ should be chosen as a compromise.
But no, I insiste on majority rule, and so we went to “A Man and a Woman”. It was really boring!
Michael Ossipoff