Smart Referendums

A survey on steroids

If the recent UK Brexit referendum has demonstrated nothing else, it is that asking a straight yes/no closed question will bring disaster.

(See also — The Importance of Don’t Know)

To a certain extent, referendums are extended opinion polls, but like most things politician-led, they do not follow the carefully worked out statistical rules that govern a good poll.

If you have done a poll for someone like YouGov, then you know that some of the questions can feel, at the least, tangential. There are good reasons at play here. Straight black/white questions do not discover the shades of grey that more accurately reflect opinion, and neither do they reveal the reasoning behind the answer.

So, we might know that someone wants to leave the EU, but we do not know what drove their decision. Immigration? Trade Deals? Sovereignty? And what does Sovereignty mean? Protecting the royal privilege? Or perhaps they just hate Europeans.

And the same goes for those who voted to remain.

This lack of knowledge creates a huge problem on several levels.

To start with, those reading the results do not know whether the respondents really understood the question. So, again to make a comparison with a statistical survey, you may ask a question several times nuanced in different ways to see whether you get the same answer.

For instance, you may run a survey on whether Red or White is the better colour.

You could ask that simple question, but you don’t know WHY someone prefers red, for instance.

They may be a petrolhead who loves Ferraris which tend to be red. That may have influenced their decision. They may be a bull who has a strong opinion about red flags.

So, if you start by asking lots of “pick one” questions that feature a red object and a white object, then you might get a very different opinion. In addition, if you then at the end ask whether the respondent likes red or white, their answer may be influenced by the previous options.

Of course, to make this more accurate, you also need a profile. So you might want to have the respondents to rank various objects — a car, a kitten, a meat pie, or whatever. That way, you get some indication of how they rationalise what they like or don’t like.

Really, the depth of what you can learn is almost limitless and becomes impossible very quickly, but some depth is vital if those commissioning the survey want to understand the answer.

So why are political referendums so simplistic?

I think there are several answers here, and not all of them very pleasant.

Firstly, there is a practical issue. A detailed survey that takes the voter through several levels before they get to answer the make or break question, takes time both to conduct and analyse.

Politicians would worry, legitimately, that people will be put off voting if it takes them longer than putting a cross in a box.

Since the result would also take several days, that will upset politicians too since they want an instant result — they want that political impact, and to hell with the accuracy.

There are technical considerations as well, and increased risks of spoilt ballot papers.

And then there are the analysis arguments.

With the present system, politicians can superimpose on the result their own reasoning and bias about why people voted a particular way. Since there is no data to prove them wrong or right, they can get away with any bull they want to dish out. But if you make the system more complicated in an attempt to find out what is going on, then that very analysis will be open to even more arguments.

There is also the issue of what I call “herding.” This is where politicians will intentionally ignore reasoning and reality and will “herd” voters into convenient groups.

We see this with every general election where a political party will assume that all who voted for them read their manifesto fully and agree with every line, even though there is evidence that most people never read the manifestos and certainly don’t agree with anything.

And with the Brexit referendum, you hear every day politicians claim “what the people want,” as if we all are clones without individual thought. Indeed, the only MP I have heard recently object to this is Jess Phillips talking on BBC Politics Live.

Herding becomes harder if you know more detail, and politicians LOVE herding. They would not want it taken away from them.

So, if referendums are bad, where do we go?

I think we have to do one of two things. We either make any referendum far more detailed so it becomes a super-survey or we abandon them all together.

Personally, I would like to see the super-survey, even though I know it will irritate people and possibly hit turn-out numbers. I reckon that a good campaign explaining to people that your government is actually giving you the chance to say why you choose a direction could mitigate some of the downside.

In addition, a “don’t know” option on the final question would also help better reflect how some people honestly feel.

I think this would need a serious uptake of technology because I can't see this happening on paper. And that brings with it the additional problem of those who really are uncomfortable with using technology for whatever reason.

However, if a way around these problems could be found, we might, for the first time ever, begin to know what 40+ million individual voters really think. And if it worked, we could take it to a general election.

I would love to see an election voting paper that included a very short satisfaction survey (whether you are happy with your life or not), followed by some broad-brushstroke policy questions, and finally, “Which candidate do you think will best reflect your opinion?”

I bet that would test all those nonsensical and destructive party loyalties!