Jordan You might remember back in January we talked about polls. We talked about how to read them, what to look for, and why you’ll see a million of them every week from now until the day Canadians vote in October. According to a new poll,
Jordan Justin Trudeau’s Liberals are going through….. You can ignore most of those, but our guest today will not.
Jordan To him, every poll is another piece of data to be fed into a complex formula that aims to project not just the popular vote, like most of the polls you see try to do, but to project who will win each of Canada’s three hundred and thirty eight federal ridings. Complex statistical projections have become sort of the math based rock stars of the political prediction world. This began in the United States back in 2012, but really anywhere you have a number of data points and someone who’s really good at math, you have the makings of an electoral projection system and ours just happens to reside in the brain and computer of an astrophysicist in Quebec, who has nailed more than 90 percent of the seats in three provincial elections in a row. How do you even begin to go about devising that kind of system? What’s in the secret sauce and what can it tell us right now about what might happen in October?
Jordan I’m Jordan Heath Rawlings and this is The Big Story. Philippe J. Fournier runs the website 338 Canada dot com. He also presents his findings in Maclean’s, and in L’actualitie in Quebec. Hi Philippe.
Philippe Good morning Jordan.
Jordan You have a really interesting backstory, can you explain to me how you went from astrophysics to election projection?
Philippe This isn’t the first time I get asked this question, because people are curious about how that lenient-how did you go from one to the other. Well, I studied physics and astrophysics in university and after when I was done I began teaching, I got really lucky. I got a position teaching at Cegep de Saint-Laurent, and I was kind of bored, so I decided to continue university while I was working full time, and I decided to take political science at Concordia University in Montreal. It was just a hobby for me, and it was really interesting, I loved it, but what I loved the most was the numbers part of it, the demographics, the election, the modelling, so I followed of course Nate Silver in the United States since he began, then I was really impressed with his work, with his science, and I’ve been testing my own models for a long time but I never really put it online, it was just a hobby, and then during the American election, the Trump vs. Clinton election, I was pulling my hair out listening to pundits and talking heads saying that Clinton was going to win for sure and that Trump had no chance, because this is not what the numbers were saying, and until shortly after that they decided to launch my Quebec model called the QC one 2 5 dot com, and within two to three months, well I don’t want to toot my own horn here, but it really became viral. Every political party in Quebec and lots of media outlets began approaching me, asking me questions, and then I find myself writing for Election Day. So that’s that’s how it went. I’m still an astrophysicist,
Philippe I still teach in this subject, but now I have this subjects that’s really fun to work on.
Jordan For people who are interested in elections, and predictions, and polling but maybe haven’t been deep into the kind of nerdy world of the Nate Silvers, and modeling, and that kind of stuff. Can you explain to me the difference between the polls that we’re used to seeing quoted on television, and the kind of work that you and Silvers in the United States and other modelers do?
Philippe Well that’s also a very good question. Basically, you have to always re-actualize when you analyze polls, because sometimes journalists and commentators give away way too much importance to a poll, and so polling is a science. But it’s a statistical science, and so it has uncertainty, and so when you have a poll that gives you information, you have to keep in mind that it contains uncertainty. So it’s only by polling more and more that you can have a clearer picture of what the landscape actually looks like, and so I treat polls as the pieces of a giant puzzle, and when they come in they always have different information, because of course there is different samples, different models different techniques, but I take them all together, and I have the speculation that I do based on amplifiers, and field dates, and the background of the pollster also is very important, because some pollsters perform better than others. Some pollsters perform better at the professional level or the federal level, and so I began delving into this, and what I really worked hard on during the 2016-17 building of my model, was using demographics-the demographic data that is available from the center, and when you go on statscan dot com, or whatever the website is, you can find this data or you find a median age, median household income, and stuff like that. Well it’s also split by electoral districts, and so I checked if there was a correlation between this demographic data, and election cycles, and sometimes there’s no correlation or no obvious correlation, but sometimes there is, and so this helps to predict the movements of the electorate from one election to the other, and so that’s how the model basically works.
Jordan Give me an example if you can of that confluence between some electoral data, and some demographic data in a riding, like what would you be looking for? Is this riding particularly young? Is it particularly diverse like stuff like that, and what does that tell you?
Philippe Well, stuff like that, of course. My first election was Ontario, but it was a practice run because my model was at first built for Quebec, and so the first thing in Quebec is language. You start with language, you have francophones, and non francophones, and then among the francophones you have urban, you have suburban, and you have rural, and then you go into this data, and you look at the educated and non educated. So you split basically everything into little pieces, and what I discovered before the last Quebec election was for instance, Quebec City, the capital swings the same way as the suburbs of Montreal. They don’t always vote the same way, but from one election to the other, the swings are similar issues, the water level is different, but the swings are similar, and so I began testing this model by linking some electoral districts together to see what it could produce, and so one of the tests that I did early on in my model to see whether it would work, is I took the data that was available only in 2012, and with the polls from 2012 to 2014, the past two elections that we had Quebec, I checked to see if my model would perform well in the 2014 election, and it did, and so that was a eureka moment for me, that I realized that I could call about 90 to 95 percent of the ridings correctly just by following those trends. I can tell you though that it is a lot of work. It would be 10 o’clock at night, and I would just be you know, sitting in front of my computer and my wife would go I’m going to bed are you coming soon, and I’d be like okay I’m coming soon, and suddenly it’s 5:00 o’clock in the morning and I have to teach at 8, and I would spend lots of all nighters working on this, but so far it’s going very well, and I’m optimistic for the federal election, it will be my first federal election coming up this fall.
Jordan Well that was kind of my next question, is how do you judge success when you’re kind of already expecting to get a vast majority of the races right, and how different does that success look from what a traditional polling company would consider a successful prediction?
Philippe Well traditional polling companies only look at the popular vote. If they get the popular vote right, then they’re happy, or within the margin of error then they will be happy. I judge my numbers based on the seat projection. So basically the popular vote-while we can have some modelling of how for instance, undecided voters will go, but you know you can burn yourself if you try to guess where the undecided go, because from one election to another, it’s always different, so you have to be very careful. But basically I will take all the polls, and I will try to correlate the movements from one region to another, and basically how I judge is how many seats do I call right, and among those I called long was the winner within the margin of error or what I call the confidence interval, and so this is how basically I-because for instance, the Alberta election that we had in April, the United conservative on average in the polls we’re getting maybe 50 percent of the popular support. But in the end they had 55 percent of the popular vote, so you could say that the polls missed the mark. I don’t think they missed it, I think they were imprecise but they did call the right winner, and by a fair margin and so many of the ridings in Alberta where I had the contributiors winning by eight or nine points, well in the end they ended up winning by 14 points. But that’s not my fault, that’s the polls, and so for me if I have the right winner that’s the win. That’s what I’m looking at, and in Quebec one hundred and twenty five ridings I call 112 right, and among the nine districts that I missed, four of them were within the margin of error. So it’s about 90 percent success, so it’s better than a roll of the dice.
Jordan Aside from running your own website, and doing work for Maclean’s, and L’actualite, and talking to curious people like us, what kind of demand is there for this data? Have you been in touch with any campaigns? Did they use this kind of modelling? What’s the future of it?
Philippe Oh absolutely. So, again my first campaign in Quebec, every single party contacted me directly, and wanted to meet me, wanted to ask me questions, I even received offers to buy me out.
Jordan Really?
Philippe Oh yeah, and I of course I declined because for the simple reason that this is my baby project, it’s mine, and I basically told them everybody has the price right?
Philippe My price is 1 billion, not 1 million, it’s 1 billion. So if you have 1 billion in cash for me okay maybe I’ll consider it, but otherwise….
Jordan In cash too.
Philippe Oh ya, in cash, I’m not-yes of course.
Philippe So they approached me, and often on Twitter, on Facebook, and sometimes by phone, I’ll have political staffers in this column, you say what you think is happening in this riding? What’s happening in Quebec and Ontario? You know, I want to be non-partisan and I want to be fair to you as well. My rule is I will answer every question that’s asked of me, but I will make those entries public. One party during the Quebec campaign asked me for a list of writings ranked by most likely to least likely, and I said sure, this is stuff that I already have on my website. I’ll do it for you, but I’ll publish it so everyone can see it, and they agreed. But there is a demand because I think political staffers, and political parties, they are good with campaigning. They’re generally not really good with math, and again when I started this project this is what drove me first because I’m a science teacher, and I teach my my students the scientific method, and I would just take an article from a newspaper and tell my students look at this. Look how the journalist says oh the Quebecois is up one point this month, and I go no it’s fluctuation, it’s noise in the data, and so I wanted to teach my students how to show the data with uncertainty, and it’s not a slight to my colleague journalists, but that’s not their specialty but it’s mine. So I thought some natural science could help a little bit in the field.
Jordan We’ve mentioned Nate Silver and 538 a couple times mostly because I think that’s the most prominent example that everybody cites. How different is it projecting elections with this kind of modeling in Canada from the United States, or is it different?
Philippe Well I will be frank with you, I followed Nate Silver and factory it for a long time, but I cannot say that I know all the details of their modelling, because some of it of course is kept secret as it should, because you don’t want to give up your recipe right? The same way I’m open about my methodology, but if I’ve been asked for my code and I go like no it’s 1 billion dollars, so that’s how it goes. But you have to compare apples with apples here. So if you compare presidential elections to let’s say a federal election in Canada, well of course the federal election in Canada it will be much more difficult to call, because the presidential election is by states. Out of the 50 states, there are 35 or 36 that are decided in advance, and so this is usually much easier to call than three hundred and thirty eight ridings which are all different. But I know that 538 also does penetrate in the Congress races, and so this is much harder and much uncertain. So far we don’t have the equivalent in Canada. I worked a bit on the by election that we’ve had, but I tell my readers that the model is made for a general election, so not to worry if the results don’t match the by election, because by elections of course are different animals, they depend on the mood of the day or the headline of the week, and of course the turnout is usually very low. So I would say that the big difference here is that you have to know the difference between the 9 0 5 and Toronto. If we work like in the United States it would just be Ontario and that’s it.
Jordan That makes sense. Do you ever deviate from the numbers? Does your instincts, gut whatever ever come into play, or do you just follow whatever the model spits out?
Philippe Well, thing is you have to remember the model-and this is a fair question, but to our listeners here, the model it’s not a magic machine that I built and you press the button and you turn some knobs to give you results. The model basically does what I tell it to do. It just does it much faster than I could on paper, and pen and paper, and so I crunched the numbers, but I do not really put my feelings into it. I will change the parameters sometimes, but I will always have to have it backed by data. But I can tell you that I’m getting a feel for these elections more and more. For instance, when I sent an article to Maclean’s last month about the P.I. election, the text basically-the model basically said that the Greens were probably going to win majority government but by a hair, and just a few hours after I published that in Maclean’s, you had another poll that was published just before the election was going on, on April 23rd I think, and it showed that the race was tightening, and I decided to redo it, I could not publish it in Maclean’s so I published it in my blog and I warned my readers saying OK I did say that in Maclean’s, but we have new data here and in science, well when new data comes you have to change your mind, you have to redo the model, and it showed that the Greens were maybe winning a minority, but maybe not, and in the end they won eight seats. My average was, I think it was eleven seats, so it was not that bad. It wasn’t a bad result, but there’s so little data about P.E.I and those electoral districts three thousand people.
Jordan Right.
Philippe So these are really hard to follow. For instance in Ontario, you have one green seat am I right? I mean Guelph. Right?
Jordan Yeah.
Philippe Mark Schreiner in Guelph. Mark Schreiner in his election, in his district, got more votes than the Green Party of P.E.I all taken together during that election, and so P.E.I was so hard to call because, very little data and small districts, so there’s very high uncertainty. Right away so far my best election was Ontario, out of the hundred and twenty four seats, I called the right winner or was within the margin of confidence in a hundred and twenty two of them. I only missed two.
Jordan That’s pretty good. I was going to ask the million dollar question but I guess for you it’s the billion dollar question. What does the model say, right now we’re kind of in the middle of May, about the federal election?
Philippe That’s a hard question to-I can’t tell you what it says now, but of course-and I just I cannot stress this enough, I don’t do predictions. I do projection.
Jordan Right.
Philippe And the difference is, I do what the data tells me now but it will change tomorrow, and it will change next week.
Jordan I will not hold it to this in October.
Philippe Very well. Well I will do one prediction. It will be October 20th, so the day before the election I will make a prediction because we’ll have all the data on hand, but what the data says right now is that there’s a lot of noise in the system. The Jodi Wilson rebels in Athens, they have a nice story in Janfield, all that we had 16 polls in April. That’s one poll every two days, and both those are not convergent, they tell different stories. We have Angus Reid and Leslie that had the conservatives ahead by 30 points, and we had the nine up in campaign research with the Conservatives ahead by three or four points, and so there’s a world of difference between a double digit lead over a conservative, or three or four point lead, and so this tells me-some people will say the point from my bias. I do not buy that, those professional polling firm live on their reputation, and so I just think that when you have a lot of fluctuation, when public opinion is in flux, you have a lot of noise, and so this increases the uncertainty. I want to I don’t want to have a magic moment say it’s going to be this it’s going to be that, I want to give a spectrum of possibilities, I want to give an honest answer to my readers, say this is what the data tells us, and if the data is uncertain, well the result is uncertain. That’s how it goes. I wish I could just confidently say the stories are going to win 172 seats and that’s it. It maybe would make better TV, maybe you would get more clicks, but I don’t think it would be honest.
Jordan It would also eliminate months of speculation for people like us. So don’t do that just now.
Philippe I won’t, and just to answer your question right now, with the numbers that we have right now, the most likely scenario would be a conservative minority, because when you look at the Greens pulling in double digits, you look at the black Quebecois, which is, well it’s not in the good old days winning 50 seats but it’s still alive there, and so you’re thinking maybe 15 to 20 seats. The NDP even though it’s not polling, too well should win maybe 20 to 30 seats. So that’s one 70 seat threshold or a majority will be very hard to get for either the Liberals or the Conservatives. When you think about four years ago to the win in 2015, it was a grand victory. It was spectacular. You won all the seats in the Maritimes, you won Quebec, you won Ontario, and you even won seats in Alberta. But you look back at the result, and Trudeau won one hundred and eighty four seats. That’s only 14 more seats than the official for majority, and so it will be very hard to replicate that. So I do not see a liberal majority right now, and it’s really hard to see the Liberal win if the Greens are pulling double digits in Canada because that’s just a dagger through the heart for them.
Jordan We will have to check in with you our closer to October.
Philippe Very well, it’s going to be a pleasure.
Jordan Thanks so much for joining us.
Philippe Take care. Thank you.
Jordan Philippe J. Fournier, astrophysicist, specialist in election projection, and the only man on this podcast who ever ask for a billion dollars in cash. That was The Big Story for more from us, we are at the big story podcast dot ca. We are on your Twitters, at the big story FPN, and at frequency pods, frequency pods also on Instagram and Facebook. Yes you can look at pictures of our podcast episodes. Hit us up wherever you get podcasts, Apple, Google, Stitcher, Spotify. Leave us a rating, leave us a review, tell your friends, take their phones, and subscribe for them when they’re not looking. Thanks for listening. I’m Jordan Heath-Rawlings. We’ll talk tomorrow.
Back to top of page