04 December 2015

Montreal - When Using Data Goes Wrong

This article is a guest contribution from Bartek Komorowski. Bartek is an urban planner and currently Project Leader in Research and Consulting at Vélo Quebec in Montreal. He and his colleagues reacted to a compartive study published last month in Canada and we're pleased to bring his thoughts here. Data is of utmost importance. More often than not, cities simply don't have enough of it. Then you have professionals who taken existing data and completely abuse it. Which is what this piece is about.

By Bartek Komorowski

Last week, the Pembina Institute, a reputable clean energy think tank, released a comparative study on cycling in Canada’s five largest cities – Toronto, Montreal, Vancouver, Calgary, and Ottawa. The study compares a number of statistics on bicycle use, safety, and infrastructure. The authors spin a narrative about Montreal being a great cycling city, mentioning its presence on the Copenhagenize Index. Strangely, their report provides statistics that seem to suggest otherwise.

I work in Vélo Québec’s Research Department and my colleagues and I know a thing or two about Montreal’s cycling statistics. We publish a report called Bicycling in Quebec every five years, which includes a subsidiary report focusing on Montreal. We are currently collecting data for next edition of Bicycling in Quebec and Montreal’s cycling statistics are at the top of our minds.

While reading through Pembina’s report, we were struck by some major discrepancies between the data it presents and the data we have recently been poring over. We noticed the following:

Daily Bicycle Trips
The study says there are 40,000 daily bicycle trips on the Island of Montreal, citing data from 2008. Actually, there were 75,000 daily bicycle trips in 2008 according to that year's regional origin-destination survey. Results from the more recent 2013 origin-destination survey, which have been released to the media, show that 120,000 bicycle trips were daily in Montreal, representing a massive 54% increase compared to 2008.

Crash Rate
The authors calculated an annual crash rate for the Island of Montreal using a bicycle trip statistic supposedly from 2008 and a crash statistic that seems to be from 2013 (found here), which is methodologically unsound, particularly in the context of rapidly increasing bicycle trip rates. This error was compounded by the use of the faulty daily trip rate statistic explained above. They got a result of 6.6 crashes per 100,000 bicycle trips, a higher rate higher than all of the other cities. In fact, according to their results, the likelihood is of having a cycling accident in Montreal is an absurd 10 times greater than in Vancouver, whose crash rate is 0.67 per 100,000 trips.

Local media in Montreal zeroed in on this result, producing shock headlines such as this one. Needless to say, for cycling advocates, news reports about the supposed dangers of cycling in Montreal, backed up by seemingly credible research from a well-known think tank, are not helpful.

I redid the calculation for Montreal using the same method but plugging in the 2013 daily bicycle trip rate (which matches the year of the collision statistic). This yielded a crash rate of 2.3 per 100,000, lower than that of all cities except Vancouver.

Incidentally, the crash rate statistic for Vancouver strikes me as suspiciously low. This might have something to do with that city’s questionably high daily bicycle trip statistic. I have trouble believing that Vancouver, with roughly one-third the population of the Island of Montreal (603,500 vs 1,890,000), has almost three times more daily bicycle trips (106,500 vs 40,000). Can Vancouverites possibly be making eight times as many daily bicycle trips per capita as Montrealers? I doubt it.

Bicycle Network Size
The authors were sloppy in comparing the sizes of the cities' cycling networks. Some cities, like Toronto, count bicycle lane kilometres. This means that if there are bicycle lanes going in both directions along 1 km of street, they count as 2 km of bicycle lanes. Montreal counts street kilometres with bicycle lanes. In this case, bicycle lanes going in both directions along 1 km of street count as 1 km. If we applied Toronto's accounting method to Montreal's network, ours would appear to be almost twice the size of theirs.

Number of Bicycle Shops
The number of bicycle shops in a city is “an indicator of the prominence of cycling culture and access to bicycles,” say the authors. According to their findings, Montreal is poorly endowed in this department, with only 25 bicycle shops serving its 1.7M inhabitants. This is statistic is way off the mark: Vélo Québec's non-exhaustive listings contain 75 shops on the Island of Montreal, while a cursory search in the phone directory suggests there are well over 100. I can only speculate that this error is attributable to authors’ lack of knowledge of French and failure to use adequate search terms to find bicycle shops.

Other Minor Mix-ups
The study contains a number of other minor statistical mix-ups. The study cites the 2011 census population of the City of Montreal (1.65M) but cites the area (500 km2), the bicycle mode share (2.9%), and other statistics for the whole Island of Montreal, on which there are 14 other municipalities and 250,000 more residents. The City of Montreal has an area of 430 km2 and a bicycle mode share of 3.2%.

There are also some factual errors in the text about Montreal. For example, the authors say that the cycling network is planned at the borough level. In fact, the network is planned and financed by the City’s central administration. Boroughs do have scope to go above and beyond the City’s recently updated master plan (see picture below), putting bikeways on local streets under their jurisdiction. It is nonetheless incorrect to say that boroughs plan the network and the planning therefore has no coherence.

There may well be other errors but, as I’m less intimately acquainted with the cycling stats of the other cities, I cannot identify them as easily. I hope that the authors did a better job for the others than they did for Montreal. Due to the paucity of compilations of data on cycling in Canadian cities, it’s likely errors from this study will continue to circulate in the media and will be reproduced in other research, unless somebody sets the record straight.

My colleagues and I hope to provide a rich and accurate picture of the current state of cycling in Montreal, as well as its evolution over time, when we release the next edition of Bicycling in Quebec next June. Stay tuned.