Goals, Metrics and Doing the Right Thing

In last week’s blog, we discussed creating a goal for Calgary Transit. This goal was to achieve 25% mode share for the commute by 2025 (i.e. 25% of people traveling to work would use transit during rush hour). The idea was to increase the amount of ridership from the current 17%.

Alderman Gian-Carlo Carra (Ward 9) criticized this goal (via Twitter). His critique wasn’t that the goal wasn’t ambitious enough, it that we were measuring it wrong.

What we measure matters. The metrics we use in our lives, whether they are test scores in school, performance measures at work or points scored in hockey, influence what we value and what we focus on. The measurements we use determine how we allocate resources, make a choice between one strategy and another and how we reward different behaviours. That is the first lesson of Gian-Carlo’s critique. In the words of Joseph Stiglitz, a Nobel laureate in Economics:

If you don’t measure the right thing, you don’t do the right thing.

Gian Carlo was pointing out that by making a goal around just commuters using transit, we end up focusing too much on work trips alone, to the detriment of other trips that transit serves. If Calgary Transit were to focus solely on improving the percentage of commuters using transit, resources would be diverted from routes that service non-work trips, such as people doing errands, going to visit friends or just going out, to those getting to work during rush hour. As well, we would miss all those workers who don’t work the typical nine to five job. If we were to focus only on the the commuter transit user, service hours during off-peak travel periods would be cut and routes not servicing work-trips would be curtailed in order to devote more service hours and resources to getting people to work during those two and a half hours each morning and evening. But by doing this, we would be improving according to the measurement we were using. That just doesn’t seem right.

If focusing on the just the commute isn’t the right metric, what is?

One alternative would be to create multiple goals for transit. We could create not only a goal for work trips, but also for non-peak trips (those trips occurring outside the rush hour) and then compare the two against each other. If we are performing well on transit ridership during rush hour, but not as well on during other parts of the day, we can allocate service hours to bolster service during those non-peak times or focus more on facilitating non-employment developments to locate closer to transit (recreation, shopping etc). The question becomes, how do we measure that?

The percentage of people using transit to work is typically taken from Census data (literally how many people respond to the question “how do you travel to work”), while the percentage of people traveling downtown using transit during rush hour (which just passed 50% this year as we noted in the last post) is calculated by a physical count of all the cars, bikes, pedestrians and transit users going downtown during a particular day each year (called the downtown cordon). However, measuring how many people take transit outside these times or for different purposes is more difficult to measure.

Asking each individual where they are traveling to and why is obviously a costly and time consuming process. Usually proxy measures are used such as fare payments, but this becomes difficult with people who use monthly passes. And then there is the whole issue of transfers. The process can become very complex and difficult very quickly.

That brings us to the second lesson: don’t use a metric just because it is easy to measure. Measuring the percentage of people using transit to get downtown, while complex and time consuming, is more straight-forward than measuring how many people take transit to get to other locations. Just because something is easy (or at least easier) to measure, doesn’t mean it is important.

Transit isn’t just for getting people to work. Great transit cities allow people to use transit to go out for dinner, to go shopping, to visit friends, see the doctor, pick up some groceries for the weekend, play sports, see a movie and many other things. A great transit city is a place where taking transit for almost any trip is convenient, safe and affordable.

One development that will greatly improve this situation is the roll out of the Connect card later this year (if things go according to plan). Apart from being incredibly convenient (passengers with a connect card simply have to “tap” a sensor to pay the fare, which would be deducted from their pre-paid card), the connect card has another benefit, it provides an incredible amount of data for Calgary Transit. Rather than relying on physical counts of passengers or fare collection, the Connect card will provide Calgary Transit with accurate measurements of how many people are using transit, at what times and on what routes (although not perfectly, as not all passengers will be using the cards, but definitely a huge improvement). This will allow us to improve the way we evaluate our transit system by opening numerous other ways to measure performance.

One of these measurements is to think in terms of “Transit Consumption”. This is the total number of transit trips taken each year, divided by the population. This is an interesting metric, because it doesn’t value any particular trip more than another. Using this metric, one person taking transit 14 times a week is equal to 7 people taking transit twice a week. Whether that trip is to get to work, the doctor’s office, or the grocery store, the value is the same. It is definitely an interesting way to look at transit (although it isn’t without its faults either), and may be one more way we can evaluate  our transit system.

We should always understand what we are measuring and why. It isn’t just important to measure our progress, it is important to understand what that measurement means. Too often we blindly focus on a improving a measurement thinking we are being diligent, when too often we are being misguided. Choosing the right metric(s) for Calgary Transit is an important part of setting a long-term strategy, and shame on us if we neglect an important part of what transit does, just because it is hard to measure.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s