Productivity visualisation

I’ve been messing around with D3, one of the better frameworks web based data visualisation. Below is the result of an exercise I set myself to learn D3 — it’s the OECD productivity data from 2012, highlighting New Zealand’s position relative to other countries. For the interactive part I’ve tried to illustrate something meaningful that is not immediately obvious from the data: the effect on income of New Zealanders if our productivity improves. Using the fitted curve you can see what happens if NZ productivity was higher, and the countries that we would be similar to in each case are highlighted.

This is my first attempt at making an interactive thingy so if you have any suggestions please tweet me.


It’s clear that Auckland is growing, not just in population but in wealth and the human capital and creativity of the people who live here. For the longest time Auckland felt like the slightly dopey cousin of Sydney and Melbourne — pleasant enough but kind of dull. But in the past few years it has started to transform and become an exciting place to live.

With growth comes change and there are plenty who seem to want their own backyard untouched with nothing in it, and free parking on the street outside too. To me it seems a lot of the growing pains and NIMBY fights reflect an unwillingness to recognise that the city is growing and as a result land is becoming more valuable, to which the best response is to use land more intensively. As well as land use, there are other less obvious attitudes and behaviours that may need to be updated.

As part of the 100 days project, on twitter Stuart Houghton has been posting one idea for improving Auckland each day. He’s about half way through now. Some of his ideas probably wouldn’t pass a cold hearted cost-benefit test but there are some that are worth reflecting on. Here are my favourites so far:

Dots are in

Since my previous post, I’ve been quite fascinated by dot maps. For the NZ Census income data, I realised you could make a poor man’s dot map by just plotting the income level as a coloured dot at the centre of each meshblock. This looks not too bad as long as you don’t zoom in too much. Here’s what I get for the North Island from the 2013 Census; again blue is deciles above the national median household income and red is deciles below:

The contrast between Northland and Auckland is stark.

And the South Island:

In denser areas you can still see some detail; here’s Auckland:

It turns out that making dot maps that look nice at different zoom levels is quite an art. Here’s a talk by Eric Fisher that explains the basics:

Eric has made some amazing dot maps, including this one showing the race and ethnicity distribution in the United States.

Visualising income distribution

Any chump with GIS software can make a geographic heatmap. Here, I made one for you — this is median household income from the New Zealand 2013 Census by meshblock, for areas around Wellington. The five shades of red are the five deciles below the overall national median, and five shades of blue are deciles above the overall median. White areas are meshblocks where median income was censored because too few people live in the area.

Here’s the same data (I think) plotted much more beautifully by Chris McDowall:

Aside from looking really nice, I think Chris’s dot density plot is a much better visualisation of the income distribution data than the heatmap for at least a couple of reasons:

  • In the heatmap, physically large meshblocks get more visual emphasis, but these areas are generally less important because fewer people live in them. The dot density plot effectively shows population density and income distribution at the same time.
  • The outline shapes of the meshblocks in the heatmap are prominent and distracting in their complexity. The exact boundaries of each meshblock are not very relevant for analysing income distribution. In the dot density plot, each data point is represented by an identical dot.

So the dot density plot is superior because it shows both more and less information at the same time. I really quite like it and I’ll have to learn how to make one.

By the way, you can see more of Chris’s awesome data visualisations on his website.

New Zealand’s sad productivity story

The OECD has upgraded its online data explorer with nice interactive embeddable charts. This one shows an index of labour productivity measured as GDP per hour worked, across all OECD countries (you can click a line to see which country it is):

New Zealand’s poor productivity performance is plain to see — on this measure we were top of the OECD in 1970, falling to the bottom half of the pack by 2013.

The next chart converts to a US dollar equivalent measure of GDP per hour worked:

On this measure, all of the countries that were above us in 1970 have remained above, and the lines for many countries cut New Zealand’s line from below at some point, ie they started out below us but surpassed us. If you want to see a really good performance, find the line for Korea — it was bottom of the OECD on the US dollar measure in 1970 by a long way, but rising to nearly match New Zealand in 2013.

That sucking sound

Paul Brislen talks a lot of sense about why Facebook is going down the gurgler.

I think Facebook as an idea was really good up to the point where they decided they needed to make money out of it. It started with the privacy issues, it got worse with the recent round of changes to the interface and the promoting of paid links. Facebook has really come off the boil and the next wave of customers simply isn’t coming through. It’s because Facebook has stopped treating you like customers and started treating you like cattle – you’re the product.

All businesses need to make money, and an advertising-based model isn’t necessarily fatally flawed. Broadcast TV and radio, newspapers and magazines survived on this model for decades, and no one hated them for it.

The problem is that the internet has given FB the tools to do things like tracking users across multiple websites and experimenting on them. FB’s users seem to dislike these things, but FB’s engineers can’t seem to keep their sticky fingers off these tools. I think if ad-supported social networks want to survive, they’ll need to be much more gentle and respectful in the way they target their ads.

I hope this is just a period where everyone has to learn that free social networks don’t work because they end up turning their customers into a product. FB might die and be replaced by another ad-supported network, but investors can’t be duped into supporting that model forever. Either ads can be made to work in a much nicer way, or eventually everyone will come around to the fact that if they pay a few dollars a month to a social network, it will have an incentive to provide them with a much better experience.

Thou shalt learn to code

Last night this appeared in my Twitter stream —

To which my reaction was –

Aside from the obvious that many of the best jobs for today’s high school students will be technology related, there are many good reasons why everyone should learn to code, such as:

  • Coding is the process of turning an abstract idea into something concrete. You have to map out a logical flow from beginning to end with no gaps. This skill is transferrable to lots of other activities.
  • A little goes a long way — with just a few concepts that you can learn in a few hours, you can do quite a lot of things.
  • It’ll expand what you can do at work, eg just a little code can automate repetitive tasks in Excel and Word.
  • It’s fun!
  • It teaches humility — you will find and fix bugs in code that you would swear should work perfectly, and you will learn something about yourself and the meaning of life in the process.

It doesn’t really matter what language you learn, the basic concepts are the same in every major language. All you need to do to pick up a new language is learn the syntax, which you can do in a couple of hours.

Codecademy is a great place to get started.

Whiz vs bang

NZTE recently published this very pretty data video:

It shows for NZ’s major trading partners the percentage of total exports going to each place, and the rate of GDP growth in each export market. The bubbles move around (and change size) over time as trade patterns and GDP growth rates change.

The most interesting thing is the dramatic rise in exports to China after 2008. There’s also a long steady decline in the proportion of exports going to Japan and the EU which is more difficult to see because it happens gradually.

This got me thinking about the value of the data video. It’s mesmerising to watch but is it good for communicating information about changes in NZ’s exports? The following occurred to me:

  • It’s not clear to me what the size of the bubbles represents.
  • What value is added (econo-pun!) by having GDP growth on one axis? It doesn’t seem like GDP growth is correlated with exports, eg exports to China rise when after its GDP growth starts slowing down. It feels a bit like GDP growth was added just to have a second dimension for the animation, but then visually tracking things in two dimensions is difficult.
  • The video takes a long time (over 3 minutes) to make its point.
  • Slow steady trends are more difficult to see than quick dramatic ones.
  • It’s also difficult to see long term trends. By the time you get to 2008, can you remember what was happening in 1988?

Here’s an alternative chart that I made. It’s more boring and I didn’t spend a lot of time to make it pretty, but I reckon it does a reasonable job of communicating information about NZ’s trade patterns.

(The data I could find from Stats NZ didn’t have the GCC region and only went back to 1988; also I don’t have forecasts)

But I think all is not lost for the video. It is nice eye candy after all, and so will probably attract more eyeballs than my dull chart. Maybe it’d be better as an interactive chart rather than a video, for example with a time slider that you can drag back and forth while watching one of the bubbles.

Graph of the century

Via @donal_curtin and @tvhe, this graph:

I can’t quite find a link to the original source, but seems it was made in 2013 by Branco Milanovic — somehow I didn’t see it until now. Some commentary here.


Last week the NZ Initiative, a free market-oriented think tank, put out a report that was generally critical of the “compact city” approach to urban planning.

One thing I liked about the report was its criticism of the very arbitrary rules that urban planners often impose on cities. This includes such things as minimum parking requirements, building height constraints, excessive “character” protections, yard and set-back restrictions, et cetera. As the report correctly noted, all of these things serve to constrain housing supply and increase prices.

The report used these arguments to launch a somewhat emotional attack on “compact city” planning that aims to limit sprawl and encourage densification. It would’ve been nice to see a bit more thought put into how a free-market based system could work instead. In particular, how can rigid planning restrictions be removed while still providing a mechanism for disagreements to be resolved, eg I like my view but you want to build in front of me.

The way I see it, a city is a platform for enabling interactions among people — if we were all hermits we could live far apart but most of us don’t seem to choose that lifestyle. So the objective of a city should be to maximise the overall quality-adjusted volume of such interactions. This is the solution to a complex constrained optimisation problem.

The fundamental constraints in this optimisation come not from city planners but from scarce resources: land obviously, but also other things like nice views, a clean environment, safety, health, etc. It’s not clear that adding additional planning constraints on top of these “natural” constraints will make anything better. On the other hand planning constraints are often well-intentioned, trying to resolve conflicts over the use and ownership of resources. An interesting question is whether there are market institutions that could do better?

The NZ Initiative report also devotes a considerable amount of ink to arguing that compact cities don’t reduce traffic congestion. This irks me because while nobody likes to waste time in traffic, congestion is really not a good measure of how well a city is solving its constrained optimisation problem.

Of course congestion itself is caused by a market failure and a lack of a price, but if congestion pricing is “too hard” to implement then it’s risky to give congestion too much weight in your evaluation of a city’s performance. For one thing congestion only affects those who drive; in many large successful cities a lot of travel is not done by driving. Furthermore it’s dubious whether building more roads “solves” congestion in the long run, since more building roads reduces the effective price of driving and induces more of it. However you can permanently solve congestion for a person by taking them off the road and either reducing their need to travel so much (eg live closer to work) or giving them an alternative transport mode.

A follow up opinion piece by one of the NZ Initiative report authors falls into this trap. The argument is that good train transport systems don’t seem to reduce road congestion, as measured by vehicle travel times or speeds on the roads. The problem is, every person on a train is not experiencing road congestion, but the road congestion measure ignores this. This nicely illustrates why it’s risky to emphasise such a narrow measure of city performance.