Dots are in

Since my previous post, I’ve been quite fascinated by dot maps. For the NZ Census income data, I realised you could make a poor man’s dot map by just plotting the income level as a coloured dot at the centre of each meshblock. This looks not too bad as long as you don’t zoom in too much. Here’s what I get for the North Island from the 2013 Census; again blue is deciles above the national median household income and red is deciles below:

The contrast between Northland and Auckland is stark.

And the South Island:

In denser areas you can still see some detail; here’s Auckland:

It turns out that making dot maps that look nice at different zoom levels is quite an art. Here’s a talk by Eric Fisher that explains the basics:

Eric has made some amazing dot maps, including this one showing the race and ethnicity distribution in the United States.

Visualising income distribution

Any chump with GIS software can make a geographic heatmap. Here, I made one for you — this is median household income from the New Zealand 2013 Census by meshblock, for areas around Wellington. The five shades of red are the five deciles below the overall national median, and five shades of blue are deciles above the overall median. White areas are meshblocks where median income was censored because too few people live in the area.

Here’s the same data (I think) plotted much more beautifully by Chris McDowall:

Aside from looking really nice, I think Chris’s dot density plot is a much better visualisation of the income distribution data than the heatmap for at least a couple of reasons:

  • In the heatmap, physically large meshblocks get more visual emphasis, but these areas are generally less important because fewer people live in them. The dot density plot effectively shows population density and income distribution at the same time.
  • The outline shapes of the meshblocks in the heatmap are prominent and distracting in their complexity. The exact boundaries of each meshblock are not very relevant for analysing income distribution. In the dot density plot, each data point is represented by an identical dot.

So the dot density plot is superior because it shows both more and less information at the same time. I really quite like it and I’ll have to learn how to make one.

By the way, you can see more of Chris’s awesome data visualisations on his website.

New Zealand’s sad productivity story

The OECD has upgraded its online data explorer with nice interactive embeddable charts. This one shows an index of labour productivity measured as GDP per hour worked, across all OECD countries (you can click a line to see which country it is):

New Zealand’s poor productivity performance is plain to see — on this measure we were top of the OECD in 1970, falling to the bottom half of the pack by 2013.

The next chart converts to a US dollar equivalent measure of GDP per hour worked:

On this measure, all of the countries that were above us in 1970 have remained above, and the lines for many countries cut New Zealand’s line from below at some point, ie they started out below us but surpassed us. If you want to see a really good performance, find the line for Korea — it was bottom of the OECD on the US dollar measure in 1970 by a long way, but rising to nearly match New Zealand in 2013.

That sucking sound

Paul Brislen talks a lot of sense about why Facebook is going down the gurgler.

I think Facebook as an idea was really good up to the point where they decided they needed to make money out of it. It started with the privacy issues, it got worse with the recent round of changes to the interface and the promoting of paid links. Facebook has really come off the boil and the next wave of customers simply isn’t coming through. It’s because Facebook has stopped treating you like customers and started treating you like cattle – you’re the product.

All businesses need to make money, and an advertising-based model isn’t necessarily fatally flawed. Broadcast TV and radio, newspapers and magazines survived on this model for decades, and no one hated them for it.

The problem is that the internet has given FB the tools to do things like tracking users across multiple websites and experimenting on them. FB’s users seem to dislike these things, but FB’s engineers can’t seem to keep their sticky fingers off these tools. I think if ad-supported social networks want to survive, they’ll need to be much more gentle and respectful in the way they target their ads.

I hope this is just a period where everyone has to learn that free social networks don’t work because they end up turning their customers into a product. FB might die and be replaced by another ad-supported network, but investors can’t be duped into supporting that model forever. Either ads can be made to work in a much nicer way, or eventually everyone will come around to the fact that if they pay a few dollars a month to a social network, it will have an incentive to provide them with a much better experience.

Thou shalt learn to code

Last night this appeared in my Twitter stream —

To which my reaction was –

Aside from the obvious that many of the best jobs for today’s high school students will be technology related, there are many good reasons why everyone should learn to code, such as:

  • Coding is the process of turning an abstract idea into something concrete. You have to map out a logical flow from beginning to end with no gaps. This skill is transferrable to lots of other activities.
  • A little goes a long way — with just a few concepts that you can learn in a few hours, you can do quite a lot of things.
  • It’ll expand what you can do at work, eg just a little code can automate repetitive tasks in Excel and Word.
  • It’s fun!
  • It teaches humility — you will find and fix bugs in code that you would swear should work perfectly, and you will learn something about yourself and the meaning of life in the process.

It doesn’t really matter what language you learn, the basic concepts are the same in every major language. All you need to do to pick up a new language is learn the syntax, which you can do in a couple of hours.

Codecademy is a great place to get started.

Whiz vs bang

NZTE recently published this very pretty data video:

It shows for NZ’s major trading partners the percentage of total exports going to each place, and the rate of GDP growth in each export market. The bubbles move around (and change size) over time as trade patterns and GDP growth rates change.

The most interesting thing is the dramatic rise in exports to China after 2008. There’s also a long steady decline in the proportion of exports going to Japan and the EU which is more difficult to see because it happens gradually.

This got me thinking about the value of the data video. It’s mesmerising to watch but is it good for communicating information about changes in NZ’s exports? The following occurred to me:

  • It’s not clear to me what the size of the bubbles represents.
  • What value is added (econo-pun!) by having GDP growth on one axis? It doesn’t seem like GDP growth is correlated with exports, eg exports to China rise when after its GDP growth starts slowing down. It feels a bit like GDP growth was added just to have a second dimension for the animation, but then visually tracking things in two dimensions is difficult.
  • The video takes a long time (over 3 minutes) to make its point.
  • Slow steady trends are more difficult to see than quick dramatic ones.
  • It’s also difficult to see long term trends. By the time you get to 2008, can you remember what was happening in 1988?

Here’s an alternative chart that I made. It’s more boring and I didn’t spend a lot of time to make it pretty, but I reckon it does a reasonable job of communicating information about NZ’s trade patterns.

(The data I could find from Stats NZ didn’t have the GCC region and only went back to 1988; also I don’t have forecasts)

But I think all is not lost for the video. It is nice eye candy after all, and so will probably attract more eyeballs than my dull chart. Maybe it’d be better as an interactive chart rather than a video, for example with a time slider that you can drag back and forth while watching one of the bubbles.

Graph of the century

Via @donal_curtin and @tvhe, this graph:

I can’t quite find a link to the original source, but seems it was made in 2013 by Branco Milanovic — somehow I didn’t see it until now. Some commentary here.


Last week the NZ Initiative, a free market-oriented think tank, put out a report that was generally critical of the “compact city” approach to urban planning.

One thing I liked about the report was its criticism of the very arbitrary rules that urban planners often impose on cities. This includes such things as minimum parking requirements, building height constraints, excessive “character” protections, yard and set-back restrictions, et cetera. As the report correctly noted, all of these things serve to constrain housing supply and increase prices.

The report used these arguments to launch a somewhat emotional attack on “compact city” planning that aims to limit sprawl and encourage densification. It would’ve been nice to see a bit more thought put into how a free-market based system could work instead. In particular, how can rigid planning restrictions be removed while still providing a mechanism for disagreements to be resolved, eg I like my view but you want to build in front of me.

The way I see it, a city is a platform for enabling interactions among people — if we were all hermits we could live far apart but most of us don’t seem to choose that lifestyle. So the objective of a city should be to maximise the overall quality-adjusted volume of such interactions. This is the solution to a complex constrained optimisation problem.

The fundamental constraints in this optimisation come not from city planners but from scarce resources: land obviously, but also other things like nice views, a clean environment, safety, health, etc. It’s not clear that adding additional planning constraints on top of these “natural” constraints will make anything better. On the other hand planning constraints are often well-intentioned, trying to resolve conflicts over the use and ownership of resources. An interesting question is whether there are market institutions that could do better?

The NZ Initiative report also devotes a considerable amount of ink to arguing that compact cities don’t reduce traffic congestion. This irks me because while nobody likes to waste time in traffic, congestion is really not a good measure of how well a city is solving its constrained optimisation problem.

Of course congestion itself is caused by a market failure and a lack of a price, but if congestion pricing is “too hard” to implement then it’s risky to give congestion too much weight in your evaluation of a city’s performance. For one thing congestion only affects those who drive; in many large successful cities a lot of travel is not done by driving. Furthermore it’s dubious whether building more roads “solves” congestion in the long run, since more building roads reduces the effective price of driving and induces more of it. However you can permanently solve congestion for a person by taking them off the road and either reducing their need to travel so much (eg live closer to work) or giving them an alternative transport mode.

A follow up opinion piece by one of the NZ Initiative report authors falls into this trap. The argument is that good train transport systems don’t seem to reduce road congestion, as measured by vehicle travel times or speeds on the roads. The problem is, every person on a train is not experiencing road congestion, but the road congestion measure ignores this. This nicely illustrates why it’s risky to emphasise such a narrow measure of city performance.

Communication is hard

The other day, Michelle Dickinson (aka Nanogirl) posted a very depressing tweet:

Followed by:

For those who don’t know, Michelle runs a nanomechanical lab at Auckland University and is an outstanding communicator of science and and inspiration to students. You can see examples of her in action on her website.

In a similar vein, a recent post on the PolicyViz blog related this anecdote:

A few months ago, I was chatting with one of my former graduate school professors. I was telling him about a client who was concerned that his organization wasn’t selling their work enough; that their reports weren’t being as widely read as they had hoped. “See, that’s what’s great about being an academic,” he said, “I don’t have to worry about selling my stuff.”

As a former academic I know first hand that there’s a small minority within academia who think that anything other than pure research is a waste of time. Thankfully there are not many who are so narrow-minded, but those who are can be quite vocal.

I remember when I left the academic world to go into consulting, a former colleague warned me that it was a one-way door and once I left, I could never get an academic job again. Presumably this is because exposure to the real world would taint my ability to analyse theoretical economic models. He wasn’t trying to bully me, he said this out of genuine concern for my wellbeing, because in his eyes an academic job was a very good one to have. Indeed, it is pretty good.

My publications when I was an academic revealed that I was a C+ researcher, so no great loss anyway. But my student evaluations rated me as an A grade teacher (despite the fact that (or maybe because) teaching a class scared the shit out of me, at least at first). However teaching skill wasn’t really rewarded — maybe things have changed now, I don’t know.

Back to Michelle’s tweet. The funny thing is that communication is actually the hardest part of research. Once you’ve done the research or prepared for a lecture, you know pretty much everything there is to know about the topic. Or at least you think you know. To be a good communicator or teacher, you then have to put yourself in the shoes of the reader or student. You have to know what they don’t know. If they don’t understand your first explanation, you have to be able to recognise this and to understand why they don’t understand. None of this is easy and it requires a lot of thought.

Specialisation is fine — those who are good at research can work in the lab, and those who are good at communication can do that. The thing is that, via journals, research has always had a worldwide audience and all that comes with that. Not so for teaching, but online education is changing this rapidly. It won’t be long before superstar teachers emerge, and hopefully they are well rewarded.

Targeted ads are targeted

About a month ago I tried out Parallels for Mac. I decided not to buy it and let the trial expire. After that, Parallels sent me an email for a $10 discount if I decided to buy. And now, on random websites, I’m getting ads showing the same thing:

There’s also an ad for Sony digital cameras because I Googled for these the other day. I looked at camera prices on Amazon, and now when I go to Facebook I get “sponsored links” from Amazon advertising the exact cameras that I looked at.

I understand free websites have to show ads, but to me this is starting to feel a bit too personal. I’ve already ticked the “do not track” setting in my browser; clearly that is ignored. I guess it’s time to take more drastic action.