What we measure is what we strive for.
This quote is from Making Globalization Work by Joseph Stiglitz. Stiglitz is talking about the weaknesses of GDP as a measure of development and arguing that one of the reasons it has been so pervasive a measure for so long is that it’s relatively easy to measure it accurately and consistently. The weaknesses of it are, hopefully, obvious. GDP as a measure of economic development does not account for development in education, health, social inequality, political agency and so on. Further, even as a measure of economic development it is fairly limited. However, it has become a near universal benchmark measure for development, simply because we can measure it. The tide is changing somewhat, but for many years, GDP growth has been the stated goal of many countries of the World. There are lots of ways you can achieve this target at the expense of long-term sustainable development – Stiglitz points out that you can easily drive GDP growth by logging your rainforests – but that’s not really development is it?
Clearly this is something research needs to reflect on. We need to be clear about what we can measure and how those measures can help, but also about what we can’t or don’t measure. In this post about action standards I talked about how making the measure the objective hugely devalues the work that we do. A scorecard approach, especially for pre-testing, loses sight of what the real objective is – great, on strategy advertising – that’s what we should be striving for, not for a high score on the parts of that we can actually measure. The rain-forest example has much in common with striving too much for efficiency in individual pieces of copy – it is important, but if it’s at the expense of the longer term campaign story and/or tone of the brand, it’s not always desirable. I can make your ad more efficient, do you want me to?