The Aesthetics of Numbers – The Beauty of PISA

By Christian Lundahl, Örebro University, Sweden

@drlundahl www.paristopisa.com

In this blog post, Christian Lundahl responds to the keynote presentation by Theodore Porter. This is the second part of a two blog postings.   

In a paper from 2009, Florian Waldow and I claimed that Sweden, as a consequence of being a numbers country, was more or less immune to ‘PISA-shocks’ compared to Germany. Our guess was that Swedes would take a critical stance to the statistics, since they were so used to them, and had a history of critical debates on this subject (Lundahl & Waldow, 2009).

 

Fig 1. The literacy of PISA charts. Google picture results: OECD PISA “Sweden” September 2016. In Sweden Swedish education is often termed as “an educational system in freefall”, as an expression for dropping fast in the PISA rankings.

But, as it turned out, Sweden was not immune, and part of the reason for that probably is related to the effective visual presentation of the PISA results. Mostly for the fun of it, I started to Google OECD PISA represented as pictures. If you do this, you will get about 400 hits in the first round. I started to look at these pictures and found that most of them were statistical charts of various kinds. There were also many pictures of children and adults. I therefore decided to see if Sweden differed in some way from other countries. For this to work, it was necessary to search in every single country’s own top domain and its own language. Doing this I found that three kinds of images dominated: charts, images of children, and images of adults. Within the charts category I separated ‘trend charts’ from the rest, to illustrate the countries’ comparisons with their own development over time. I also counted the number of times Andreas Schleicher, the head of OECD’s PISA test, appeared, and as a reference figure I also looked at how many times the actual PISA tower was in the retrieval.   As Table 1 shows, there were some clear differences between the countries.

Table 1. Google picture results: OECD PISA “Country”. Pictures of people from PISA report covers, and pictures obviously not about school and education (eg. famous actors), are not included except for the leaning tower of Pisa.

I found that charts dominated the retrievals in all countries. I also saw that there was a tendency towards more pictures of children in PISA-successful countries. In Sweden, there were instead a lot of pictures of concerned or serious-looking adults.  Andreas Schleicher also popped up rather often in the Swedish pictures.  Most strikingly Sweden really stood out in its use of trend charts, illustrating what politicians in Sweden often refer to as a “school system in free fall” (fig 1). Some of the pictures retrieved from Google uses tags (Machin 2012), that increase the likelihood to find them using a search engine, i.e. someone wants them to stand out. Google also uses a built in Page­­­­­­­-Rank-algorithm making it easier to find pictures others have looked at since they will be highly ranked (Orlikowski, 2007). Even if this kind of ’archaeology of Google’ needs to be developed further we can assume that Google retrievals quite well represent a nation’s ‘picture discourse’ on PISA.

When describing PISA in Sweden, there is obviously a preoccupation with Sweden’s downfall as measured over the last three surveys. Why Sweden is failing in PISA, and what to do about it, are interesting questions, but based on these pictures we can also raise questions of another kind: what knowledge travels, where does it go and through what media? What shape does knowledge about education have and when does it change its shape, and for what reason? Pictures like these are the perfect “quick language” (Lundahl 2008). They transfer fast, and allow for translations without a spoken word. They can be understood in the right way or in the wrong way – it doesn’t really matter – and they strongly structure the discourse and attach countries to a specific “image”. This is the aesthetic power of numbers.

In his paper, Porter (2016) writes:

Numbers, properly designed and presented, can help to reveal the play of power and even provide a basis for challenging it, but scrupulous interpretation, including effective visual tools, have to be part of the mix. /…/ It is equally important to generate numbers that can be used to challenge power.

I suggest that we also ask ourselves: how can we generate images of education that challenge unscrupulous visualisation and use of numbers?

When understanding the impact of PISA in a country, we need to understand the ‘numbers culture’ (or ‘charts culture’), and its roots in movements that both trust and mistrust it, in that particular country (see part one of this blogpost). If we use an Actor Network Theory (ANT) perspective advanced by (Latour, 1998), and focus on nets of relations, “mistrust” in numbers can be viewed as a part of the network that enables trust in numbers (and vice versa, of course). If the mistrust (as activism) isn’t clever enough, it might strengthen the (blind) trust in numbers, and of course, again, vice versa. We see that, when PISA critics become too anti-PISA and don’t acknowledge the usefulness of PISA at all, they rather delegitimise the ‘mistrust in numbers’ movement. They risk putting themselves in what Biesta (2011) has called a “silly position”. A better path, I think, is to re-visualise international education in ‘thicker’ and more complex way, e.g., combining PISA results with other images of education.

References

Biesta, G. (2011). God utbildning i mätningens tidevarv [Good education in the era of measure]. Stockholm: Liber AB.

Latour, B. (1998). On Actor Network Theory: A few clarifications. http://www.nettime.org/Lists-Archives/nettime-l-9801/msg00019.html

Lundahl, C. (2008): Inter/national assessments as national curriculum: the case of Sweden. In Martin Lawn (ed): An Atlantic Crossing? The work of the International Examination Inquiry, its researchers, methods and influence. Oxford: Symposium Books, 157-180.

Lundahl, C. & Waldow, F. (2009). Standardisation and ”quick languages”: The shape-shifting of standardised measurement of pupil achievement in Sweden and Germany. Journal of Comparative Education, vol 45, no 3, 365-385.

Machin, D. (2004). Building the world’s visual language: the increasing global importance of image banks in corporate media. Visual Communication, (3), 316. https://doi.org/10.1177/1470357204045785

Orlikowski, W. J. (2007). Sociomaterial Practices: Exploring Technology at Work. Organization Studies, 28(9), 1435–1448. https://doi.org/10.1177/0170840607081138

Porter, T. M. (2016). Politics by the Numbers. Keynote presentation at the The Futures and Promises of International Education Assessment, convened by the Laboratory of International Assessment Studies, in Berlin 15th-16th September 2016.

 

Christian Lundahl is Professor in Education at Örebro University in Sweden.

To reference this blog: Lundahl, Christian. (2016). The Aesthetics of Numbers – The Beauty of PISA. Laboratory of International Assessment Studies Blog. Accessed at http://wordpress-ms.deakin.edu.au/international-assessments/the-aesthetics-of-numbers-the-beauty-of-pisa/