The Aesthetics of Numbers – Trust and Mistrust

By Christian Lundahl, Örebro University, Sweden

@drlundahl www.paristopisa.com

In this blog post, Christian Lundahl responds to the keynote presentation by Theodore Porter. This is the first part of a two blog postings.   

At the conference The Futures and Promises of International Education Assessment, convened by the Laboratory of International Assessment Studies, in Berlin 15-16 September 2016, I had the privilege to comment upon Theodore Porter’s paper Politics by the Numbers.  The video of Porter’s presentation is available here.

 

Fig 1. Standardising teachers’ grading using national tests. Distributing life chances: “Test gives fair grades.” Husén et al. 1956a. Illustration by Yngve Svalander.

Porter and his seminal work Trust in Numbers (1996) is well known to all of us that do research on assessment in education. In the paper presented in Berlin, he departed from the very basic idea that there isn’t a “clear boundary between science and other knowledge forms”. But, he argued, “that doesn’t mean we accept every truth claim as equally valid and, of course, we still need to make distinctions.  But whatever functions as knowledge in the world must be regarded as knowledge, even when we doubt its correctness” (Porter 2016). This is where the topic of numbers becomes important to scrutinise, Porter states:

“They are brought to bear on every subject matter and for a multitude of purposes. The forms of quantification that are most respected by institutionalized science are not necessarily the most important, and may not be the most truthful for every purpose.  … The power of the statistical significance test in science as in administrative matters arose as a beacon of objectivity, a promise of incorruptibility, resting on the supposition that measurement and calculation stand above self-interested manipulation.  Are these therefore free of politics?  /…/ There are good reasons for investing trust in science, but to pretend that it can rise wholly above ordinary human foibles and provide easy, formulaic answers to complex questions is to turn it into politics by other means.” (Porter 2016)

I live in Sweden – what we can call a ‘true numbers’ country. Sweden was the first country in the world with a national bureau of statistics (SCB), founded in 1749. During most of the 20th century, Sweden was characterised by far-reaching social engineering in many areas. In education, the famous comparativist Thorsten Husén played a major role both in forming UNECSO Institute of Statistics and IEA (International Association for the Evaluation of Educational Achievement). We have had national tests in education since the 1950s and, during the 1990s, Sweden was one of the first countries to introduce what’s called a ‘standards based curriculum’ that also has a very detailed grading system – and, not least, is suffering from a severe PISA-crisis. The politics of numbers is something I constantly meet both as a citizen and as a scholar. Needless to say, numbers have always had great legitimacy in Sweden, but have also met with a lot of mistrust. My response to Theodore Porter’s keynote departs from the constant negotiations and calibrations in the legitimising and delegitimising of the use of numbers in Swedish education. My comments concern two minor, but nonetheless important themes in his presentation: 1) what he mentions as “visualisation of numbers” and 2) what has been called ‘statactivism’ (Porter, 2009, 2016; Bruno, Didier & Vitale, 2014).

I will elaborate on these themes in terms of the aesthetics and mistrust of numbers in this two-part response.  This, Part 1 of the response, relates to the aesthetics of numbers. When national tests were introduced in Sweden in the 1950s, there already existed several assessment tools in education – primarily teachers’ observations, diaries and grading. The major reason to introduce standardised tests in Sweden was to let the more ‘objective’ and ‘quick’ results of the test influence teachers’ grading of their students using a nationally norm-referenced scale. However, this was a threat to teachers’ autonomy and many teachers instantly rejected the idea – even if they saw some advantages to it, mainly in forms for which the higher status collaboration with science could provide. In other words, there was a need to legitimise the test. One way to legitimise the new tests was by claiming that they provided objective information about the pupils’ abilities. These claims were posted using ‘soft power’ strategies to co-opt the teachers to want the outcomes of the state rather than coercing them (Nye, 2004). Torsten Husén and his colleagues at the Teachers’ College in Stockholm were, at that time, the ones in charge of developing the national standardised test for the state. First of all, they published a scientific book describing the advanced work of constructing these tests and using them as a norm for teachers’ grading (Husén et al., 1956b). In this, the concepts of validity and reliability were elaborated in depth, thus rooting the reform in the traditions of psychometrics. Secondly, they popularised their findings in a pamphlet targeting teachers and the public. In this text, they used the skill of a Swedish illustrator, Yngve Salander, to aestheticise the process of test construction and of numbers. We don’t know their reason for this, but one effect was that standardised testing appeared as a solid enterprise, providing fair grades in a ‘harmless way’ (figs. 1 and 2).

 

Fig. 2. Calibrating the instruments. Husén et al., 1956a. Illustration by Yngve Svalander.

It is not possible to precisely assess the impact of these texts, but they were part of a general discourse – also described by Porter (1996) – that aimed to convince (predominantly female) schoolteachers to adopt the logic of numbers as promoted by (predominantly male) scholars of education. In Sweden, the state used these tests to produce statistics on education. A major aim of Swedish educational policy since the 1940s has been to provide equal opportunities for all children to go to school and pass the educational system with satisfactory grades. During the 1970s and 1980s, a new generation of educational scholars, trained in sociological theories and Marxism, started to use the statistics produced by the state to challenge the state’s claim that it provided equitable comprehensive schooling (Fig 3). We can call this a form of statactivism, to “present good statistics as a moral and political cause” (Porter, 2016). But, paradoxically, the numbers used to challenge the state were also thought to support an unequal society, since the numbers in reality also represented students’ life chances in the form of grades the students would get (Fig 4). In this case statistics played a double role in representing as well as criticizing reality (cf. Bruno, Didier & Vitale, 2014) The goal of the early statactivist in Sweden then became to use the representations to criticise the reality, with the aim of abolishing both national testing and grading. Out of this situation a mistrust in numbers started to grew. There was a belief that if children weren’t assessed, they wouldn’t be victims to social stratification. At the same time, who would find out since abolishing national testing and grading in the long run would lead to a situation where Sweden couldn’t know if the school system was equitable or not.

 

Fig 3. Statactivism – grades and tests both strengthen and visualise social stratification. Social class and grade average. From a teachers’ magazine, early 1980s, in Lundahl (2006).

 

Fig 4. The teacher and student test-dropout in the 1980s. Postcard saying: “Don’t take the test!” (in Lundahl, 2006).

The mistrust in numbers in Sweden at this time was essentially about fighting the number-producing instruments using numbers produced by the same instruments, but visualised in new ways. This activism and mistrust in the production of numbers had some effect in reducing their use, and changing the claims of national tests for several decades up until the PISA results of 2006 and 2009 (Landahl & Lundahl 2013).

In the second part of this blogpost I will have a closer look at the persuasiveness of the aesthetics of PISA.

References

Bruno, I., Didier, E & T. Vitale Tommaso (2014), “Statactivism: forms of action between disclosure and affirmation”, in Partecipazione e conflitto. The Open Journal of Sociopolitical Studies, vol. 7, n. 2, pp. 198-220

Husén, T., Björnsson, C. H., Edfeldt, Å. W. & Henrysson, S. (1956a): Betyg och standardprov. En orientering för föräldrar och lärare. [Grades and national test. An orientation for parents and teachers] Almquist & Wiksell: Stockolm

Husén, T., Björnsson, C. H., Edfeldt, Å. W. & Henrysson, S. (1956b): Standardproven. En redogörelse för konstruktion och standardisering. [Standardised test. An account on their construction and standardisation] Almquist & Wiksell: Stockolm.

Nye, J. S. (2004). Soft Power. The Means to Success in World Politics. New York: Public Affairs. Landahl, J. & Lundahl, C. (2013): (Mis)trust in numbers. Stuggling with transparency. In Lawn, M. (ed): The Rise of Data in Education Systems. Oxford: Symposion Books. s. 57 – 78

Landahl, J. & Lundahl, C. (2013): (Mis)trust in numbers. Stuggling with transparency. Lawn, M. (ed): The Rise of Data in Education Systems. Oxford: Symposion Books, 57 – 78.

Lundahl, C. 2006. Viljan att veta vad andra vet. Kunskapsbedömning i tidigmodern, modern och senmodern skola [To Know What Others Know. Assessment in Education in Pre-modern, Modern, and Late-Modern Times]. Dissertation. Uppsala University.

Porter, T. M. (1996). Trust in numbers: The pursuit of objectivity in science and public life. Princeton University Press.

Porter, T. M. (2009). How Science Became Technical. Isis, no. 100, 292-309

Porter, T. M. (2016). Politics by the Numbers. Keynote presentation at the The Futures and Promises of International Education Assessment, convened by the Laboratory of International Assessment Studies, in Berlin 15th-16th September 2016.

 

Christian Lundahl is Professor in Education at Örebro University in Sweden.

To reference this blog: Lundahl, Christian. (2016). The Aesthetics of Numbers – Trust and Mistrust. Laboratory of International Assessment Studies Blog. Accessed at http://international-assessments.org/the-aesthetics-of-numbers/