FEATURED RESEARCH PROJECT: ILSA Inc. How is the global ILSA industry shaping international data on learning outcomes?

ILSA Inc. How is the global ILSA industry shaping international data on learning outcomes?

By Camilla Addey, Teachers College, Columbia University

This blog presents the ILSAINC research project. Dr Addey has been funded a Marie Sklodowska-Curie Individual Fellowship, part of the Horizon 2020 Programme, by the European Commission to carry out this project. Dr Addey will be based at GEPS, the Globalisation, Education and Social Policies research centre at the Universitat Autònoma de Barcelona. 

 

We hear a lot about the global shift from data on education access to data on learning outcomes, and the rise of learning outcomes quantified as internationally comparative data in the form of International Large-Scale Assessments (ILSAs), but we know very little about the fast-expanding business opportunity ILSAs have created since their tremendous growth in the mid 1990s. And we know next-to-nothing about the global International Large-Scale Assessment (ILSA) industry that has emerged as a result.

Although the global ILSA industry (ILSA Inc.) has been almost entirely neglected by scholars (whilst the wider education industry has received much attention), scholars are now increasingly expressing concern about the privilege and power afforded to the private actors inscribing their agendas, profit-interests, power practices and values in ILSAs (i.e. Lingard et al. 2013). Indeed, the OECD has recognized that the private sector has reason to lobby for the growth of ILSAs (OECD 2013). Nonetheless, to date no scholarship has looked into the global ILSA industry. Should we therefore assume that it is not a matter of concern? According to the body of scholarship identified as Critical Data Studies, it is indeed important to interrogate the practices, values and power dynamics encoded in data and thus, to pay careful attention to the actors involved in data making (Kitchin and Lauriault 2014). This concern is made all the more urgent and timely as ILSAs are growing in scope, scale, reach (Sellar and Lingard 2014), thus making the global ILSA industry bigger, more complex and more powerful in setting the global learning outcomes’ agenda.

But let’s take a step back. How did this fast-expanding business opportunity arise?

In order to participate in ILSAs, governments have to pay a substantial participation fee which consists of a flat fee plus a country-specific overhead which is calculated based on the size of each country’s economy. This fee covers the costs covered by International Organizations (IOs) of outsourcing ILSAs – from test development all the way to data analysis and report writing – through competitive tendering processes. As an example of the business opportunity created through PISA for Development (PISA-D), the OECD contracted a consortium of contractors (as they are called by the OECD): Education Testing Services to develop the test items and the manage PISA-D, Pearson to develop the frameworks upon which the tests are developed and interpreted, The Learning Bar to develop the background questionnaires, cApStAn to check the linguistic/psychometric/cultural equivalence of the translated tests,Westat to sample the test-takers in each participating country, Positivo BGH to provide tablets, and Microsoft to develop software. The PISA-D example is a small snapshot of the ILSA-outsourcing business opportunity which has given life to what I call ‘the global ILSA industry’.

And how does the global ILSA industry operate?

After a public call for tender, IOs contract a consortium of private actors (this information is made public) but nothing is known about who made offers, the evaluation of the bids, the negotiation of the contractual Terms of Reference, the details of the work, the amount of money IOs receive from governments, nor how much the private contractors are paid. By applying a network ethnography (Ball 2016), the ILSA Inc. research project aims to map the global ILSA industry and apply the lens of the Global Education Industry (Verger et al. 2016) to theorize how the global ILSA industry emerged, stabilised and operates – focusing in particular on how its actors struggle to maintain and transform the ILSA Inc. A deeper understanding of how the global ILSA industry operates will show how the shared cognitive frames, networks and institutions, that have been normalised in the global ILSA industry, are shaping ILSAs.

But do we really know next-to-nothing more about this global ILSA industry?

A preliminary mapping which I have carried out shows that the global ILSA industry is a small, well-established network of actors which has now normalised a set of rules of the game and power dynamics. Following the actors’ moves shows how hard it is to access this industry and that the few new actors tend to emerge from the edges (for example, they are part of ILSA advisory boards and then start a private company that wins a contract) thus showing the value of network capital (Urry 2003) and intimately knowing the rules of the game. The extensive experience and the sophisticated expertise that the actors of the global ILSA industry have developed over the last twenty years, but also the size of the contracts (these are often very big and require a team of people to prepare long, complicated bidding offers in response to the Calls for Tender) make it difficult for smaller, less experienced private companies to access the global ILSA industry. However, this first mapping does not provide insights into the cooperative, conflictual and competitive relationships amongst the actors, the rules of the game that actors abide by to profit from the ILSA business opportunity, and what agendas and interests are being inscribed in ILSAs and the data they produce. The ILSAINC project intends to respond to these questions.

A deeper understanding of the global ILSA industry may stimulate greater transparency and accountability of those developing the very instruments of global accountability, but also allow a fairer analysis of the private sector practices and interests embedded in the data.

 

To reference this blog: Addey, Camilla. 2018. ILSA Inc. How is the global ILSA industry shaping international data on learning outcomes? Laboratory of International Assessment Studies Featured Research blog series. Published on 1st October 2018. Accessed at https://bit.ly/2Nflplm

Author bio: Camilla Addey is a Lecturer in International and Comparative Education at Teachers College, Columbia University. She has been co-directing the Laboratory of International Assessment Studies since 2013. Camilla’s research focuses on the politics of ILSA in lower and middle income contexts, global education policy, and education privatizations.

The ILSAINC project: Drawing on global education policy scholarship on education privatization, Critical Data Studies, and Business Studies, and applying a network ethnography approach, this ILSAINC research project aims to: 1) map the global ILSA industry by identifying the actors to whom ILSAs are outsourced; 2) to theorize how the global ILSA industry emerged, stabilised and operates – particularly focusing on the global ILSA industry institutions and relationships, the network functioning, forms of agency, and power practices; and 3) to investigate how ILSAs are shaped by the practices, values, profit interests and power dynamics of the private sector. The network ethnography approach will generate empirical data through Internet searches, participant observations, interviews and document analysis. Interviews will be carried out with all the actors involved in the ILSAINC. For further information about this research project, please contact Camilla Addey at addey@tc.columbia.edu 

 

References (not hyperlinked in the text)

Kitchin, R. and Lauriault, T. P. (2015). Towards Critical Data Studies: Charting and Unpacking Data Assemblages and Their Work. In Eckert, J. et al. (Eds) Geoweb and Big Data. University of Nebraska Press.

Verger, A., Lubienski, C., & Steiner-Khamsi, G. (2016). The Emergence and Structuring of the Global Education Industry: Towards an Analytical Framework, In  A. Verger, C. Lubienski and G. Steiner-Khamsi (eds). World Yearbook of Education 2016: The Global Education Industry. New York: Routledge.