FEATURED RESEARCH PROJECT: Quality as a tool of governance

Quality as a tool of governance: A comparative study in Brazil, China, and Russia

By Nelli Piattoeva, Jaakko Kauko, and Annariikka Paatola, University of Tampere, Finland

 

We are currently finalising a research project, funded by the Academy of Finland, investigating the Transnational Dynamics in Quality Assurance and Evaluation Politics of Basic Education in Brazil, China and Russia (BCR), 2014-2017.

Quality of education is the ultimate goal towards which almost all countries have promised to strive. At the same time, through categorising individuals’ and organisation’s performance for decision-making, quality can be a political tool for steering change. This is one of the reasons why quality has been of growing interest to international organizations and national policymakers. Previous research on quality assurance and evaluation (QAE) has often concentrated on the role of international organisations, while lesser emphasis has been placed on the role that national and local actors play in shaping the QAE as a central element of transnational policies and practices.

This is precisely the gap our comparative research project aimed to address.

Local level actors cooperate with but often deviate from transnational quality assurance agendas, while the agendas themselves are porous enough to allow for multiple interpretations as to how they ought to be implemented. That is why QAE policies lead to diverse local forms and outcomes, and their effects are not confined to the educational sphere only. Our research was designed to address the questions arising from these observations, namely:

• How has QAE changed the roles of and interaction between different actors in policymaking in the context of formal schooling?
• How does QAE change the room for action that exists for politicians, experts, teachers, and other educational actors?

Quality assurance and evaluation policies manifest the transnationalisation and datafication of education policymaking. They are linked to important developments in the latter half of the 20^th century, which has seen the expanding possibilities of transnational data flows and an increasing emphasis on economic efficiency in public services. With this in mind, our research also asked:

• With various expert institutions competing in QAE data production, who has the right to produce data, who owns it, and how is it interpreted by different actors?

To answer the questions above, we undertook a large-scale qualitative comparative research project focusing on three heterogeneous countries: Brazil, China, and Russia, that have received less attention in research on transnationalization and datafication of education policy and QAE. These countries, like many others, have increasingly focused attention on QAE in education.

The research team carried out extensive fieldwork in these countries, generating 200 interviews, documentary material, and observation data. We are currently finalizing a book that presents our findings. These can be summarised under three headings:

New actors channelling international influences Our research shows that QAE has indeed changed the roles of national and international actors in policymaking. For instance, the researched countries have been able to channel the influence of international organisations instead of being passive recipients to it. The roles of different actors are shifting: in Brazil, international organisations now work with subnational as opposed to national organisations, and civic movements have been able to take a more active role in education thanks to the QAE infrastructure and the possibilities to engage in political contestation with the help of quality indicators. In China and Russia, national authorities retain a tighter grip on what is happening, but think tanks and expert networks have interesting new roles.

Governance at a distance through data Various QAE practices produce quantitative data about education, and data are said to allow the nation-state to extend is capacity to govern across territory and into the classroom. This so-called governance at a distance relies on the production and circulation of data. However, data does not always flow as smoothly as depicted in the academic literature or political rhetoric. We set out with the assumption that all governance attempts are potentially incomplete and ridden with tensions and unfulfilled expectations. And indeed, our analysis showed that governance through data produced in National Large-Scale Assessments (NLSA) is very uncertain. Many unresolved issues in the data circulation process deserve further attention. These include, for instance, the role of trust between civil servants and assessment experts because of the civil servants’ reliance on the technical and analytical capacity of the experts, as the case of Russia particularly illustrates. Our analysis points to the significance of a working relationship between the national and sub-national administration levels, especially because of the national level’s dependence on the willingness and possibilities of local actors to engage with the NLSA. This relationship seems deeply distrustful, rooted in controversies concerning NLSA use as data for both evidence-based policy and accountability, as well as attempts to partly recentralise certain features of the education system by means of NLSA data (especially in Brazil). Moreover, the simplification of complexity presents an argument against NLSA data which is difficult to refute, and various actor segments may make accusations of reductionism at any time. As the case of Brazil shows, this kind of criticism is made more easily the more data are made publicly available to diverse groups of actors. In these contexts, the realities of governance at a distance seem messy and unfulfilled, perhaps fortunately so.

Opportunities for schools as political actors In addition to changing the room for action for national level actors, QAE policies also change school governance on a subnational level. In our research, we not only identified ways in which QAE is used to govern schools, but also analysed the opportunities that this governance has opened up or terminated for schools as political actors. For instance, we demonstrate that QAE instruments get reinterpreted locally in accordance with the already existing practices of quality control and school governance, and may cater to the political interests of local actors. As a result, high-performing schools can utilize QAE policies to draw power from sources such as expertise, access agenda setting, or build networks and coalitions, while low-performers become increasingly disadvantaged. Schools can sometimes practice open or hidden resistance and avoid the penetration of QAE tools into internal processes. For instance, in some cases, evaluation results were fabricated, and it was possible to keep traditional practices while formally implementing new regulations. In the case of Brazil, a school retained low-performing students and those who did not reach the minimum required school attendance despite the authorities’ call to dismiss students who compromise the quality indicator.

As well as assessing the current situation, we also wanted to look into the future by asking the Brazilian, Chinese, and Russian educational actors how they see the future development of quality assurance and evaluation in their country. Our data shows that views on alternative futures in Brazil, China, and Russia are rooted in domestic pedagogical traditions, including QAE practices, and that they express a wish to retain what is seen as valuable in these traditions, instead of simply accepting external recommendations or envisioning completely ‘new’ futures. We suggest that education policy research, including studies on QAE, should consider, alongside the past and current phenomena, the future aspirations of national participants in greater depth.

Finally, our research sought to contribute to comparative education and education policy research by developing and probing a new theoretical-methodological framework for comparative education. We call this the Comparative Analytics of Dynamics in Education Politics (CADEP), which aims to tease out comparable and related patterns of action across the cases. CADEP directs analytical focus to three dimensions. Political situation unravels the constellations of actors in a particular socio-historical context; political possibilities refer to how discourses and institutional path dependencies shape what the actors see as possible. The last dimension seeks to describe the extent to which actors can capitalise on an existing political situation and political possibilities.

More details of our research in our forthcoming book, The Politics of Quality in Education: A Comparative Study on Brazil, China, and Russia (Routledge 2018). Published articles include:

• Gurova, G., Piattoeva, N., & Takala, T. (2015). Quality of Education and Its Evaluation: An Analysis of the Russian Academic Discussion. European Education, 47 (4), 346–364.
• Kauko, J., Centeno, V.G., Candido, H., Shiroma, E. & Klutas A. (2016). The emergence of quality assessment in Brazilian basic education. European Educational Research Journal, 15(5), 558-579.
• Piattoeva, Nelli (2016) The imperative to protect data and the rise of surveillance cameras in administering national testing in Russia. European Educational Research Journal 15 (1), 82-98.
• Suominen, O., Kallo, J., Rinne, R. & Fan, Y. (2017). Subtle convergences: Locating similarities between Chinese educational reforms and global quality assurance and evaluation trends. Quality Assurance in Education, 25 (2), 146-160.

The research team is based in the research group Knowledge, Power and Politics in Education (EduKnow) at the Faculty of Education, University of Tampere, and the Centre for Research on Lifelong Learning and Education (CELE), University of Turku, Finland.

The Brazilian team: Associate Professor Jaakko Kauko (leader of the consortium and team leader), Dr Vera G. Centeno (post-doctoral fellow), Dr Helena D. Candido (doctoral student), Iris Santos (research fellow).

The Russian team: Professor Tuomas Takala (team leader), Dr Nelli Piattoeva (university lecturer), Galina Gurova (doctoral student), Anna Medvedeva (research fellow).

The Chinese team: Professor Risto Rinne (team leader), Dr Johanna Kallo (post-doctoral fellow), Olli Suominen (doctoral student), Zhou Xingguo (doctoral student).

 

The featured photo shows the research team that focused on the Russian part of the study when they presented the research findings at a school in Cheboksary, Russia, October 2016.