THE LAB INTERVIEW SERIES: AN INTERVIEW WITH SYLVIA SCHMELKES

The Lab Interview Series features interviews with the Lab Expert Advisors and other internationally recognised experts working in education and assessment. The Lab is committed to providing a forum for dialogue across diverse perspectives on international large-scale assessment and does not necessarily endorse the positions taken by interviewees.

 

Sylvia Schmelkes is a Mexican sociologist with extensive experience as an education researcher in educational quality, values education, adult education and intercultural education. She is a member and former president of the Governing Board of the Mexican National Institute of Educational Evaluation.

 

Could you tell us a little about your background and how you got involved in education?

I am a sociologist that got involved very early on (in my third year of College) in educational research.  I had the privilege of working with Pablo Latapi, who was the founder of educational research in Mexico and you could also say in Latin America. His orientation was one of “committed research” – committed especially with the transformation of inequality and poor quality education. I consider him my mentor.  Pablo Latapí founded a non-profit organization dedicated to educational research in 1963.  I joined it in 1970 and was there for 24 years.  I started out as a research assistant and gradually increased my responsibility until I was responsible for large research projects in the fields of education quality, adult education, rural education and intercultural education.  My training as a researcher was on-the-job.  I culminated my MA degree in educational research late, in 1993.  I have had the opportunity of heading two educational research institutions. I am a founding member of the National Association of Educational Researchers, which now has more than 500 members.

In 2001, I was invited to lead a new department within the Ministry of Education dedicated to intercultural and bilingual (indigenous languages) education.  10% of Mexican population live in households where the head of the family speaks a native tongue, and 21% consider themselves indigenous.  There are 68 different native languages spoken in the country.  The education that they receive is very poor in quality, generally not in the mother tongue of the students, and not culturally relevant. As head of this department, I had the opportunity, among many other projects, of founding nine intercultural universities situated in highly indigenous regions. I consider this one of the most fascinating projects that I have developed, and in which I have learned the most.

In 2013, I was asked to participate in a selection process carried out by the Senate for the governing board of the National Institute for Educational Evaluation which was founded in 2003 but given full autonomy in 2013 as part of a broader and ambitious education reform. I was selected by the Senate and elected as President by my four fellow board members.  My term just ended in April (2017), but I remain a member of the Governing Board, which is a full-time job.

What points can you highlight in regard to education in Mexico?  What are some of the major successes and challenges?

Mexico is a federal democratic Republic comprised of 32 states. It is a large country: 123,500,000 – the eleventh in population in the world, and a young one – 45% of the population is under 25 years of age. It is also very diverse, as I have already mentioned. Even though 77% of the population lives in localities of 2,500 inhabitants or more, and – considered urban, there are 123,000 localities with less than 100 inhabitants, so it is concentrated and disperse at the same time. Its economy grew importantly and continuously between 1940 and 1980, and despite a slow recovery after the mid-1990s, its growth has slowed down notoriously.  Its GNP per capita is $8,102 US, just a little bit higher than China and very similar to Brazil. It is the tenth most unequal country in the world, according to recent World Bank information. Over the last three decades, it has been strongly hit by violence from organized crime. Poverty, corruption, and impunity are three of its main problems. Until the year 2000, it had been ruled by a single political party for seventy-one years.  Democracy is improving, but it is still fragile.

In this complex context, education has expanded notoriously, especially since 1950. The average schooling of the population is 9.2 years. The illiteracy rate is down to 5.5% from 25.8% in 1970 and 9.5% in 2000. However, 3 million children between 3 and 17 years of age – corresponding to compulsory schooling – are still out of school, mainly in grade one of preschool and in higher secondary. In the latter level, the net enrollment rate is only 59.5%, and 27% of those who enter this level complete it in three years.  PISA tests 15-year-olds, most of which are enrolled in the first year of higher secondary. Lack of access to compulsory schooling primarily affects indigenous children, internal migrant workers’ children who travel with their families, and rural children. Child labor aggravates the lack of attendance and drop-out rates.

Equity is, in my view, the most serious educational problem in Mexico. The quality of inputs, teacher training, and infrastructure, is not well distributed. The inequality of the country is not only reflected but made worse by the inequality of education. The poverty of educational supply meets poverty on the demand side. Achievement results, therefore, correlate with SES, the degree of marginality and rurality of the region, ethnicity – all variables that are themselves highly correlated between them.  Unequal education is one of the causes (instead of one of the solutions) of inequality.

Quality is another problem. Mexico always comes out last of OECD countries in PISA. In Latin America, it is behind Chile and Uruguay, but higher than Peru, Colombia, Brazil and Dominican Republic.  In Mexican standardized tests, between 48 and 50% of the children achieve below the level considered basic in reading and Mathematics. Teachers are in general poorly trained, and there are scarce and poor-quality opportunities for on-the-job training.

Mexico has been participating in PISA for several years – could you give us a sense of how this engagement has been part of the changing landscape of education in Mexico? How has education in Mexico benefited from participation in PISA? What new challenges has it created?

Mexico has been participating in PISA since 2000, because Mexico joined OECD in 1994, despite the fact that it is a developing country. Since the first PISA exercise, Mexico has come out last among the OECD countries in all areas. It has been so ever since. Mexico suffered the PISA shock since the first results were published, and continues to do so every three years. As in many other countries, the results of PISA 2000 were the first public exposure of an international comparison of student achievement. Mexico had participated in TIMMS in 1995 and decided to drop out once it received the results but before their publication. PISA results, however, had to be made public – it was a requirement for participating. The obligation of publishing results is undoubtedly one of the benefits of PISA for education in Mexico – transparency of evaluation results. In fact, in 2002, the National Institute for the Evaluation of Education was established as a decentralized institution dependent on the Ministry of Education. One of its objectives was to evaluate student learning nationwide and to make the Government accountable to society in general regarding student achievement and inequality in student learning. Participating in PISA most probably influenced the government’s decision to evaluate student learning with national standardized tests and to make results widely available.

In 2003, when the National Institute for the Evaluation of Education took charge of PISA (in addition to its own standardized testing), Mexico decided to oversample students in order to represent the 32 states that constitute the Mexican Republic. This continued until PISA 2012.  Thus, an internal ranking was established and comparing states among themselves became the second most important issue discussed when PISA results came out, second only to the poor quality of our education system as compared with other countries. While public opinion did question education policy, in many cases it blamed teachers for the poor results. As a consequence, not only did teacher morale suffer, but also standardized tests became a target of severe uneasiness on the part of teachers. We are still experiencing the effects of this unexpected phenomenon.

PISA results also affected policy.  In fact, in the 2007-2012 sectorial program, Mexico set the goal of a 435 score in PISA 2012, without however explaining what had to be done to reach the goal.  It was an ambitious goal and was obviously not reached. Mexico achieved 413 in Mathematics in 2012, one of the countries with the biggest leaps forward since 2003, comparable to that of Poland.

PISA results had an impact on curriculum, but not until 2005, when a competence-based curricular reform came into place for lower secondary and a little later for higher secondary education, both levels where 15 year-olds are enrolled. Mexico improved its PISA results in Mathematics in 2012 when compared to 2003, and some have hypothesized that curricular reform in part explains this.  Strangely, however, PISA results had no impact whatsoever on teacher training.

Many have criticized PISA for having little impact on improving teacher practice and actual learning.  In Mexico, the National Institute for the Evaluation of Education made efforts to make results useful for teachers. It published a series called PISA for Teachers, which analyzes item-by-item results for possible causes of failure in replying correctly and gave teachers advice on how to teach students what they needed to know to avoid such mistakes. The series was well received, but there is no information on how teachers used it and whether it had an impact – not on tests, but on student learning, which is, of course, what matters.

Among the negative effects of PISA is the implied desired homogeneity of educational results across the world. Cultural contexts are mostly ignored. A unilinear educational development model underlies the PISA exercise, in the sense that all countries, whatever the epistemological foundations of their culture, should aim at similar learning goals.  Fortunately, PISA is based on competencies, but even competencies have a cultural bias. This global effect was, in the case of Mexico, replicated at the country level where despite enormous contextual differences among states and regions, all were considered equal and judged in consequence.

Based on your experience with PISA in Mexico, what advice would you have for policy makers in countries like Ecuador, Senegal and Cambodia which have recently joined PISA?

My advice to these and other countries joining PISA is the following:

1) Avoid rankings. Comparisons of this sort only lead to compliance on the part of those with higher scores, and frustration on the part of those with low scores. This frustration can have a lasting effect and lead to skepticism as to the possible role of schools and education in general.

2) Do not place the blame on teachers. Quality of education is multifactorial. The most important factors affecting achievement are contextual. Education policy also plays an important role and results should lead to a reflexive and critical revision of policy decisions, from affirmative action to the curriculum. Of course what happens in the classroom is important, but rarely is it attributable to the volition of the teachers, but rather to the training they have received, to their working conditions, to the management of the school in which they work and, of course, to the overall characteristics of their students.

3) Face pedagogical challenges. One of the problems of PISA is that it lacks in pedagogical emphasis. Blame should not be placed on teachers for poor student results, but it is also true that they can be a part of the solution if adequate curricular, methodological and training measures are taken and if teachers have the knowledge, skills, support, and monitoring needed to modify their practice.

4) Face management challenges. Good teaching takes place in well-managed schools, where teaching is always protected and continuously enhanced. This involves instructional leadership on the part of principals and team work on the part of teachers.  Effective system monitoring and support are crucial. PISA leads to knowledge about types of schools that show a greater need for both pedagogical and management enhancement, and this information should lead to policy improvement regarding schools.

5) Complement PISA with countries’ assessment procedures based on national learning objectives. This counteracts the homogenizing effect of PISA’s educational philosophy and balances the country’s view of its problems and progress.

 

To reference this interview: Schmelkes, Sylvia. 2017. The Lab Interview Series: An Interview with Sylvia Schmelkes. The Laboratory of International Assessment Studies Interview Series (Interview 2). Accessible at http://wordpress-ms.deakin.edu.au/international-assessments/the-lab-interview-series-an-interview-with-sylvia-schmelkes/