Featured Research Project: “The Ranking Storm” on the 2016 TIMSS and PISA data release

By Erika Kessler and Oren Pizmony-Levy, Teachers College, Columbia University

The ranking storm is coming! A comparative research project on international large-scale assessments and the public

In the coming days, the two most influential international assessments of student achievement – TIMSS and PISA – will release their 2015 data and reports. TIMSS 2015, which will be released on Tuesday 11/29/2016, is the sixth wave of the assessment and provides 20 years of achievement trends of fourth- and eighth-grade students in mathematics and science. PISA 2015, which will be released a week after on Tuesday 12/6/2016, is also the sixth wave of the assessment and provides 15 years of achievement trends of 15-years old in mathematics , science, and reading. Mark your calendar!

This is the first time since 2004, when both TIMSS and PISA 2003 were released, that the schedule of both assessments collides. Within one week, stakeholders in 70 plus countries will be exposed to numerous ranking tables and international report cards of their respective education systems. In the past, in some countries, the release of these assessments caused a disturbance (or shock) in the education world and led to various policy/practice reactions. This (human made) ranking storm affected not only politicians and policy makers, but also parents, teachers, students, and the researcher community.

Most people generally consume news about TIMSS and PISA through newspapers, radio, TV, and other online sources. In an era of sensationalizing media and increasing uncertainty, the results from TIMSS and PISA could perhaps trigger an intense media focus and bring education further into political and economic debates. Thus, it is important to investigate how the media presents these results to the general public, who is speaking about the results, and what kind of arguments they advance. It is also important to examine the extent to which the general public is actually engaged with the results and the (un)intended consequences of media coverage on attitudes toward education and schools.

The Ranking Storm Project at Teachers College, Columbia University aims to examine the ways in which the general public engages with TIMSS and PISA. Professor Oren Pizmony-Levy is co-leading the project with doctoral student, Linh Phoebe Doan, and Masters’ students Erika Kessler and Jonathan Carmona. They are assisted with a team of more than 60 volunteers from Teachers College and other institutions worldwide.  The research team will carry out an extensive analysis of over 150 media news sources from about 40 countries/systems and intends to collect over 5,000 survey responses, before and after results, on attitudes toward education in separate countries.

To study how people engage with TIMSS and PISA through the media (newspapers, radio, TV, and online sources), it is important to examine three related questions:

1. How do the media present the results?
2. Which interpretations and voices are more common?
3. Does the release of comparative performance information inform public opinion of education and schools?
4. To what extent do countries vary in their engagement with TIMSS and PISA?

By now, we have volunteers working on about 40 countries! We are looking for colleagues who are interested and available to help us with: (a) monitoring media coverage of TIMSS and PISA in their county and (b) disseminating links to public opinion survey through social media and other listservs.

Two sub-projects:

The Ranking Storm Project includes two elements:

1. Media Analysis: All volunteer research assistants will participate in conducting media analysis. Time needed: 35-40 hours in total.
2. Public Opinion Survey: All volunteer research assistants will be invited to participate in collecting public opinion data via social media. We recognize that not all volunteers are available to take part in this element. Time needed: 20-25 hours in total.

Skills:

No previous research experience is required. We hope that after you participate in this project, you will be gain more research experience, using both qualitative and quantitative methods such as:

1. Conducting media analysis
2. Collecting data using Qualtrics
3. Analysing data using Stata
4. Learning basic concepts of social network analysis

 Volunteer responsibilities:

1. Follow the timeline and submit deliverables on time
2. Attend all the training sessions
3. Communicate with research coordinators on a weekly basis
4. Commit and have fun!

You can join us and/or follow our findings and insights on social media:

Facebook: www.facebook.com/ILSAProject

Twitter: https://twitter.com/ILSAProject

Questions? Please contact Linh, Erika or Jonathan:

Linh Phoebe Doan at doan@tc.edu/(206)-501-0310

Erika Kessler at elk2144@tc.columbia.edu

Jonathan Carmona jc4618@tc.columbia.edu