The crucial importance of education to our nation raises the question of how the U.S. ranks in its preparation of students for the workplace, society and political participation. Next Tuesday, an important triennial study called the Program for International Student Assessment (PISA) 2015 will grab headlines by comparing the performance of U.S. students to those in other countries.
The PISA results will attract publicity worldwide and be used by educators, politicians and policy analysts to provide broad prognostics on what is wrong or right in the U.S. educational system and elsewhere. But if past versions of PISA are any indication, much of the conclusions and comparisons will be misguided.
[ 5 Things to Know About the PISA Exam]
PISA represents a massive evaluation of schools and student achievement in 72 economies. It has been designed and undertaken by the Organization for Economic Development and Cooperation (OECD), an entity comprised primarily of developed countries that addresses education, economics and social developments. About half of the participating countries belong to the OECD, while the remainder are partners of the organization.
The goal of PISA is to measure student outcomes at age 15 and to compare these among national or regional entities in relation to school and out-of-school opportunities. This year’s results focused primarily on science, but also on reading and mathematics. PISA studies are careful and scientific, requiring representative samples of students.
The tests go beyond knowledge of facts, by requiring students to apply their knowledge to address real-world situations. The achievement test scores are used to compare educational quality among nations. I have been a consultant to PISA and have great respect for the overall effort and intention.
Historically, U.S. achievement has sat in the middle of the range among developing countries, well below Asian countries such as South Korea, Japan, Singapore and the city of Shanghai as well as Canada, Finland and the Netherlands. The U.S. results are about average for OECD countries and have led to hand-wringing on causes and consequences. On the plus side, the results often lead to a serious discourse on how educational attainment can be improved. PISA provides insights and detailed data that can contribute to this exchange.
However, there are dangers in believing that PISA scores can be raised by simply adopting the practices of the high-performing countries.
The statistical approach of PISA cannot provide causal links between school differences and student performance, only descriptive information. Why? Because PISA is primarily a source of extensive data collection on education, not a statistical study that isolates cause and effect.
[READ: These Are the Best Countries for Education]
Unfortunately, most of those who read the headlines will conclude that the report has discovered the answers to high academic performance. And some politicians and educators will use PISA to argue for their own favored educational practices.
In the past, for example, politicians who favor school choice and vouchers have used the positive PISA results for the Netherlands to promote vouchers for the U.S. But they fail to acknowledge that the very highest-performing countries such as South Korea and Japan obtain their lofty results by assigning students to schools.
Here are a few other examples of why PISA cannot provide solid evidence for why student achievement varies from country to country:
— PISA does not measure government assistance for poor families that accounts for student success. Most OECD countries provide much greater assistance than the U.S. for family housing, health care, family income and universal early childhood education, all of which improve student achievement.
— School data is collected only on a student’s present school and does not measure the characteristics or results of previous schools attended. Almost all 15-year-olds in Korea and Japan are in the first year of the high school that they are attending when PISA data are obtained, so it is unlikely that the data on these schools can be used to account for their high performance.
— Korea’s exemplary results may not reflect schools as much as outside assistance. Parents spend almost as much sending their children to after-school programs as the government does on schools, a pattern found in Japan, China and much of Asia.
— PISA does not measure the soft-skills acquired in school, such as collaboration and persistence, which have been identified recently as contributing as much to employment and earnings as test scores, nor does it assess the skills that enable one to be effective as citizens — a major goal of schools.
The quality of an educational system can be based partially on its achievement scores, but comprehensive comparisons require a much deeper evaluation of causes and effects as well as performance on broader purposes of education.
Despite the stratospheric results for Chinese students in PISA 2012, for example, students sent by Chinese families to American high schools ballooned 50-fold from 2006 to 2014. Perhaps we should ask the families of these students why the highest test results in the world were not adequate to keep their students in Chinese schools. Their insights may better assist in understanding the strengths and challenges of PISA.
More from U.S. News
Worldwide, Public Education Is Up for Sale
These Are the Best Countries for Education
An International Education for Chinese Students Abroad
PISA 2015 Won’t Provide the Answers to Educational Excellence originally appeared on usnews.com
