TIMSS, Reading, and Disciplinary Literacy

Bill Brozo 23/02/2021

With the recent release of the results of the 2019 cycle of Trends in Mathematics and Science Study (TIMSS), it is important to consider the extent to which literacy skills and abilities are essential for success on this assessment. 

TIMSS is an international assessment of mathematics and science at the fourth and eighth grades conducted every four years since 1995.  Like other cognitive-based assessments that require language and reasoning skills, TIMSS science and math items often challenge students in ways that place a premium on their reading and thinking abilities specific to these disciplines, as well as their relevant prior knowledge acquired through reading in these domains.

For years I have emphasized to teachers across the US and internationally that students who are better readers are higher achievers in all subjects.  As a content/disciplinary literacy researcher this maxim always made complete sense to me.  And there seemed to be plenty of evidence to support such a claim.  I recall when working as a consultant for Louisiana in the US, I directed school and district curriculum specialists to run correlations between students’ reading scores from the state assessment and their overall grade point averages—a measure of subject area achievement at the school level in the States.  Invariably, the better readers were better students in science, history, math, etc.  These were vital data to buttress my case that all teachers regardless of their disciplines should work to leaven the literacy abilities of their students.

            We find on a global level the same relationship between achievement in the various school subjects and overall reading ability holds true, as well.  For instance, a strong association can be found between performance on PIRLS (Progress in International Reading Literacy Study) and TIMSS (Martin & Mullis, 2013) and among the three PISA domains—reading, science, and math literacy (OECD, 2019).

Science and Reading

Regarding the relationship between reading literacy and science literacy on PISA, Cromley (2009) determined that reading skill was a much stronger predictor of science achievement than an interest in and enjoyment of science.  Cromley proposed that extensive reading practice leading to elaborate background knowledge, refined reading strategies, and expansive general vocabulary is the driver of higher science proficiency.

            American researchers O’Reilly and McNamara (2007) confirmed this close relationship between reading and science achievement in a large-scale study of secondary school students in the US.  They compared the performance of more than 1,600 students in grades 9-12 on multiple measures of science knowledge and reading skill.  Although both reading skill and science knowledge were significant predictors of science achievement, it was found that reading skill had a larger effect than science knowledge on most of the measures of science achievement.  The authors interpreted these results as indicating that reading skill helps students compensate for gaps in science knowledge, since students in their study with higher reading skills but lower science knowledge scored as high or higher than students with lower reading skills but higher science knowledge.  Another critical insight from this study is that although reading skill was a key factor in higher science scores for students with lower science knowledge, it had an even greater impact on the performance of students with higher science knowledge.  Given these results, it is not surprising that O’Reilly and McNamara came to the conclusion that students should be encouraged and provided numerous opportunities to read science-related books and articles in order to build domain-specific reading skills and enrich their science knowledge.

In Martin and Mullis’ (2013) study, the researchers examined the relationship between PIRLS and TIMSS using data from 34 countries for 185,475 fourth-grade students who had participated in both assessments.  TIMSS items were categorized according to the level of reading demands they placed on students: low, medium, and high.  Students were also categorized based on their PIRLS scores into an upper, middle, or lower tercile.  As expected, students across all participating countries in the upper tercile of reading achievement had the highest scores on the TIMSS science items as compared with students in the middle and lower terciles. And the opposite was found to be true–the lowest science scores came from students in the lower reading tercile.  This pattern held true regardless of the reading difficulty level of the science items, suggesting the relationship between reading and science achievement goes beyond the difficulty of any particular science question and reinforces the contention that students should be provided wide and numerous opportunities to read and reason through science-related texts to build literacy and thinking skills integral to overall science achievement.

            It’s noteworthy that the significant reading demands TIMSS science items place on students may account for 8^thgrade girls’ significantly higher achievement relative to boys on this portion of the assessment (international average: 495 for girls; 485 for boys), since girls consistently demonstrate higher reading literacy scores on PIRLS (Mullis et al., 2020), PISA, and most other measures of verbal ability (Brozo, 2019).

Math and Reading

Like science, reading skill appears to play an important role in mathematics achievement, particularly for certain math-related tasks (Brozo & Crain, 2018; Rupley, 2006).  As was shown by Martin and Mullis (2013), general reading ability based on PIRLS scores tracked closely to achievement on TIMSS math.  The top third of the 4th-grade readers in their study answered more TIMSS math questions correctly than the middle third, which in turn had a higher percent correct than those in the lowest third. This pattern held across all three levels of reading difficulty in all 34 countries.  Especially noteworthy among their findings was that the more reading required on TIMSS math questions the greater disadvantage students from the lowest tercile were placed.  Simply stated, the best readers were the highest achievers on the math portion of the TIMSS assessment.

Since it has been verified that cognitive growth can be developed in and through both reading and mathematics, it is not surprising that educators are recognizing the importance of reading comprehension skills to students’ mathematical success (Capraro, Caproro, & Rupley, 2012).  Others, like Thurber, Shinn, and Smolkowski (2002), argue that reading is a critical factor in overall math competence and should not be overlooked in instructional regimens to develop mathematics skills.  Thurber and his colleagues came to this conclusion based on their correlational analysis of 4^th graders’ math and reading performance on various measures of these skills, finding that the higher students’ reading achievement, the higher their math achievement; and vice-versa.

A longitudinal study that followed students from the 4th to the 7th grade revealed how reading and math achievement were related over this four-year timeframe (Shin et al., 2013).  As expected, reading and math were highly correlated at 4th grade.  However, the researchers also found that 4th graders with higher reading achievement experienced higher math growth rates in each successive grade.  These results led the authors to conclude that “a solid foundation in reading may facilitate gains in more than just reading and would directly affect the growth of mathematics” (p. 100).

Again, for those of us interested in patterns of gender performance on international assessments, 8th grade girls demonstrated equal levels of achievement on TIMSS math with an overall international average of 491 versus 488 for boys.  Surely, as the evidence argues here, this result is closely related to girls’ well-documented overall superior reading ability (Brozo, 2019).

Science, Math, and Disciplinary Literacy

As I pointed out in the opening of this post, my decades-long admonishment to teachers that doing all they can to improve reading abilities of their students is likely to also produce gains in subject-area achievement is only reinforced by the findings and sources cited here.  How best to accomplish this goal presents challenges, but the TIMSS/PIRLS correlational data alone, apart from plenty of other supportive evidence, strongly suggest elevating general reading skill may be sufficient to raise science and math achievement.  But what if teachers focused reading skill development on the actual knowledge, application, and reasoning processes specific to science and math?  Would achievement in all three domains (reading, science, math) expand even more?

            I’m referring, of course, to disciplinary literacy, and I contend that teaching students the ways of reading, writing, and thinking inherent in doing actual science and math will have a more direct impact on achievement in those domains.  Consider that TIMSS assessment tasks involve forms of reasoning such as using evidence and science understanding to support the reasonableness of explanations, solutions to problems, and conclusions from investigations and making valid inferences on the basis of information and evidence.  This type of mental work is consistent with the disciplinary practices of scientists and mathematicians and requires specialized ways of reading and writing students can be taught to emulate and apply (Brock et al., 2014; Brozo, 2017). 

References

Brock, C. H., Goatley, V.J., Raphael, T.E., Trost-Shahata, E., & Weber. C.M. (2014). Engaging students in disciplinary literacy, K6: Reading, writing, and teaching tools for the classroom. Teachers College Press.

Brozo, W.G. (2019). Engaging boys in active literacy: Evidence and practice. Cambridge University Press.

Brozo, W.G. (2017). Disciplinary and content literacy for today’s adolescents. Pearson.

Brozo, W.G. & Crain, S. (2018): Writing in math: A disciplinary literacy approach. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 91(1), 7-13.

Capraro, R. M., Capraro, M. M., & Rupley, W. H. (2012). Reading-enhanced word problem solving: A theoretical model. European Journal of Psychology of Education, 27(1),91–114.

Cromley, J. G. (2009). Reading achievement and science proficiency: International comparisons from the Programme on International Student Assessment. Reading Psychology, 30(89), 89-118.

Martin, M. O., & Mullis, I. V. S. (Eds.). (2013). TIMSS and PIRLS 2011: Relationships among reading, mathematics, and science achievement at the fourth grade—implications for early learning. Boston College.

Mullis, I. V. S., Martin, M. O., Foy, P., Kelly, D. L., & Fishbein, B. (2020). TIMSS 2019 international results in mathematics and science. Retrieved from https://timssandpirls.bc.edu/timss2019/international-results/

OECD (2019). Comparing reading, mathematics, and science performance across PISA cycles. In PISA 2018 results (Volume I): What students know and can do. OECD Publishing.

O’Reilly, T., & McNamara, D. S. (2007). The impact of science knowledge, reading skill, and reading strategy knowledge on more traditional “high-stakes” measures of high school students’ science achievement. American Educational Research Journal, 44(1), 161-196.

Rupley, W. H.  (2006).  Reading and mathematics: Introduction.  Reading Psychology: An International Journal, Special Issue on Reading and Mathematics, 27(2-3), 87-89.

Shin, T., Davison, M. L., Long, J. D., Chan, C. K., & Heistad, D. (2013). Exploring gains in reading and mathematics achievement among regular and exceptional students using growth curve modeling. Learning and Individual Differences, 23(1), 92-100.

Thurber, R. S., Shinn, M. R., & Smolkowski, K. (2002). What is measured in mathematics tests? Construct validity of curriculum-based mathematics measures. School Psychology Review, 31(4), 498-513.