#3 Blog from the TWG International Literacy Assessments. See other blogs through this page.

Bill Brozo & Sari Sulkunen What Should We Make of the Overall Narrowing of the Gender Gap in Reading Literacy on PISA 2018?

In previous posts, broad-level findings from PISA 2018 have been presented and analyzed.  In this post, we focus more narrowly on patterns of gender performance in reading literacy. 
For Bill, this has been an area of scholarly interest for over 20 years, culminating in his newest book on the topic (Engaging Boys in Active Literacy: Evidence and Practice, 2019, Cambridge University Press).  For Sari, gender has been a relevant factor in all of her research, including her doctoral work on text authenticity in PISA 2000. See also Sulkunen and Kauppinen, 2018, “I know that at Least Some Boys in my Class Read Something”: Boys’ (Partly Hidden) Literacies].

A perusal of titles of popular press books, reports, and news articles will quickly leave one with the impression that boys are in serious trouble.  With media’s sensationalizing of the issue, and publishers and authors grasping for headlines such as–“Boys Adrift,” “Schools are Failing our Boys,” and “The Boy Crisis: At Every Level of Education They’re Falling Behind, What to Do?”–it is not surprising to find large percentages of the public in both North America and Europe “worried” about the future of boys (EURYDICE, 2010; Whitmire, 2011).  In the European context, the EU High-Level Group of Experts on Literacy (2012) report identified the gender gap in reading in favor of girls as one of the four gaps that needed to be closed.  In North America, numerous reports have appeared over the past two decades drawing attention to the disparities between the reading achievement of boys and girls and warning of the consequences of boys’ continued underachievement in reading.  As recently as 2015, the Brown Center Report on American Education devoted a large section to boys, girls, and reading, concluding with rather a dire speculation that the death of the novel may come when girls and women stop reading.

Nevertheless, behind the hype about a so-called “boy crisis”, there remain some concerning trends that require our attention and analysis.  Now armed with the results from PISA 2018 (OECD, 2019a; 2019b), the most recent cycle and the third since 2000 to emphasize reading literacy, an analysis of the findings reveals a pattern of female superiority in reading evidenced on all previous PISA cycles.  Comparing the 62 countries that participated in both PISA 2009, the previous cycle that emphasized reading literacy, with PISA 2018 reveals the gender gap in reading narrowed in 36 of them, attributable to improved scores for boys in 17 of the countries, while in 11 countries the narrowing gap was attributable to a decline in girls’ scores.  Looking at the gender-based achievement gap in reading for just those OECD countries with comparable data, girls outperformed boys on average by 32 points in 2000 (boys=479/girls=511), 39 points in 2009 (boys=476/girls=515), and 30 points in 2018 (boys=475/girls=505). 

Regarding the gender gap in reading for our two countries, it was 29 score points in favor of girls in 2000 for US students, 25 points in 2009, and 24 points in 2018.  Since the 2009 PISA cycle, both US boys’ and girls’ overall performance has remained stable.  By comparison, Sari’s homeland of Finland continues to experience one of the world’s highest achievement gaps in favor of girls: 51 points in 2000, 55 points in 2009, and 52 points in 2018.  And while the gender gap has been rather stable over this period, overall performance on the part of Finnish 15-year-olds has steadily declined, and so too has the average achievement levels of both boys and girls.

Worth noting, however, is the reading gender gap in favor of 15-year-old girls among the OECD participating countries has narrowed by 9 points since the previous cycle in 2009.  At first glance, this gap reduction would seem to warrant guarded optimism because although still large and significant, it suggests that boys’ reading achievement is improving.  Disappointedly, what this trend actually reflects is a decline in girls’ reading performance and not necessarily an improvement in boys’ performance.  For instance, while boys’ overall average scores on PISA 2000 (479), 2009 (476), and 2018 (475) have degraded negligibly, girls’ average scores have decreased more significantly: 511 in 2000, 515 in 2009, 505 in 2018. 

Furthermore, the drop off in achievement has been particularly steep for low-achieving girls.  Compared with the performance of the lowest-achieving 10% of boys, the lowest-achieving 10% of girls has experienced a steeper decline in overall average performance.  To illustrate, in 2000 this group of boys’ score was 347, in 2009, 349, and in 2018, 336–a decline of 13 points. For these same PISA cycles, a similar group of girls had scores of 391, 400, and 379 respectively for an overall decline of 21 points.

One additional finding of interest related to this issue is that from 2009 to 2018 there was an increase of 7 percentage points for boys but 9 percentage points for girls who agreed that “I read only if I have to”.  This is yet another indication that the closing of the gender gap may be explained in part to the erosion of positive attitudes toward reading by girls.

How Reliable is PISA as an Indicator of the Gender Gap in Reading?

We would be remiss if we did not acknowledge the legitimate challenges researchers have made to generalizations about boys’ underachievement relative to girls on PISA reading.  For example, it has been asserted that the reading item formats may be favoring girls.  About half of the reading questions are open-response type, requiring answers of either short written explanations or longer constructed responses.  Empirical studies have determined that boys perform significantly less well than girls on these open-ended items, although boys appear to be equally competent as girls on the closed-ended response items (e.g., multiple-choice) (Rauch & Hartig, 2010; Oddny & Lundetrae, 2016; Scwabe, McElvany, & Trentel, 2015).  This has led to speculation about whether the gender-based reading achievement gap is an artifact of item formatting on PISA rather than a genuine reflection of inherent differences in reading ability between boys and girls (Brozo et al., 2014).  An extension of this logic suggests the potential diminution or outright disappearance of a gender gap on PISA if there were fewer or no constructed response items.  We urge caution, however, when it comes to this interpretation since questions requiring short and longer constructed-responses are designed to tap higher-level reading processes, which all of us would agree are as important to boys as to girls.

A related critique is that boys would likely demonstrate reading achievement levels comparable to girls on assessments like PISA if they were equally engaged as girls.  There are data that seem to support this contention. Generally speaking, girls tend to have greater reading motivation than boys (Marinak & Gambrell, 2010; McGeown, Goodwin, Henderson, & Wright, 2012).  Additionally, close analysis of previous PISA results showed that reading engagement was the variable with one of the strongest links to reading performance for all the countries participating, and girls had significantly higher indices of reading engagement than boys (Brozo et al., 2014).  It is also noteworthy that, on average, girls invested more effort in completing the PISA 2018 test than boys.  This held true for nearly all OECD countries.  Moreover, both boys and girls deemed the PISA test low-stakes, with both groups reporting they would invest more effort in the assessment if it affected their school marks (OECD, 2019a).

With respect to our two countries, surveys in the US reveal that girls have more positive attitudes toward reading (McKenna et al., 2012) and are reading for pleasure daily at a percentage that is nearly twice as high as that of boys (Common Sense Media, 2014; Rideout, Foehr, & Roberts, 2010).  And in Finland, although the attitudes of Finnish students to reading have become more negative, PISA 2018 found a substantial percentage of Finnish boys (63%) reporting they read only if they have to.

PISA 2018 findings also reinforce the trend toward a decline in leisure reading among adolescents; in fact, compared with the 2009 assessment, a growing percentage of youth consider reading a “waste of time.”  Only 24% of 15-year-old boys on the 2018 assessment reported that reading was one of their favorite hobbies as compared with 44% of girls.  Furthermore, a whopping 60% of boys indicated they only read to obtain needed information, whereas 39% of girls offered a similar response.  Even more revealing is that 75% of boys reported spending no time or less than 30 minutes per day reading for pleasure; whereas 43% of girls said they spent at least 30 minutes per day in pleasure reading.

Continuing to look on the testing front, challengers to the assertion that boys are less competent readers than girls point out that males eventually catch up with and may even surpass their female counterparts as they enter and progress through adulthood.  Support comes from the Program for International Assessment of Adult Competencies or PIAAC, another OECD-sponsored international survey completed in 2012 by individuals age 16 to 65 in more than 30 countries across the globe.  Overall test results revealed that men and women were statistically indistinguishable up to age 35, including in our own countries, Finland and the United States.  After age 35, men’s higher scores in reading, up to the oldest group–age 55 and beyond–were statistically significant. 

How Can PISA Results Related to Gender Performance be Used?

After 20 years and the results of seven PISA cycles, a clear pattern emerges that boys from across all demographic groups are at a disadvantage when it comes to reading.  Taking a closer look at the results of the 2018 assessment, however, suggests that although girls continue to demonstrate superior reading achievement, their advantage over boys may be narrowing.  As we demonstrated, this narrowing appears to be due to a decline in female’s achievement, particularly for the lowest-achieving girls.  Furthermore, a greater erosion of reading engagement among girls as compared with boys may also be contributing to the closing gender gap. 

We acknowledge that certain boys, particularly those of color, those in poverty, and immigrant boys, continue to demand extra supports to meet their literacy and life needs.  However, it may be the case that increased attention also needs to be directed at girls from these same demographic groups.  In addition, efforts to keep both boys and girls engaged in reading would seem warranted based on the most recent assessment results.  PISA 2018 may be warning us of a pattern related to the gender gap in reading we might expect to see reinforced in upcoming cycles, suggesting the gap will continue to shrink as a result of declining achievement and motivation by girls. (Since OECD has decided to delay by one year the conduct of the originally scheduled 2021 PISA cycle due to the COVID-19 pandemic, we will have to wait until at least 2023 for the next assessment results to determine if the decline in girls’ reading achievement and engagement highlighted here becomes a more serious trend.)

With disadvantage in reading, regardless of gender, comes other disadvantages in school and life.  Responsive literacy policies and practices targeting underachieving and disengaged male and female students have the potential to confer on them the numerous academic, personal, social, and economic benefits of an active literate life (Marchant, McCreary, & Terwillager, 2018).  In collaboration with other ELINET colleagues, we identified innovative approaches for responding to boys’ reading needs being employed in countries across Europe, Asia, and the United States (Brozo et al., 2014).  Similar energy around creative instruction and programs is needed to ensure struggling and unmotivated readers of both genders receive the support they deserve.

References

Brozo, W.G., Sulkunen, S., Shiel, G., Garbe, C., Pandian, A., & Valtin, R. (2014). Reading,

gender, and engagement: Lessons from five PISA countries. Journal of Adolescent & Adult Literacy, 57(7), 584-593.

Common Sense Media. (2014). Children, teens, and reading: A Common Sense Media research

brief. Retrieved from file:///C:/Users/wbrozo/AppData/Local/Temp/csm-childrenteensandreading-2014_0.pdf

EU High Level Group of Experts on Literacy. (2012). Final report. Retrieved from

http://ec.europa.eu/dgs/education_culture/repository/education/policy/school/doc/literacy-report_en.pdf

EURIDYCE. (2010). Gender differences in educational outcomes: Studies on the measures

taken and current situation in Europe. Brussels: Education, Audiovisual and Culture 

Executive Agency. Retrieved from http://eacea.ec.europa.eu/education/eurydice/documents/thematic_reports/120en.pdf

Marchant, G.J., McCreary, J.J., & Terwillegar, W.M. (2018). The national and international

reading dilemma for boys: A call to action. Paper presented at the annual meeting of the American Educational Research Association, New York.

Marinak, B.A., & Gambrell, L.B. (2010). Reading motivation: Exploring the elementary gender

gap. Literacy Research and Instruction, 49(2), 129-141.

McGeown, S., Goodwin, H., Henderson, N., & Wright, P. (2012). Gender differences in reading

motivation: Does sex or gender identity provide a better account? Journal of Research in Reading, 35(3), 328-336.

McKenna, M., Conradi, K., Lawrence, C., Jang, B.G., & Meyer, J.P. (2012). Reading attitudes of

middle school students: Results of a U.S. survey. Reading Research Quarterly, 47(3),

283-306.

Oddny, J.S., & Lundetræ, K. (2016). Can test construction account for varying gender

differences in international reading achievement tests of children, adolescents and young adults? – A study based on Nordic results in PIRLS, PISA and PIAAC. Assessment in Education: Principles, Policy & Practice. Retrieved from http://www.tandfonline.com/doi/full/10.1080/0969594X.2016.1239612

OECD. (2019a). PISA 2018 results (volume I): What students know and can do. Paris: OECD

Publishing. Retrieved from https://www.oecd-ilibrary.org/docserver/5f07c754-en.pdf?expires=1595887640&id=id&accname=guest&checksum=0F1D99B12ED389BD962DD01E2F03C194

OECD. (2019b). PISA 2018 results (volume II): Where all students can succeed. Paris: OECD

Publishing. Retrieved from https://www.oecd-ilibrary.org/docserver/b5fd1b8f-en.pdf?expires=1596052766&id=id&accname=guest&checksum=F616D3C39C4148383970571B5196A541

Rausch, D.P., & Hartig, J. (2010). Multiple-choice versus open-ended response formats of

reading test items: A two-dimensional IRT analysis. Psychological Test and Assessment Modeling, 52(4), 354-379.

Rideout, V.J., Foehr, U.G., & Roberts, D.F. (2010). Generation M2: Media in the lives of 8- to

18-year-olds. Menlo Park, CA: Kaiser Family Foundation.

Scwabe, F., McElvany, N., & Trentel, M. (2015). The school age gender gap in reading

achievement: Examining the influences of item format and intrinsic reading motivation. Reading Research Quarterly, 50(2), 219-232. 

Sulkunen, S., & Kauppinen, M. (2018). “Ainakin muutaman pojan luokallani tiedän lukevan”:

poikien (osin piiloiset) luku- ja kirjoitustaidot. In English (freely translated): “I know that at least some boys in my class read something”: Boys’ (partly hidden) literacies. In A. Kivijärvi, T. Huuki, & H. Lunabba (Eds.) Poikatutkimus (Research on boys) (pp. 146−173).

Whitmire, R. (2011). Why boys fail: Saving our sons from an educational system that’s

leaving them behind. New York: AMACOM.