How Multimodal Texts can Strengthen Literacy Learning: Pathways, Conditions, and Classroom Implications

Literacy is no longer shaped by print alone. Young people read and write in environments saturated with images, audio, video, hyperlinks, emoji, and platform conventions. The 'multiliteracies' tradition argues that literacy pedagogy must respond to both expanding communication channels and increasing linguistic and cultural diversity (New London Group, 1996). Multimodality offers a way to describe how meaning is designed across modes, not just encoded in alphabetic text (Kress, 2010). In schools, this matters because many curriculum tasks still assume that comprehension and knowledge are best demonstrated through print-only reading and writing, even when students’ everyday literacies are richer and more varied.

This article asks: How do multimodal texts and multimodal reading practices improve literacy, and under what conditions? It provides a structured synthesis of research across second language and mainstream settings, with a focus on mechanisms and how change happens.

 

Conceptual Foundations

Multimodality is not simply 'adding pictures'. Modes shape what is easy or hard to express. Image can condense setting and emotion; layout can guide attention; audio can carry tone and pace; gesture can clarify reference and stance. A key implication is that literacy teaching must address how modes work together, and how students choose modes for a purpose (Jewitt, 2008; Kress, 2010).

In second language (L2) and multilingual classrooms, multimodal pedagogy is also framed as a response to reduced specialist language support and increasingly heterogeneous classrooms, positioning all teachers as teachers of language and meaning-making (Lotherington and Jenson, 2011). That framing shifts the goal from 'simplifying texts' to 'expanding resources for making meaning'.

 

Multiple Pathways for Expression and Engagement

A consistent finding is that multimodal tasks can widen how students show understanding. Lotherington and Jenson (2011) argue that digital and multimodal literacy teaching opens space for meaning-making through design and performance, not only through conventional reading and writing.

Design-oriented approaches emphasise that students can learn to read multimodally (interpreting how modes combine) and compose multimodally (choosing modes deliberately). Lim and Tan-Chia’s practitioner-facing design work presents multimodal literacy as teachable: students can be taught to 'view' and 'represent' with structured routines rather than being left to 'be creative' without guidance (Lim and Tan-Chia, 2023). 

 

Support for Emergent Multilingual and Culturally Diverse Learners

The evidence base is particularly strong for emergent bilingual learners in multimodal composing research. In a systematic review of secondary classroom studies, Smith, Pacheco and Khorosheva (2021) found that digital multimodal composition frequently supports identity expression, challenges narrow language ideologies, and enables learners to use multiple semiotic resources to communicate powerfully.

Zaidi and Sah’s scoping review similarly concludes that multilingual and multimodal literacy engagements can support immigrant students’ exploration of intersectional identities and experiences of inequality, highlighting how these practices can serve both learning and inclusion aims (Zaidi and Sah, 2024).

 

Motivation, Creativity, and Student Agency

Research commonly reports improved engagement when students design digital multimodal texts such as comics, videos, or interactive narratives. Dahlström’s study of students as digital multimodal text designers highlights how learners draw on out-of-school experiences and resources, and how that can increase participation and success in school text-making when tasks legitimise those resources (Dahlström, 2022).

Work on digital storytelling in EFL contexts also reports positive effects on learners’ confidence and engagement. Yoon’s doctoral study positions digital storytelling as a route to develop multimodal digital literacy while sustaining motivation in English learning (Yoon, 2014).

 

Bridging home literacies and school expectations

A major equity argument for multimodality is that it can connect classroom literacy to home and community practices, where meaning is often shared through talk, gesture, images, and bilingual resources. Chung’s case study of a Korean immigrant family shows how a parent used multimodal cues (including gesture and audio language) during shared reading to support vocabulary inference, attention, and confidence (Chung, 2024). 

Ramos Lopez’s doctoral case study of Latino students’ multimodal writing in English and Spanish also frames digital literacies as a route to biliteracy development, emphasising how students draw on full linguistic repertoires when composing multimodally (Ramos Lopez, 2019). 

 

Cognitive and academic benefits

Claims about 'better comprehension' need careful reading. One of the clearer outcome studies examined a scaffolded digital reading environment (ICON) that embedded strategy instruction and interactive vocabulary supports. Dalton et al. found significant vocabulary advantages for conditions that included interactive vocabulary supports, while comprehension effects were more limited and depended on text type and learner group (Dalton et al., 2011).

Other research offers analytic frameworks showing how multimodality can deepen narrative understanding by making structure and authorial choices more visible. Kim, Yatsu and Li propose a framework for analysing middle school English learners’ digital stories across representational, interpersonal, compositional, and sociocultural dimensions, illustrating how 'remixing' multimodal resources can support voice and identity work alongside meaning-making (Kim, Yatsu and Li, 2021).

 

Evidence-based Pedagogical Models and the Role of Scaffolding

Two classroom implications recur across the literature.

First, teachers need to teach the 'how' of multimodal composing. In a study of pre-service teachers working with English language learners, Tour and Barnes found that participants valued digital multimodal composing but often struggled to name specific literacy benefits and to attend to multiple dimensions of composing beyond interface engagement (Tour and Barnes, 2022).

Second, scaffolding matters and it is multifaceted. Pacheco et al. identify several scaffold types (for example, collaboration, direct instruction, exemplar texts, translanguaging, structured discussion, encouragement, and questioning), and show that scaffolds function to support identities, resources, and contexts, not just 'skills' (Pacheco et al., 2021).

Pragmatically, this means multimodal literacy teaching benefits from routines such as:

• explicit modelling of how modes work together (not just tool tutorials)
• exemplars with guided critique ('What is the image doing that the words are not?')
• structured collaboration and feedback protocols
• assessment criteria that reward meaning-making, not decoration.

 

Conclusion

Multimodal texts can strengthen literacy because they expand the tools students use to interpret and communicate meaning. The strongest evidence is for equity and participation benefits for emergent bilingual and culturally diverse learners, and for targeted outcomes such as vocabulary learning when digital environments are intentionally scaffolded. At the same time, multimodality is not a shortcut. Without clear literacy goals, explicit instruction in multimodal design, and assessment practices that value reasoning and interpretation, multimodal work can become busy and shallow. The research points to a straightforward lesson: multimodal literacy improves literacy when it is taught as literacy - deliberate, critical, and purposeful.

 

References

• Chung, S. (2024) ‘A Parent’s Multimodal Support to Bilingual Child’s Reading: A Case Study of Korean Immigrant Family’, Journal of Language, Identity and Education. 
• Dahlström, H. (2022) ‘Students as digital multimodal text designers: A study of resources, affordances, and experiences’, British Journal of Educational Technology, 53(2), pp. 391–407. 
• Dalton, B., Proctor, C.P., Uccelli, P., Mo, E. and Snow, C.E. (2011) ‘Designing for Diversity: The Role of Reading Strategies and Interactive Vocabulary in a Digital Reading Environment for Fifth-Grade Monolingual English and Bilingual Students’, Journal of Literacy Research, 43(1), pp. 68–100.
• Jewitt, C. (2008) ‘Multimodality and Literacy in School Classrooms’, Review of Research in Education, 32(1), pp. 241–267.
• Kim, D., Yatsu, D.K. and Li, Y. (2021) ‘A multimodal model for analyzing middle school English language learners’ digital stories’, International Journal of Educational Research Open, 2, 100067.
• Kress, G. (2010) Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.
• Lim, F.V. and Tan-Chia, L. (2023) Designing Learning for Multimodal Literacy: Teaching Viewing and Representing. 1st edn. London: Routledge.
• Lotherington, H. and Jenson, J. (2011) ‘Teaching Multimodal and Digital Literacy in L2 Settings: New Literacies, New Basics, New Pedagogies’, Annual Review of Applied Linguistics, 31, pp. 226–246. 
• New London Group (1996) ‘A Pedagogy of Multiliteracies: Designing Social Futures’, Harvard Educational Review, 66(1), pp. 60–92. 
• Pacheco, M.B., Smith, B.E., Deig, A. and Amgott, N.A. (2021) ‘Scaffolding Multimodal Composition with Emergent Bilingual Students’, Journal of Literacy Research, 53(2), pp. 149–173.
• Ramos Lopez, L.B. (2019) Digital Literacies to Develop Biliteracy: A Case Study of Latino Students Exploring Multimodal Writing in English and Spanish.
• Smith, B.E., Pacheco, M.B. and Khorosheva, M. (2021) ‘Emergent Bilingual Students and Digital Multimodal Composition: A Systematic Review of Research in Secondary Classrooms’, Reading Research Quarterly, 56(1), pp. 33–52.
• Tour, E. and Barnes, M. (2022) ‘Engaging English Language Learners in Digital Multimodal Composing: Pre-Service Teachers’ Perspectives and Experiences’, Language and Education, 36(3), pp. 243–258.
• Yoon, T. (2014) Developing Multimodal Digital Literacy: The Application of Digital Storytelling as a New Avenue for Effective English Learning with EFL Elementary School Students in Korea.
• Zaidi, R. and Sah, P.K. (2024) ‘Affordances of multilingual and multimodal literacy engagements of immigrant high school students: A scoping review’, SAGE Open, 14(1), pp. 1–17.