The impact of orthography versus images on foreign language learning: Evidence from behavioral and neural markers

• Cieśla, Mathew ^[1] ; Kapnoula, Efthymia C. ^[2] ; Pozdniakov, Maksym ^[3] ; Gruszecka, Justyna ^[3] ; Jankowiak, Katarzyna ^[3]
  1. [1] Northumbria University

     Northumbria University

     Reino Unido

     
  2. [2] Basque Center on Cognition, Brain and Language

     Basque Center on Cognition, Brain and Language

     San Sebastián, España

     
  3. [3] Adam Mickiewicz University in Poznań

     Adam Mickiewicz University in Poznań

     Poznań, Polonia

     

  Mostrar afiliaciones +
• Localización: Bilingualism: Language and cognition, ISSN 1366-7289, Vol. 29, Nº 3, 2026, págs. 563-575
• Idioma: inglés
• DOI: 10.1017/s1366728925100242
• Texto completo no disponible (Saber más ...)
• Resumen

  • A central question in foreign language (LX) learning is how vocabulary acquisition is affected by using image versus orthographic referents. According to the picture superiority effect (PSE) and bilingual/dual coding theory (b/DCT), images should lead to better novel word encoding and retrieval. We tested this prediction using behavioral and event-related potential (ERP) measures. Thirty Polish native speakers learned 40 LX (artificial language) words using either image or L1/orthographic referents. After 24 hours, participants were tested using a translational priming paradigm in congruent and incongruent training-testing modalities. Behavioral results showed higher accuracy and faster responses for LX words learned and tested with images, in line with the PSE and b/DCT. ERP results revealed smaller Late Positive Complex (LPC) amplitudes for words preceded by image compared to lexical primes, likely reflecting less cognitively demanding lexical retrieval. These results provide converging evidence that visual referents provide a more salient modality for L2 learning.

• Referencias bibliográficas
  • Altarriba, J., & Knickerbocker, H. (2011). Acquiring second language vocabulary through the use of images and words. In McDonough, K.,...
  • Alvarez, R. P., Holcomb, P. J., & Grainger, J. (2003). Accessing word meaning in two languages: An event-related brain potential study...
  • Alzahrani, A. (2025). The acceptability and validity of AI-generated psycholinguistic stimuli. Heliyon, 11(2), e42083. https://doi.org/10.1016/j.heliyon.2025.e42083.
  • Audacity Team. (2022). Audacity(R): Free audio editor and recorder (3.3.3). Audacity Team. https://audacityteam.org
  • Aurnhammer, C., Delogu, F., Brouwer, H., & Crocker, M. W. (2023). The P600 as a continuous index of integration effort. Psychophysiology,...
  • Baese-Berk, M. M., Kapnoula, E. C., & Samuel, A. G. (2025). The relationship of speech perception and speech production: It’s complicated....
  • Baggio, G., & Hagoort, P. (2011). The balance between memory and unification in semantics: A dynamic account of the N400. Language and...
  • Bakker, I., Takashima, A., Van Hell, J. G., Janzen, G., & McQueen, J. M. (2014). Competition from unseen or unheard novel words: Lexical...
  • Bakker, I., Takashima, A., Van Hell, J. G., Janzen, G., & McQueen, J. M. (2015). Tracking lexical consolidation with ERPs: Lexical and...
  • Bartolotti, J., & Marian, V. (2017). Bilinguals’ existing languages benefit vocabulary learning in a third language. Language Learning,...
  • Batterink, L., & Neville, H. (2011). Implicit and explicit mechanisms of word learning in a narrative context: An event-related potential...
  • Berger, A., & Kiefer, M. (2021). Comparison of different response time outlier exclusion methods: A simulation study. Frontiers in Psychology,...
  • Borovsky, A., Elman, J. L., & Fernald, A. (2012). Knowing a lot for one’s age: Vocabulary skill and not age is associated with anticipatory...
  • Brundage, T., & Barile-Spears, A. L. (2015). Effects of stimulus mode and mode congruency on degraded image identification for the picture...
  • Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal...
  • Culbertson, J., & Schuler, K. (2019). Artificial language learning in children. Annual Review of Linguistics, 5(1), 353–373. https://doi.org/10.1146/annurev-linguistics-011718-012329.
  • Davis, M. H., & Gaskell, M. G. (2009). A complementary systems account of word learning: Neural and behavioural evidence. Philosophical...
  • Defeyter, M. A., Russo, R., & McPartlin, P. L. (2009). The picture superiority effect in recognition memory: A developmental study using...
  • Degani, T., Prior, A., & Hajajra, W. (2018). Cross-language semantic influences in different script bilinguals. Bilingualism: Language...
  • Dumay, N., & Gaskell, M. G. (2007). Sleep-associated changes in the mental representation of spoken words. Psychological Science, 18(1),...
  • Duñabeitia, J. A., Crepaldi, D., Meyer, A. S., New, B., Pliatsikas, C., Smolka, E., & Brysbaert, M. (2018). MultiPic: A standardized set...
  • Emirmustafaoğlu, A., & Gökmen, D. U. (2015). The effects of picture vs. translation mediated instruction on L2 vocabulary learning. Procedia...
  • Ettlinger, M., Morgan‐Short, K., Faretta‐Stutenberg, M., & Wong, P. C. M. (2016). The Relationship Between Artificial and Second Language...
  • Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*power 3.1: Tests for correlation and regression...
  • Ferman, S., Olshtain, E., Schechtman, E., & Karni, A. (2009). The acquisition of a linguistic skill by adults: Procedural and declarative...
  • Francis, W. S., Augustini, B. K., & Sáenz, S. P. (2003). Repetition priming in picture naming and translation depends on shared processes...
  • García-Gámez, A. B., & Macizo, P. (2022). Lexical and semantic training to acquire words in a foreign language: An electrophysiological...
  • Golubović, J., & Gooskens, C. (2015). Mutual intelligibility between west and south Slavic languages. Russian Linguistics, 39, 351–373....
  • Goodall, G. (2023). Constructed languages. Annual Review of Linguistics, 9(1), 419–437. https://doi.org/10.1146/annurev-linguistics-030421-064707.
  • Gratton, G., Coles, M. G. H., & Donchin, E. (1983). A new method for off-line removal of ocular artifact. Electroencephalography and Clinical...
  • Havas, V., Taylor, J., Vaquero, L., De Diego-Balaguer, R., Rodríguez-Fornells, A., & Davis, M. H. (2018). Semantic and phonological schema...
  • Hayakawa, S., Ning, S., & Marian, V. (2020). From Klingon to Colbertian: Using artificial languages to study word learning. Bilingualism:...
  • Hockley, W. E., & Bancroft, T. (2011). Extensions of the picture superiority effect in associative recognition. Canadian Journal of Experimental...
  • Imbir, K. K. (2015). Affective norms for 1,586 polish words (ANPW): Duality-of-mind approach. Behavior Research Methods, 47(3), 860–870. https://doi.org/10.3758/s13428-014-0509-4.
  • Imbir, K. K. (2021). Corrigendum: Affective norms for 4900 polish words reload (ANPW_R): Assessments for valence, arousal, dominance, origin,...
  • Isphording, I. E., & Otten, S. (2014). Linguistic barriers in the destination language acquisition of immigrants. Journal of Economic...
  • Jankowiak, K. (2021). Current trends in electrophysiological research on bilingual language processing. Language and Linguistics Compass,...
  • Jankowiak, K., & Rataj, K. (2017). The N400 as a window into lexico-semantic processing in bilingualism. Poznan Studies in Contemporary...
  • JASP Team. (2023). JASP (0.17.2.1). https://jasp-stats.org/
  • Juola, P. (2018). Authorship attribution, constructed languages, and the psycholinguistics of individual variation. Digital Scholarship in...
  • Kapnoula, E. C., & McMurray, B. (2016). Newly learned word forms are abstract and integrated immediately after acquisition. Psychonomic...
  • Kapnoula, E. C., Packard, S., Gupta, P., & McMurray, B. (2015). Immediate lexical integration of novel word forms. Cognition, 134, 85–99....
  • Kolk, H., & Chwilla, D. (2007). Late positivities in unusual situations. Brain and Language, 100(3), 257–261. https://doi.org/10.1016/j.bandl.2006.07.006.
  • Krepel, A., De Bree, E. H., & De Jong, P. F. (2021). Does the availability of orthography support L2 word learning? Reading and Writing,...
  • Kroll, J., & Stewart, E. (1994). Category interference in translation and picture naming: Evidence for asymmetric connections between...
  • Kroll, J., Van Hell, J., Tokowicz, N., & Green, D. (2010). The revised hierarchical model: A critical review and assessment. Bilingualism:...
  • Kroll, J. F., Michael, E., Tokowicz, N., & Dufour, R. (2002). The development of lexical fluency in a second language. Second Language...
  • Kutas, M., & Federmeier, K. D. (2000). Electrophysiology reveals semantic memory use in language comprehension. Trends in Cognitive Sciences,...
  • Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential...
  • Lang, P.J. (1980). Behavioral treatment and bio-behavioral assessment: Computer applications. In Sidowski, J. B., Johnson, J.H., & Williams,...
  • Leach, L., & Samuel, A. (2007). Lexical configuration and lexical engagement: When adults learn new words. Cognitive Psychology, 55(4),...
  • Li, P., Zhang, F., Yu, A., & Zhao, X. (2020). Language history questionnaire (LHQ3): An enhanced tool for assessing multilingual experience....
  • Lindgren, E., & Muñoz, C. (2013). The influence of exposure, parents, and linguistic distance on young European learners’ foreign language...
  • Liu, X., Horinouchi, H., Yang, Y., Yan, Y., Ando, M., Obinna, U. J., Namba, S., & Kambara, T. (2021). Pictorial referents facilitate recognition...
  • Luck, S. J. (2014). An introduction to the event-related potential technique (2nd ed.) The MIT Press.
  • Madlener-Charpentier, K. (2018). Do findings from artificial language learning generalize to second language classrooms? In Tyler, A. E.,...
  • Malik-Moraleda, S., Taliaferro, M., Shannon, S., Jhingan, N., Swords, S., Peterson, D. J., Frommer, P., Okrand, M., Sams, J., Cardwell, R.,...
  • Mandera, P., Keuleers, E., Wodniecka, Z., & Brysbaert, M. (2015). Subtlex-pl: Subtitle-based word frequency estimates for polish. Behavior...
  • McClelland, J. L., McNaughton, B. L., & O’Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex:...
  • McLaughlin, B. (1980). On the use of miniature artificial languages in second-language research. Applied PsychoLinguistics, 1(4), 357–369....
  • McLaughlin, J., Osterhout, L., & Kim, A. (2004). Neural correlates of second-language word learning: Minimal instruction produces rapid...
  • McMurray, B., Kapnoula, E. C., & Gareth Gaskell, M. (2016). Learning and integration of new word-forms: Consolidation, pruning, and the...
  • Mestres-Missé, A., Rodriguez-Fornells, A., & Münte, T. F. (2007). Watching the brain during meaning acquisition. Cerebral Cortex, 17(8),...
  • Mohamed, A. A. (2018). Exposure frequency in L2 reading: An eye-movement perspective of incidental vocabulary learning. Studies in Second...
  • Morett, L. M. (2019). The power of an image: Images, not glosses, enhance learning of concrete L2 words in beginning learners. Journal of...
  • Nunez, P. L., & Srinivasan, R. (2006). Electric fields of the brain. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195050387.001.0001.
  • Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9(1), 97–113.
  • Paivio, A. (1971). Imagery and verbal processes. Psychology Press.
  • Paivio, A., & Desrochers, A. (1980). A dual-coding approach to bilingual memory. Canadian Journal of Psychology/Revue Canadienne de Psychologie,...
  • Palma, P., & Titone, D. (2021). Something old, something new: A review of the literature on sleep-related lexicalization of novel words...
  • Palmer, S. D., & Havelka, J. (2010). Age of acquisition effects in vocabulary learning. Acta Psychologica, 135(3), 310–315. https://doi.org/10.1016/j.actpsy.2010.08.002.
  • Peirce, J., Gray, J. R., Simpson, S., MacAskill, M., Höchenberger, R., Sogo, H., Kastman, E., & Kindelov, J. K. (2019). PsychoPy2: Experiments...
  • Perry, L. K., Samuelson, L. K., & Bursey, A. H. (2014). Does perceived affordance influence word learning? Evidence from contextually...
  • Peters, E., & Webb, S. (2018). Incidental vocabulary acquisition through viewing L2 television and factors that affect learning. Studies...
  • Pu, H., Holcomb, P. J., & Midgley, K. J. (2016). Neural changes underlying early stages of L2 vocabulary acquisition. Journal of Neurolinguistics,...
  • Roediger, H. L., Weldon, M. S., & Challis, B. H. (1989). Explaining dissociations between implicit and explicit measures of retention:...
  • Rugg, M. D., & Curran, T. (2007). Event-related potentials and recognition memory. Trends in Cognitive Sciences, 11(6), 251–257. https://doi.org/10.1016/j.tics.2007.04.004.
  • Rugg, M. D., Johnson, J. D., & Uncapher, M. R. (2015). Encoding and retrieval in episodic memory: Insights from fMRI. In Addis, D. R.,...
  • Stenberg, G. (2006). Conceptual and perceptual factors in the picture superiority effect. European Journal of Cognitive Psychology, 18(6),...
  • Stenberg, G., Radeborg, K., & Hedman, L. R. (1995). The picture superiority effect in a cross-modality recognition task. Memory &...
  • Talamas, A., Kroll, J. F., & Dufour, R. (1999). From form to meaning: Stages in the acquisition of second-language vocabulary. Bilingualism:...
  • Tang, K., & Baer-Henney, D. (2023). Modelling L1 and the artificial language during artificial language learning. Laboratory Phonology,...
  • Trost, S. (2022). WordCreator (22.7.4). Stefan Trost Media.
  • Weiss, D. J. (2020). Introduction: The use of artificial languages in bilingualism research. Bilingualism: Language and Cognition, 23(1),...
  • Weldon, M. S., & Roediger, H. L. (1987). Altering retrieval demands reverses the picture superiority effect. Memory & Cognition, 15(4),...
  • Wilkins, D. A. (1972). Linguistics in language teaching. Cambridge: MIT Press.
  • Williams, G. P., Panayotov, N., & Kempe, V. (2020). How does dialect exposure affect learning to read and spell? An artificial orthography...
  • Williams, G. P., Panayotov, N., & Kempe, V. (2022). Exposure to dialect variation in an artificial language prior to literacy training...
  • Yum, Y. N., Midgley, K. J., Holcomb, P. J., & Grainger, J. (2014). An ERP study on initial second language vocabulary learning: Initial...
  • Zhang, J., Huang, Y., Jiang, C., Xu, Y., Rao, H., & Xu, H. (2023). Dynamic brain responses to Russian word acquisition among Chinese adult...
  • Zhang, Y., Chen, B., Tang, Y., Yao, P., & Lu, Y. (2018). Semantic similarity to known second language words impacts learning of new meanings....
  • Zhang, Y., Lu, Y., Liang, L., & Chen, B. (2020). The effect of semantic similarity on learning ambiguous words in a second language: An...