Probabilistic language models in cognitive neuroscience: promises and pitfalls

Krstijan Armeni, Roel M. Willems, and Stefan L. Frank


Cognitive neuroscientists of language comprehension study how neural computations in the brain relate to cognitive computations during comprehension. On the cognitive part of the equation, it is important that the computations and processing complexity are explicitly de ned. Probabilistic language models can be used to give a computationally explicit account of language complexity during comprehension. Whereas such models has been so far predominantly evaluated against behavioral data, only recently have the models been used to explain neurobiological signals. Measures obtained from these models emphasize the probabilistic, information-processing view of language understanding and provide a set of tools that can be used for testing neural hypotheses about language comprehension. Here, we provide a cursory review of the theoretical foundations and example neuroimaging studies employing probabilistic language models. We highlight the advantages and potential pitfalls of this approach and indicate avenues for future research.

Keywords: cognitive neuroscience of language, computational linguistics, EEG, MEG, fMRI, probabilistic language models, information theory, surprisal, entropy