John praised Mary because he ? Implicit Causality Bias and Its Interaction with Explicit Cues in LMs

Some interpersonal verbs can implicitly attribute causality to either their subject or theirobject and are therefore said to carry an implicit causality (IC) bias. Through this bias,causal links can be inferred from a narrative,aiding language comprehension. We investigate whether pre-trained language models(PLMs) encode IC bias and use it at inferencetime. We find that to be the case, albeit todifferent degrees, for three distinct PLM architectures. However, causes do not alwaysneed to be implicit—when a cause is explicitlystated in a subordinate clause, an incongruentIC bias associated with the verb in the mainclause leads to a delay in human processing.We hypothesize that the temporary challengehumans face in integrating the two contradicting signals, one from the lexical semantics ofthe verb, one from the sentence-level semantics, would be reflected in higher error ratesfor models on tasks dependent on causal links.The results of our study lend support to this hypothesis, suggesting that PLMs tend to prioritize lexical patterns over higher-order signals.