METEMDonate

2026.10.27 — Research

Hearing Without Judging

What a moralizing machine does to human feeling

Abstract

When someone describes a wrongdoing followed by a gesture of pleasure, language models usually respond with what the rule predicts, remorse, rather than with what the gesture shows. We measure this substitution on 200 first-person twin narratives: the same wrongdoing, two opposite gestures, and no feeling ever named. Five human readers identify the feeling in 99% of cases. Some twenty language models almost always identify the negative feeling, but identify the positive one in only 1% to 87% of cases, depending on the system. Given the gesture alone, without the wrongdoing, the same models read it correctly in 68% to 97% of cases: the ability is there, and it is the wrongdoing that switches it off. Removing censorship does not fix the problem and degrades moral judgment. The bias predates alignment, and it also affects non-generative systems (emotion classifiers, entailment encoders and sentence-transformers), whose representations place most positive feelings that follow a wrong on the negative side. Its source lies in the data: texts in which feelings are attributed by readers. In response, we propose a corpus of feelings reported in the first person, starting with divergent ones, which the Metem Foundation is committed to building and releasing openly, and on which models, generative or not, should be trained and evaluated. We discuss what this substitution might do when repeated at scale, and propose a design principle: the right to be heard.

Read the publication