Can LLMs Reliably Judge Empathic Communication

Generating empathy is not the same as judging it

Language models have turned out to be surprisingly good at sounding caring. People often rate a model’s supportive reply as more empathic than a human’s, and some report feeling less anxious after talking to a chatbot built for support. That skill has pulled these systems into emotionally loaded corners of life, from customer service to companionship to informal mental health support.

Here is the distinction the paper hangs on. Being good at producing an empathic message is not the same as being good at judging one. A model that writes a warm reply might still be a poor grader of whether some other reply was warm, and the second skill is the one we need if we want oversight. To know whether an AI companion is communicating well, or to give a trainee counsellor useful feedback at scale, you need a reliable judge, not just a fluent talker.

The reason this matters is that the alternative is either slow or unreliable. Human experts can judge empathic communication well, but there are not enough of them to review the flood of conversations these systems generate. Crowdworkers are cheap and plentiful, and, as the study shows, not very reliable at this particular task. If a language model could stand in as a careful judge, it would open a path to accountability that neither of the human options offers on its own. That same tension between scaling automation and trusting it runs through our piece on why agentic AI adoption is outrunning its reliability.

How you measure a judge

The clever part of the study is methodological. To ask whether a model is a good judge, you first need a fair way to score judging itself, and the authors are pointed about doing it right. They collected annotations from three groups on the same two hundred conversations, each a dialogue where one person shares a personal difficulty and another tries to help. Three experts, a pool of crowdworkers, and three language models all rated the same exchanges against the same rubrics.

Those rubrics came from four established frameworks for empathic communication, drawn from psychology, communication studies, and natural language processing, broken into twenty one specific subcomponents. Some are concrete, such as whether the responder gave practical advice or asked questions to draw the other person out. Others are subtle, such as whether the response made the discloser feel understood or truly seen. The mix is deliberate, because it lets the study see where judging is easy and where it gets slippery.

The statistics that keep it honest

The heart of the measurement is interrater reliability, how much two judges agree beyond what chance would give. The main tool is a weighted version of Cohen’s kappa, which rewards agreement and, crucially, treats a small disagreement as less serious than a large one.

The weights are quadratic, which means a judge rating something a four when another says five is barely penalized, while a four against a one counts heavily. This suits an ordinal rating scale, where being one step off is almost agreement and being far off is a real dispute.

The authors also report Krippendorff’s alpha, a related measure that generalizes to any number of annotators, and they compare all of this against the classification metrics that most similar studies lean on. That comparison turns out to be one of the paper’s sharpest contributions.

What they found

The central result is that language models are genuinely reliable judges, closing most of the gap to human experts. Experts agreed with each other at a median weighted kappa of about 0.58, the natural ceiling for a subjective task where even trained people sometimes see things differently. When a model was guided by a good prompt, its agreement with experts landed at a median of 0.60, essentially matching the expert to expert level.

Two findings sit alongside that headline and deserve weight. First, the model’s reliability tracked the experts’ own. Where experts agreed strongly, the model agreed strongly with them, and where experts wavered, so did the model, a correlation of 0.67 between the two patterns. The model was not confidently wrong in the hard cases. It was uncertain in the same places a human expert would be, which is exactly the behaviour you want from a trustworthy judge.

Second, the crowd came off poorly. Crowdworkers agreed with experts at only about half the rate the models did, and their ratings were systematically inflated, scoring responses as more empathic than experts did on eighteen of the twenty one subcomponents. The authors read this as lay raters leaning on a gut sense that it is the thought that counts, giving credit for apparent good intentions rather than judging the message itself. It is a useful caution against the common assumption that crowdsourced labels are a safe ground truth. For a different angle on scoring how much to trust a model’s output, our look at quantifying uncertainty in LLM answers tackles the trust question from the generation side rather than the judging side.

Across all four frameworks, LLMs consistently approach this expert level benchmark and exceed the reliability of crowdworkers. These results demonstrate how LLMs, when validated on specific tasks with appropriate benchmarks, can support transparency and oversight in emotionally sensitive applications. Kumar and colleagues, Nature Machine Intelligence, 2026

Why the prompt did the heavy lifting

The models did not reach this level out of the box. The authors found that how you ask matters enormously, and they landed on a prompt with three ingredients. It spelled out the evaluation framework grounded in the research literature, it showed three worked examples with the scores experts had given, and it asked the model to rate a new conversation on the same scale. Stripping out the examples, or the framework, lowered reliability.

This is a concrete lesson in prompt design for evaluation tasks. Giving a model a rubric and a few expertly scored examples is what let it internalize what a four means as opposed to a five on a particular dimension, the same in context learning that powers so much modern practice. Readers building this kind of thing will find the groundwork in our explainer on prompt engineering and context engineering. The study also checked three different models, and found they behaved similarly, which suggests the reliability comes from the method rather than from one vendor’s secret sauce.

Some things are just hard to judge

The most useful nuance is that reliability was not uniform. It rose and fell with how observable the thing being judged was. Anyone rating whether a responder gave practical advice, or asked questions to encourage the other person to elaborate, could point to specific words on the page, and agreement there was high for experts and models alike.

The subjective dimensions were harder for everyone. Judging whether a response truly demonstrates understanding, or correctly interprets what the other person is feeling, means reading intentions and inner states that the transcript never spells out. Experts disagreed more there, and so did the models. This is not a failure of the model so much as a property of the task. Some aspects of empathy are inherently contestable, and no judge, human or machine, can be crisply reliable about something that reasonable people genuinely see differently.

That insight feeds the paper’s methodological argument. Because these judgements live on a scale and some disagreement is unavoidable, the authors show that the accuracy style metrics common in this field are misleading. A single accuracy number can look impressive by nailing a lopsided majority class while missing every hard case, or can look terrible while actually capturing agreement to within one point on the scale. Reliability measures that respect the ordering of the ratings tell a truer story, and the choice of where to draw a threshold for a pass or fail score can swing a reported number wildly, in one case from sixteen to ninety five out of a hundred on the very same data.

From a benchmark to a real deployment

This is where care has to take over from enthusiasm, and the authors are thoughtful about it. Showing that a model can reliably judge empathic communication in a controlled study is not the same as safely turning it loose in a real emotionally sensitive product, and the distance between those two things is where the risk lives.

The conversations studied were between strangers, a reasonable stand in for many companion style interactions but a poor match for the long, entangled relationships people form with an AI they talk to every day. The study measured whether a model agrees with experts on a rating, not whether deploying such a model as a live monitor would actually keep anyone safe. The authors are explicit that their frameworks capture a slice of empathic communication, that the subjective subcomponents remain shaky, and that any real deployment should be validated against domain specific standards for acceptable error before it is trusted in professional training or in watching over an AI companion.

The stakes are not hypothetical. The paper opens by noting that unreliable or biased evaluation could quietly propagate poor practices, and that unchecked empathic responses from AI companions have in some cases fostered unhealthy dependence or worse, including the most serious harms. A reliable judge is meant to be part of the safety net that catches those failures, which makes it all the more important that the judge is validated for the exact job it is doing rather than assumed to generalize. A tool that is reliable on a benchmark of stranger conversations is a promising building block, not a finished safeguard.

Where the study falls short

Beyond that gap, the authors name several honest limits.

The expert benchmark itself is a conservative floor rather than a gold standard. The three experts followed a brief shared rubric rather than the iterative, deeply coded process that communications researchers usually use to build reliable annotation, so the expert agreement reported here may understate what carefully trained coders could reach. That cuts both ways. It means the bar the models cleared might rise with stricter human coding, and it means further prompt refinement might lift the models too.

The scope is also bounded. Everything rests on English language text conversations between strangers, four particular frameworks, and a specific set of models at a moment in time. Empathic communication in other languages, in voice or video, or within a long relationship might behave differently, and the models tested will be superseded. The study is a rigorous snapshot, not a permanent verdict, and the authors frame it that way.

Finally, reliability is not the same as validity. A model can agree with experts and still be agreeing about the wrong thing if the framework itself is flawed, and the paper shows that some of these widely used frameworks are internally muddy, with overlapping or redundant subcomponents. Judging consistently against a shaky rubric produces consistent answers, not necessarily correct ones. The authors are careful to present their models as reliable relative to experts on these frameworks, not as arbiters of empathy in some absolute sense.

Why it matters

Step back and the contribution is a piece of infrastructure for trust. As language models move deeper into emotionally charged roles, the ability to audit how they communicate becomes a safety requirement, not a nicety. This study shows that another language model can do a meaningful part of that auditing, reliably enough to be useful and cheaply enough to scale, provided it is prompted carefully and validated on the specific task.

The methodological lesson travels well beyond empathy. The insistence on measuring a judge against human agreement rather than against an assumed ground truth, and on using reliability statistics rather than accuracy for subjective tasks, is a better template for evaluating language models on any fuzzy, human judgement laden problem, from grading essays to moderating content. Too many evaluations quietly pretend a single correct label exists where it does not. This one refuses that shortcut, and the field is better for it. It sits alongside a growing effort to put LLM evaluation on firmer ground, a theme our coverage of trust scoring for model outputs shares. Used with that discipline, an automated judge can add transparency to systems that badly need it. Used carelessly, it just launders an unreliable opinion through a confident machine, and knowing the difference is the whole point.

A reference implementation

Because this study analyses annotations rather than training a model, a fabricated neural network would misrepresent it. The code below implements the paper’s genuine analytical core in plain Python, the weighted Cohen’s kappa and Krippendorff’s alpha used to measure how well two judges agree, and it demonstrates them on synthetic expert, model, and crowd ratings that mimic the reported pattern. The authors full code and data are linked under the block.

# Measuring how reliably a judge agrees, the analytical core of the paper
# Weighted Cohen's kappa and Krippendorff's alpha on ordinal Likert ratings.
# No model is trained here, the study analyses annotations.

import numpy as np


def weighted_kappa(a, b, k):
    # quadratic weighted Cohen's kappa between two raters on a k point scale
    a, b = np.asarray(a), np.asarray(b)
    obs = np.zeros((k, k))
    for x, y in zip(a, b):
        obs[x, y] += 1
    obs /= obs.sum()

    ra, rb = obs.sum(axis=1), obs.sum(axis=0)      # marginal distributions
    exp = np.outer(ra, rb)                          # agreement expected by chance

    idx = np.arange(k)
    w = (idx.reshape(-1, 1) - idx.reshape(1, -1)) ** 2 / (k - 1) ** 2
    return 1.0 - (w * obs).sum() / (w * exp).sum()


def krippendorff_alpha(ratings, k):
    # ratings is a list of columns, one per item, each holding every rater's score
    num, den = 0.0, 0.0
    all_vals = np.concatenate(ratings)
    # expected disagreement across the whole pool of ratings
    de = np.mean([(x - y) ** 2 for x in all_vals for y in all_vals])
    for col in ratings:
        m = len(col)
        if m < 2:
            continue
        do = np.mean([(x - y) ** 2 for x in col for y in col]) * m / (m - 1)
        num += do
        den += de
    return 1.0 - num / den


if __name__ == "__main__":
    rng = np.random.default_rng(0)
    k = 5
    n = 200
    truth = rng.integers(0, k, size=n)             # a latent quality per conversation

    # experts and the model stay close to the truth, the crowd is noisy and inflated
    expert1 = np.clip(truth + rng.integers(-1, 2, n), 0, k - 1)
    expert2 = np.clip(truth + rng.integers(-1, 2, n), 0, k - 1)
    model = np.clip(truth + rng.integers(-1, 2, n), 0, k - 1)
    crowd = np.clip(truth + rng.integers(-1, 3, n) + 1, 0, k - 1)   # upward bias

    print(f"expert vs expert  kappa {weighted_kappa(expert1, expert2, k):.3f}")
    print(f"expert vs model   kappa {weighted_kappa(expert1, model, k):.3f}")
    print(f"expert vs crowd   kappa {weighted_kappa(expert1, crowd, k):.3f}")

    cols = [np.array([expert1[i], expert2[i], model[i]]) for i in range(n)]
    print(f"alpha across expert and model raters {krippendorff_alpha(cols, k):.3f}")

Conclusion

The core finding is that a language model, prompted with care, can judge empathic communication about as reliably as a human expert, and far more reliably than a crowdworker. Across four frameworks and two hundred real conversations, the model agreed with experts at a level matching how experts agree with one another, and it grew uncertain in exactly the places experts do, rather than being confidently wrong.

The conceptual shift worth remembering is the insistence on the right yardstick. By measuring a judge against expert agreement instead of an invented ground truth, and by using reliability statistics that respect an ordinal scale rather than accuracy scores that flatten it, the authors reveal what most evaluations of subjective tasks paper over. The reliability of a judge depends on how observable the thing being judged is, and pretending otherwise with a single accuracy number hides the parts that are genuinely hard.

The approach travels because its discipline is general. Any effort to have one model evaluate another on a fuzzy, human judgement laden task, from moderation to grading to safety review, would be more honest for adopting the same practices, benchmarking against human agreement and reporting reliability rather than accuracy. Paired with an open release of the data and code, that makes this work both a specific result about empathy and a template for evaluating language models where no single right answer exists.

The honest limits keep it grounded. The expert benchmark is a conservative floor rather than a gold standard, the scope is bounded to English text conversations between strangers and a fixed set of models, some frameworks are internally muddy, and reliability against experts is not the same as being right in any absolute sense. Most importantly, a judge that is reliable on a benchmark is a building block for oversight, not a finished safeguard for the emotionally sensitive products where these systems increasingly live.

Future directions follow naturally. Sharpen the frameworks so the subjective dimensions become more consistently judgeable, extend the evaluation to voice, to other languages, and to the long relationships that companion apps actually create, and pair automated judging with clear standards for acceptable error before it watches over anyone. If those steps hold up, the field gains a scalable and transparent way to keep an eye on how machines talk to us when it matters most, which is precisely when we should least be willing to take their empathy on faith.

Frequently asked questions