Tcr-Translate: An AI That Translates Targets Into T Cell Receptors

The immune system’s matching problem

Start with the biology, because the whole idea rests on it. A T cell patrols the body carrying a receptor, its own personal sensor, tuned to recognize short protein fragments called peptides. Those peptides are held up for inspection by molecules on the surface of other cells, a display system known as the MHC. When a receptor meets a peptide and MHC combination that fits, the T cell springs into action. This is how your body spots an infected or cancerous cell hiding among healthy ones, sometimes down to a single amino acid of difference.

The catch is diversity. Your immune repertoire holds an enormous variety of receptors, generated by a random shuffling process, so that between them they can recognize almost anything. That is a triumph of biology and a nightmare for engineering. If a doctor wants a T cell that attacks a particular tumor, they need to find, among a practically infinite space, the rare receptor that reads that tumor’s peptide and leaves healthy cells alone. Traditional discovery means screening cells in the laboratory, a slow process with low yield, and the data that exists is heavily skewed toward a handful of well studied diseases.

The specific part of the receptor that does most of the recognizing is a short loop known as CDR3, and this work focuses on the beta chain version, written as CDR3b. Get that loop right and you have most of the specificity. So the design problem narrows to a concrete question. Given a target, what CDR3b sequence should a receptor carry?

Translating between two biological languages

Here is where the framing gets clever. The authors, Dhuvarakesh Karthikeyan and colleagues in the personalized immunotherapy group at the University of North Carolina, noticed that this looks a lot like translation. On one side you have a sentence in the language of targets, the peptide and MHC. On the other you have a sentence in the language of receptors, the amino acids of the CDR3b loop. Translation models learn exactly this kind of mapping between two sequences, so why not borrow their machinery?

They built their system, called TCR-TRANSLATE, on the encoder decoder transformers that power modern machine translation, adapting two well known architectures named BART and T5. The encoder reads the target, the decoder writes the receptor one amino acid at a time, each choice conditioned on the target and on everything written so far.

The real difficulty is data. Machine translation has millions of aligned sentence pairs. Receptor biology has comparatively few confirmed target and receptor pairings, and they cluster around a few famous viruses. So the team reached into the toolbox of low resource translation, the set of tricks developed for languages where parallel text is scarce. One trick teaches the model to translate in both directions at once, from target to receptor and back, so it can share what it learns across the two. Another adds a fill in the blank objective on unlabeled sequences. Turning a data starved biology problem into a well studied translation problem is the move that makes everything else possible, and it sits alongside a broader wave of generative design work like the protein backbone generator we covered in Salad and sparse denoising protein design and the conformational model in Mac-Diff for diverse protein shapes.

Twelve models and a cautionary tale about metrics

Rather than trust a single design, the team trained twelve variants across the two architectures and the different training tricks, then measured them carefully. What they found is one of the most valuable parts of the paper, and it is a warning as much as a result.

The training tricks that shared information between the two translation directions did raise the accuracy scores. But when the authors looked closely at why, they found something uncomfortable. Those models were quietly leaning on promiscuous receptors, the ones that bind many unrelated targets. Immunologists have a name for this, polyspecificity, and while a degree of it is a real and useful feature of the immune system, a model that mostly emits these jack of all trades receptors is gaming the benchmark. A promiscuous sequence is likely to count as a hit for many targets at once, which inflates the score without producing anything genuinely useful for a specific disease.

The evidence was stark. The more a model leaned on these promiscuous receptors, the fewer unique sequences it produced, a near perfect inverse relationship. In plainer terms, the model looked better on paper precisely because it had stopped being creative and started repeating a small set of safe, broadly binding answers. The authors call this metric hacking, and they treat it as a reason to distrust a model rather than to celebrate it.

Picking a champion, and what it learned

With that caution in mind, the team selected the model that balanced accuracy against genuine diversity and leaned least on the promiscuity crutch. That model, a fine tuned T5 variant, became the flagship they call TCRT5, and the rest of the study examines it closely.

TCRT5 captures the natural length of receptor loops well, with an average close to the real repertoire. But it has a revealing bias. Its sequences are biologically more probable than the reference set, which sounds good but actually signals a narrowing. The model gravitates toward the common, high likelihood receptors and misses the rarer ones, losing some of the diversity that makes a real immune repertoire powerful. Much of that loss concentrates near the start of the loop, where the model overuses a common structural motif. This is the same mode seeking tendency that shows up in ordinary language models, where the most probable output is often the blandest one, and the decoding method the authors use, beam search, is known to favor these safe, common patterns.

Yet within those limits, the model did something remarkable on close inspection. Out of two thousand generated sequences for its validation targets, nearly all were biologically valid, over a hundred and eighty were confirmed real binders, and seven were genuine receptors that had never appeared in its supervised training. One of those seven was absent even from the larger unlabeled corpus the model had seen, hinting that it can occasionally invent a plausible receptor from scratch rather than merely recalling one.

The model was not just memorizing famous receptors. It pulled real, functional sequences out of the space it had never been told to associate with a target, which is the difference between a lookup table and a design tool.A plain reading of the generation analysis

The test on targets it had never seen

The point of a design tool is to work on new problems, so the authors built a demanding benchmark of recent targets deliberately kept distant from anything in training, and pitted TCRT5 against two published competitors and an unconditional baseline that ignores the target entirely. Across the board, conditioning on the target helped, and TCRT5 came out ahead, even recovering exact receptor matches for a couple of targets, a needle in a haystack outcome.

The most vivid demonstration was a simulated screen against a single Epstein Barr virus target that had hundreds of known real receptors. The authors generated a thousand candidates from each model and checked how many clustered with the real ones.

TCRT5 also placed its best candidates near the top of its ranked list rather than buried in the noise, which is what a scientist running a limited number of laboratory tests actually needs. A design tool that is right but only if you test ten thousand candidates is not useful. One that puts the good answers first is.

From code to a living cell

Computational wins are cheap in this field. What lifts this paper above most is that the authors took their model to the wet laboratory and tested whether a designed receptor would actually work. They chose a hard and meaningful target, a peptide from the Wilms tumor antigen, which is associated with leukemia and is a non viral target, exactly the kind of thing missing from the virus heavy training data.

Their approach was to take an existing receptor known to recognize this target and swap in the CDR3b loops that TCRT5 designed, then express these engineered receptors in a line of reporter cells rigged to light up when the T cell activates. They built forty such constructs. The first result was encouraging on its own. Every single one of the forty folded and appeared correctly on the cell surface, which is not guaranteed and shows that this loop swapping strategy is mechanically sound.

Then came the payoff. One designed receptor, labeled F8, produced a clear activation signal against the leukemia target, well above the negative controls. The receptor genuinely fired. And there was a twist that speaks to what the model had learned. F8 was present in the large unlabeled corpus the model read during its early training but absent from the labeled data used to fine tune it. In other words, the model reached back into a vast pile of receptors of unknown function and correctly pulled out one that recognizes this specific leukemia target. It found a real, naturally occurring receptor and correctly guessed what it does.

The hit rate was one in forty. That sounds low, and the authors are honest that F8’s activity was weaker than the established control receptor. But set against traditional discovery, where useful hits can be far rarer, one in forty from a model reading only the target is a striking improvement in the odds.

The specificity problem, when a receptor is too friendly

The same experiment surfaced the field’s deepest challenge, and the authors do not hide it. When they tested whether F8 was specific to the leukemia target, they found it also reacted strongly to an unrelated pool of bacterial and viral peptides, though not to one particular control antigen. F8, in short, is somewhat promiscuous. It recognizes its intended target but also things it should ignore.

For a therapy, that is a serious problem. A receptor that attacks healthy tissue along with the tumor is dangerous, and specificity is the whole game in cell therapy safety. The finding ties directly back to the earlier warning about promiscuous receptors. The very property that inflates benchmark scores, broad binding, is also a liability in a real medicine, and it appears to be woven into both the biology and the biased training data. The paper frames this as a fundamental tension. The immune system evolved a degree of promiscuity on purpose, for broad surveillance, but a therapeutic receptor needs the opposite, a sharp and narrow focus.

The clinical translation gap

It is worth stating plainly how far this sits from a treatment, because the subject matter invites overexcitement. A receptor that activates a reporter cell in a dish has cleared the very first hurdle of many. It has not been shown to kill actual tumor cells, to spare healthy tissue reliably, to work in an animal, let alone a person, or to be safe. The cross reactivity that F8 displayed is precisely the kind of finding that would stop a candidate from advancing without substantial redesign.

Real cell therapies take years of preclinical work, careful safety engineering, and staged clinical trials before they reach patients, and most candidates fail somewhere along that path. What this paper offers is not a therapy but a faster way to propose starting points, narrowing an impossibly large search space down to a shortlist worth testing. That is genuinely valuable, because the search space is the bottleneck, but it is a tool for the earliest, most exploratory end of the pipeline. Reading it as anything closer to the clinic would misrepresent both the paper and the science. The authors themselves frame their contribution as reducing the search space for novel targets, not as designing a drug.

Honest limitations

The paper is admirably candid about its boundaries, and several deserve emphasis. The most fundamental is that the model designs only one loop of the receptor, the CDR3b, and grafts it onto an existing scaffold. Real receptor specificity also depends on the other chain of the receptor and on genetic segments the model never touches, so a designed loop is only part of the story. A receptor is not just its most important loop, and the rest matters for both binding and safety.

The evaluation itself rests on imperfect proxies. Exact sequence recovery, similarity thresholds, and clustering are stand ins for the thing that actually matters, real function, and they are noisy. The authors also flag a subtle circularity in the benchmarks. The reference sets are built from receptors that scientists happened to discover, which skews toward common, easily observed sequences, so a model can score well partly by reproducing that discovery bias rather than by capturing true biology. Beam search compounds this by favoring the common and probable, which is why the generated repertoire loses diversity. And the whole enterprise is constrained by the scarcity and bias of the underlying data, concentrated as it is on a few well studied viruses, which is a large part of why a non viral target like the leukemia antigen was such a challenge in the first place. For readers interested in how other groups are pushing generative models into cancer and genomics with the same mix of promise and caution, our coverage of a cell free RNA foundation model for liquid biopsy and of a generative classifier for blood cell morphology are useful companions.

The core method, in runnable PyTorch

The block below is a faithful, self contained reference implementation of the translation idea at the heart of the paper, namely an amino acid tokenizer, a compact encoder decoder transformer that maps a target to a CDR3b sequence, the conditional training objective, a switch for the bidirectional and multitask training schemes, and a beam search decoder, with a smoke test. It follows the framework described in the paper rather than reproducing the authors’ exact models, which use full scale BART and T5 backbones.

# TCR-TRANSLATE: casting antigen-specific receptor design as translation.
# A compact encoder-decoder that maps a target (pMHC) to a CDR3-beta sequence,
# with conditional / bidirectional / multitask training and beam search.
import torch
import torch.nn as nn
import torch.nn.functional as F

AA = "ACDEFGHIKLMNPQRSTVWY"                         # 20 amino acids
SPECIAL = ["<pad>", "<bos>", "<eos>", "<mask>", "<tcr>", "<pmhc>"]
VOCAB = {t: i for i, t in enumerate(SPECIAL + list(AA))}
PAD, BOS, EOS = VOCAB["<pad>"], VOCAB["<bos>"], VOCAB["<eos>"]

def encode(seq, add_dir=None):
    ids = [VOCAB[a] for a in seq if a in VOCAB]
    if add_dir: ids = [VOCAB[add_dir]] + ids       # direction tag for bidirectional
    return [BOS] + ids + [EOS]

# ----------------------------------------------------------------------
# Encoder-decoder transformer (T5/BART-style, scaled down)
# ----------------------------------------------------------------------
class TCRTranslate(nn.Module):
    def __init__(self, vocab=len(VOCAB), d=256, heads=8, layers=4, ff=1024):
        super().__init__()
        self.emb = nn.Embedding(vocab, d, padding_idx=PAD)
        self.pos = nn.Parameter(torch.randn(1, 128, d) * 0.02)
        self.core = nn.Transformer(d, heads, layers, layers, ff,
                                   batch_first=True, norm_first=True)
        self.out = nn.Linear(d, vocab)
        self.d = d

    def _embed(self, x):
        return self.emb(x) * (self.d ** 0.5) + self.pos[:, :x.size(1)]

    def forward(self, src, tgt_in):
        src_pad = (src == PAD); tgt_pad = (tgt_in == PAD)
        L = tgt_in.size(1)
        causal = torch.triu(torch.ones(L, L, device=src.device) * float("-inf"), 1)
        h = self.core(self._embed(src), self._embed(tgt_in),
                      tgt_mask=causal,
                      src_key_padding_mask=src_pad, tgt_key_padding_mask=tgt_pad,
                      memory_key_padding_mask=src_pad)
        return self.out(h)

# ----------------------------------------------------------------------
# Training: conditional pMHC->TCR, plus optional bidirectional / multitask
# ----------------------------------------------------------------------
def seq2seq_loss(model, src, tgt):
    logits = model(src, tgt[:, :-1])
    return F.cross_entropy(logits.reshape(-1, logits.size(-1)),
                           tgt[:, 1:].reshape(-1), ignore_index=PAD)

def training_objective(model, pmhc, tcr, scheme="conditional"):
    # scheme in {"conditional", "bidirectional", "multitask"}
    loss = seq2seq_loss(model, pmhc, tcr)                 # target -> receptor
    if scheme in ("bidirectional", "multitask"):
        loss = loss + seq2seq_loss(model, tcr, pmhc)     # receptor -> target too
    if scheme == "multitask":
        # masked-reconstruction term (fill-in-the-blank) on each sequence
        for seq in (pmhc, tcr):
            corrupt = seq.clone()
            m = (torch.rand_like(seq.float()) < 0.15) & (seq != PAD)
            corrupt[m] = VOCAB["<mask>"]
            loss = loss + 0.5 * seq2seq_loss(model, corrupt, seq)
    return loss

# ----------------------------------------------------------------------
# Beam search decoding (mode-seeking, as used in the paper)
# ----------------------------------------------------------------------
@torch.no_grad()
def beam_search(model, src, beams=5, max_len=30):
    model.eval()
    beam = [([BOS], 0.0)]                                 # (tokens, log-prob)
    for _ in range(max_len):
        cand = []
        for toks, score in beam:
            if toks[-1] == EOS:
                cand.append((toks, score)); continue
            tgt = torch.tensor([toks], device=src.device)
            logp = F.log_softmax(model(src, tgt)[0, -1], dim=-1)
            top = torch.topk(logp, beams)
            for lp, tok in zip(top.values.tolist(), top.indices.tolist()):
                cand.append((toks + [tok], score + lp))
        beam = sorted(cand, key=lambda x: x[1], reverse=True)[:beams]
        if all(t[-1] == EOS for t, _ in beam): break
    inv = {i: t for t, i in VOCAB.items()}
    best = beam[0][0]
    return "".join(inv[i] for i in best if i > VOCAB["<pmhc>"])   # amino acids only

# ----------------------------------------------------------------------
# Smoke test on dummy target/receptor pairs
# ----------------------------------------------------------------------
if __name__ == "__main__":
    model = TCRTranslate()
    # toy pair: a peptide 'GILGFVFTL' -> a receptor loop 'CASSIRSSYEQYF'
    src = torch.tensor([encode("GILGFVFTL")])
    tgt = torch.tensor([encode("CASSIRSSYEQYF")])
    opt = torch.optim.AdamW(model.parameters(), lr=1e-3)
    for step in range(50):                              # overfit one pair as a sanity check
        loss = training_objective(model, src, tgt, scheme="multitask")
        opt.zero_grad(); loss.backward(); opt.step()
    print("final loss:", round(float(loss), 3))
    print("designed CDR3b:", beam_search(model, src))
    print("smoke test ok")

Conclusion

The central achievement of this work is a reframing that turns an intractable biological search into a familiar machine learning task. By casting receptor design as translation, the authors could bring the full toolbox of sequence models, including the hard won tricks for languages with little data, to bear on a problem where paired examples are painfully scarce. The result is a model that reads a disease target and proposes the receptors most likely to recognize it, and that puts its best guesses near the top of the list where a scientist can actually test them.

What makes the paper trustworthy is that it does not stop at benchmark scores, and in fact treats a rising score with suspicion. The discovery that some training tricks boost accuracy by generating promiscuous, broadly binding receptors is a genuine contribution to how this field should measure itself. It is a reminder that in generative design, as in language modeling, a model can improve on a metric by learning to exploit it, and that diversity and true specificity can quietly erode while the numbers climb.

The wet laboratory validation is what sets this apart from the many purely computational entries in the field. Building forty designed receptors, watching all of them express correctly, and finding one that fires against a leukemia associated target is a real experimental result, not a simulation. That the successful receptor was pulled from a sea of unlabeled sequences the model had merely read, rather than been trained to associate with the target, is the strongest evidence that the system is designing rather than memorizing.

The honest limitations are substantial and the authors name them without flinching. The model shapes only one loop of a larger receptor, the evaluation leans on proxies for function, the benchmarks carry a discovery bias, and the one successful receptor also bound things it should not have. That last point, the cross reactivity of the winning design, is not a footnote but the crux. It is the exact obstacle, specificity, that separates an interesting demonstration from a safe therapy, and the paper surfaces it rather than burying it.

Read as a cure, this is nothing of the sort, and the disclaimer at the top of this article is not a formality. Read as a method, which is whether a target can be translated into a functional immune receptor from its sequence alone, it offers a careful and partly validated yes, along with a clear map of the specificity problem that stands in the way. In a domain where the search space has always been the enemy, a tool that shrinks that space to a testable shortlist, and that is honest about what it cannot yet promise, is a real step forward.