Why Even The Best Protein Localization Predictor Still Misses Where Proteins Actually Go

Why we need a reliable protein localization predictor

Every cell is really a city of tiny neighborhoods. Mitochondria run the power grid. Lysosomes handle waste disposal. The nucleus holds the blueprints and the nucleolus builds the machinery that reads them. A protein has to reach the right neighborhood to do its job, and when it does not, the consequences show up as disease. The paper opens with this framing directly, noting that determining where a human protein localizes offers real insight into both its normal role and the molecular mechanisms behind a wide range of human diseases.

Computational tools that predict localization straight from a protein’s amino acid sequence have an obvious appeal. Wet lab imaging experiments are slow, expensive, and cannot realistically be run on every one of the roughly twenty thousand human proteins, let alone on every possible disease causing mutation of each one. A reliable sequence based predictor could flag likely sorting signals, help annotate proteins nobody has studied yet, and hint at which mutations might cause a protein to end up somewhere it should not be. That last use case, spotting mislocalization caused by disease mutations, is exactly what this paper eventually stress tests, and exactly where the story gets uncomfortable.

A field that graded its own homework on an easy test

Sequence based localization prediction is not new. DeepLoc, published years ago, was the first deep learning model for the task, using convolutional and recurrent networks trained on labeled sequences. Since then models such as MULocDeep added attention mechanisms and support for both broad and fine grained compartments, while DeepLoc2 and LAProtT5 moved to protein language models, large networks such as ProtT5 and ESM1 that are pretrained on millions of unannotated sequences and produce rich per residue representations without needing a multiple sequence alignment.

The authors argue that despite this progress, the field has been evaluating itself under conditions that flatter the models. Many published tools only support single label classification, which quietly ignores the fact that close to half of all human proteins localize to more than one compartment and frequently perform genuinely different jobs in each location. Others restrict predictions to about ten coarse compartments, following DeepLoc’s original scheme, which sidesteps the finer architecture of the cell entirely. Several models were evaluated mostly on nonhuman proteins despite well documented species and cell type specific localization patterns. And the test sets themselves have often been small. DeepLoc2, one of the strongest existing tools, was evaluated on just 1717 proteins, covering only 8.5% of the human proteome.

Building a test set three databases can agree on

The three sources are genuinely different kinds of evidence. UniProt’s manually curated SwissProt subset draws on literature across more than 500,000 proteins and uses over 300 distinct localization terms sourced from decades of individual experiments. The Human Protein Atlas contains antibody based immunofluorescence images for more than 13,000 human proteins captured across 37 different cell lines, annotated by domain experts. OpenCell instead profiles 1310 proteins that were endogenously tagged with CRISPR engineered fluorescent labels in a single cell line, HEK293T cells, which avoids some of the antibody specificity issues that can affect HPA but limits coverage to far fewer proteins and only one cell type.

These three sources frequently disagree, and the paper is unusually candid about quantifying exactly how much. Among proteins shared by HPA and UniProt, fewer than 27% have identical localization label sets, though more than 80% share at least one compartment label. HPA reports multiple compartments for 62.9% of its level one annotated proteins, compared with 36.7% in UniProt and 31.6% in OpenCell, a gap the authors trace partly to the fact that imaging based methods like HPA and OpenCell are simply better at catching multilocalization than older biochemical approaches. The team also flags 583 specific proteins where HPA and UniProt share no common fine grained localization annotation at all, and publishes that list as a supplementary resource for anyone who wants to dig into why.

Rather than trying to force these disagreements into a single answer, the researchers used the disagreement itself as a filter. Only proteins with at least one label independently corroborated by two of the three databases made it into the HOU test set, and the remaining proteins from HPA and UniProt became a fivefold cross validation training set, built with sequence similarity under 40% identity between training and test data and across folds so that models could not simply memorize near duplicate sequences.

How these models are actually built, and which one won

Nearly every sequence based localization model follows the same basic shape. A protein sequence goes into either a protein language model or a position specific scoring matrix built from sequence alignments, producing a representation for every residue. Those per residue vectors then get compressed into one fixed length summary for the whole protein using some aggregation strategy, and a multilayer perceptron turns that summary into a set of predicted compartments.

The authors first benchmarked three previously published systems on their new test set, DeepLoc2 and LAProtT5, both built on protein language models, and MULocDeep, built on the older position specific scoring matrix approach, alongside a random Bernoulli baseline for context. They then ran their own systematic sweep, pairing four widely used protein language models, ESM2, ESM3, ProtT5, and ProtBert, with four aggregation strategies, max pooling, mean pooling, light attention, and multihead attention, testing all 16 combinations under both focal loss and cross entropy loss.

ProtT5 paired with multihead attention pooling, which the authors shorten to ProtT5 MHA, came out on top across most averaged metrics and became the reference model for every later analysis in the paper. Interestingly, mean pooling proved surprisingly effective as well, particularly when paired with ESM2, while max pooling consistently underperformed the attention based approaches. LAProtT5 stood out for a different reason, it predicted an average of fewer than two level one labels per protein, a conservative habit that boosted its precision on large compartments, while DeepLoc2 with ProtT5 assigned an average of 3.5 labels per protein, a much looser prediction style.

Performance climbs when the question gets easier

One of the clearest patterns across every single model, old and new, is that accuracy improves substantially as the label set gets coarser. All models did well on major membrane bound organelles such as the plasma membrane, the endoplasmic reticulum, and mitochondria. Performance consistently fell for cytoskeletal compartments and for nuclear substructures such as nuclear bodies and the nucleoli fibrillar center. Even the random baseline scored moderately well on the two most common compartments, cytosol and nucleoplasm, which the authors use as a pointed reminder that raw performance numbers can look inflated simply because some classes are common rather than because a model has actually learned something useful.

Where the best model actually breaks

The paper’s confusion matrix analysis of LAProtT5 and ProtT5 MHA is where the limitations stop being abstract. Both models frequently confused nuclear speckles with nuclear bodies, mixed up lipid droplets with lysosomes, and blurred the line between the Golgi apparatus and generic vesicles. Both also produced occasional false positives across an entire cluster of rare nuclear and cytoskeletal subcompartments. Every model tested except ESM3 failed to correctly identify intermediate filaments for any protein at all, a failure the authors attribute to how scarce that particular label is in the training data rather than any deep flaw unique to one architecture.

Multilocalizing proteins told a more nuanced story. Proteins annotated with a single subcellular location were predicted with higher plain accuracy than proteins with multiple labels. Yet the macro averaged precision was actually higher for multilocalizing proteins, and the authors offer a sensible read on why, many multilocalizing proteins have one dominant compartment that happens to be large and well represented, such as the nucleus or the cytosol, which the model reliably catches even when it misses the rarer secondary location.

Structural context mattered too. Proteins with a signal or transit peptide, the short sequence tags that route proteins to the mitochondria, nucleus, or secretory pathway, were predicted far more accurately, which lines up with decades of biology showing these tags are strong and evolutionarily conserved routing signals. Membrane associated proteins were also easier to classify, likely because recurring transmembrane helix motifs give the model something concrete to latch onto. Intrinsically disordered proteins, by contrast, were harder to classify on both accuracy and precision, plausibly because their function and location can shift with cellular context in ways a static sequence cannot fully capture.

Peeking inside the model’s attention

To find out whether ProtT5 MHA was learning anything biologically sensible rather than just pattern matching on label frequency, the authors examined its attention weights directly, comparing the regions of a sequence the model focused on against two external sources, curated sorting signals from DeepLoc2 and generic sequence motifs from the PROSITE database.

The results were genuinely encouraging on one front. Fifty percent of the model’s attention peaks overlapped an annotated sorting signal, and the model detected at least one peak overlapping 76% of all known sorting signals in its test set. Overlap with the broader, less localization specific PROSITE motifs was much lower, only 18% of attention peaks overlapped a PROSITE motif and only 22% of those motifs were detected at all, which the authors note is expected since most PROSITE motifs describe general protein function rather than trafficking. The model reliably picked up on nearly every category of sorting signal except peroxisome targeting signals, likely reflecting how rare peroxisomal proteins are in the training data.

The team also went looking for motifs the model had learned that nobody had explicitly labeled, clustering attention peaks from correctly classified proteins that fell outside any known annotation. They surfaced nine recurring fuzzy motifs shared across three or more species. One example, a hydrophobic rich sequence pattern the authors label as resembling plasma membrane targeting sequences, showed up consistently across species in plasma membrane localizing homologs. Others, such as a motif found in vesicle localizing proteins, had no obvious biological explanation the team could confidently attach, which is exactly the kind of honest uncertainty a benchmark paper should report rather than paper over.

Adding interaction networks did not deliver the boost anyone hoped for

Proteins that physically interact generally have to be in the same place to do so, which makes protein protein interaction networks a plausible extra source of localization signal. The authors extended ProtT5 MHA with GraphSAGE, a graph neural network architecture, to build a combined model called ProtT5 MHA PPI, drawing interaction data from the BioPlex and STRING databases.

The result was underwhelming. ProtT5 MHA PPI did not meaningfully outperform the plain sequence only model on any averaged metric, and it only clearly helped for three specific compartments once the team compared confusion matrices built with and without the interaction edges. For the endoplasmic reticulum, true positive predictions rose by 10%, an additional 16 proteins correctly flagged. For nuclear speckles, true positives rose by 18%, ten more correct detections. For the nucleoli fibrillar center, the effect ran the other way, interaction data helped mainly by cutting false positives, increasing correct true negative predictions by 13%, though it also introduced 66 additional false positives along the way. Digging into why, the team found that the median fraction of a protein’s interactors sharing its exact compartment label was under 0.5 for nearly every class, meaning the interaction networks were simply too noisy and too incomplete, and too often capturing interactions that happen at the interface between two compartments, to reliably reinforce a clean localization signal.

Clinical translation gap

This is the section where the paper’s findings matter most directly for anyone thinking about using these tools near real patients, and where the gap between a benchmark leaderboard and a clinically useful tool becomes hardest to ignore.

The authors evaluated ProtT5 MHA against real experimentally measured protein variants from a separate study by Lacoste and colleagues, which had systematically characterized the subcellular localization of over 3000 missense variants across more than 1000 genes, and had found that roughly one sixth of pathogenic variants actually mislocalize compared with their normal counterpart. After filtering for genes and variants that mapped cleanly onto their labeling scheme, the team retained 103 genes and 308 variants, of which 169 were classified as mislocalized and 139 as stationary, meaning they localized identically to the unmutated wild type protein.

The model’s accuracy dropped sharply once mutated sequences entered the picture. ProtT5 MHA scored well on wild type protein sequences, moderately lower on stationary variants, and then fell substantially on mislocalized variants across nearly every compartment, with the plasma membrane hit especially hard. The most telling number in the entire paper may be this one, when the researchers checked how often the model actually changed its predicted localization in response to a disease causing mutation, it did so for only 11 out of 308 variant sequences. In other words, the model was largely insensitive to the exact kind of single residue change that, in real patients, can be the difference between a correctly folded, correctly routed protein and one that misfolds, gets stuck in the wrong compartment, and contributes to disease.

Discussion, and where the authors think the field needs to go

The paper’s own discussion is refreshingly blunt for a benchmark study. Despite years of steady published progress, the authors conclude that subcellular localization prediction has not actually been solved, and that even the strongest model in their comparison shows limited performance on multilocalizing proteins and rare compartments, and fails to generalize to mislocalizing variants in any meaningful way.

Their concrete recommendations fall into a few buckets. Future models could explicitly guide attention toward a wider set of known localization signals and function related motifs rather than relying on the model to discover them implicitly, an approach DeepLoc2 already partially uses. Models that incorporate protein protein interaction data could adopt more flexible architectures that selectively weight interaction information only where it is likely to help, rather than treating every interactor equally the way the current graph extension does. And models that explicitly try to predict membrane insertion behavior or structural stability directly, rather than inferring everything from sequence patterns alone, may be needed to catch the instability driven mislocalization that sequence alone cannot see.

More ambitiously, the authors suggest the field’s underlying framing may itself be too narrow. Nearly every current approach assumes a protein has one canonical sequence, even though most human proteins have multiple isoforms that can localize completely differently depending on cell type, an issue the authors ran into directly while filtering their own variant dataset because the specific isoforms studied by Lacoste and colleagues did not always match UniProt’s chosen canonical sequence. Every current approach also treats localization as fixed, despite clear evidence that localization patterns shift across the cell cycle and across cell types. And framing the whole problem as multilabel classification compresses what is, in reality, a continuous spatial pattern captured by microscopy into a small set of discrete labels, something newer sequence to image models are starting to address more directly by modeling localization as a distribution over the cell rather than a checklist of compartments.

Clinical limitations of the underlying evidence

A few constraints are worth stating plainly for anyone weighing how far these results should be trusted for anything approaching clinical use. The pathogenic variant analysis rested on a relatively small filtered sample, 308 variants across 103 genes, drawn from a single external dataset, and 45 of the 103 wild type proteins involved were already present in the model’s own training set, which the authors kept only because the filtered dataset was already limited in size. That overlap means the wild type performance numbers in that comparison are somewhat optimistic relative to a fully independent test.

The benchmark itself, while much larger than prior efforts, still leans heavily on annotations derived from a small number of experimental platforms, chiefly antibody based imaging in HPA and CRISPR tagging in OpenCell, both of which the paper documents as having their own biases, antibody nonspecificity in HPA’s case and a single cell type restriction in OpenCell’s. The 583 proteins where HPA and UniProt could not agree on any fine grained label at all are a visible reminder that ground truth itself is not fully settled for a meaningful slice of the human proteome. Given these constraints, and given that the paper is itself a peer reviewed Registered Report rather than a preprint, its central conclusion, that current models remain unreliable specifically for rare compartments and for disease causing variants, should be read as a call for more rigorous tool development rather than as a verdict on any single patient’s genetic result.

Complete PyTorch implementation of a sequence to localization model

The code below reimplements the general architecture shown in the paper’s own Figure 2a, a protein language model style embedding compressed by a choice of aggregation strategy and classified into multiple possible compartments by a small feed forward head. Since running an actual multi billion parameter protein language model is not practical in a short demo script, the embedding step below is a compact stand in that produces per residue vectors, letting you see the full aggregation and multilabel classification pipeline end to end and swap in a real pretrained encoder later without changing anything downstream.

Running this script trains a tiny model under each of the four pooling strategies compared in the paper and prints the loss trajectory for each, which should confirm the whole encoder, pooling, and classification pipeline works end to end. Swapping the toy encoder for a real frozen ProtT5 or ESM2 model and pointing the training loop at the actual HOU dataset described in the paper is the natural next step for anyone who wants to reproduce the real benchmark numbers, and the original authors have released their full dataset construction and training code publicly for exactly that purpose.

What this means for the next generation of tools

Reading this paper alongside the broader direction of protein modeling research, a pattern becomes clear. Structure prediction had its breakthrough moment years ago and localization prediction has quietly lagged behind, in part because the problem is genuinely harder in some respects. A protein’s three dimensional fold is, for the most part, a direct physical consequence of its sequence. Where that protein ends up inside a specific cell, at a specific moment, depends on additional context including cell type, cell cycle stage, interaction partners, and the mechanical process of membrane insertion itself, none of which live cleanly inside a single amino acid string.

The authors’ own closing suggestions point toward two plausible paths forward that go beyond simply scaling up existing protein language models further. One is building models that treat protein stability, localization, and interaction data as parts of one shared learning problem rather than as separate benchmarks bolted together after the fact, since their own attempt to bolt on interaction data as a late addition delivered only marginal gains. The other is looking further afield, toward RNA localization and biomolecular condensates, since the authors note that as much as 20% of human proteins are thought to dynamically associate with condensates that concentrate messenger RNA and RNA binding proteins to control where and when translation happens in the first place, meaning some proteins may never sit still long enough for a single discrete compartment label to be the right way to describe them at all.

None of this means today’s tools are useless. ProtT5 MHA and its peers clearly learned real, biologically grounded signal, the attention analysis alone shows the model consistently zeroing in on genuine sorting signals rather than noise. What this paper firmly establishes is where the boundary of that usefulness currently sits, at well annotated, single location, common compartment proteins with intact sorting signals, and not yet at the rare compartments, the multilocalizing proteins, or the disease causing mutations that matter most for actually helping patients. Closing that gap, rather than posting a marginally higher benchmark score, is the challenge the authors are explicitly calling the field to take on next.

Frequently asked questions

Limitations of this article

This article summarizes claims and figures exactly as reported in the source paper and its supplementary materials. Every statistic cited above, including sample sizes, accuracy figures, and percentages, comes directly from the paper’s text, tables, or figure legends rather than from outside sources or estimation. Readers who want to explore the granular per compartment breakdowns, the full confusion matrices, or the supplementary tables referenced throughout should consult the original Registered Report and its extended data directly, since this summary necessarily condenses a fifty five page paper into a shorter overview and cannot capture every nuance of the Methods section.

Related reading