Salad Designs Large Proteins Fast With Sparse Denoising

Why big proteins break the usual generators

Computational protein design aims to produce protein structures with a specified fold, function, or dynamics, and it has become a genuinely useful tool, applied to enzymes, antibodies, vaccine scaffolds, and biosensors. The dominant recipe pairs a generative model that produces a protein backbone with a sequence designer such as ProteinMPNN and a structure predictor such as AlphaFold 2 or ESMFold, which together select which designs are worth making in the lab. Diffusion style generators have taken over the backbone step, and this whole line of work sits alongside other diffusion models for protein structure.

Two problems hold these generators back, and this paper names both. The first is size. Most protein diffusion models borrowed their architecture from structure predictors, which lean heavily on residue pair features and pair attention. Those give strong results but carry a heavy price, pair features that grow with the square of the protein length and pair attention that grows with the cube. Past about 400 amino acids the quality drops off, which is a real limitation, since many interesting molecular machines are larger than that.

The second problem is rigidity. Current models often need extra training to handle a design task they were not built for. Adding a new conditioner, say to scaffold a functional motif, is not straightforward and can require hand crafted energy functions. So every new kind of design tends to mean a new training run, which is slow and forbidding for anyone wanting to prototype an idea quickly.

Making attention sparse

The efficiency fix in salad, which stands for sparse all atom denoising, is to stop attending to every pair of residues. The model is built on a sparse version of invariant point attention, the geometry aware attention introduced in AlphaFold 2, the same building block used in related structure work like base pair conditioned RNA structure generation. Instead of computing the full attention matrix and pair features across all residues, salad builds a small set of neighbours for each amino acid and attends only within it.

That single change is what tames the cost. Full attention grows with the square of the number of residues, while sparse attention over a fixed neighbour set grows only with the number of residues times the neighbours per residue.

The neighbour set is chosen with care rather than at random. For each residue salad picks the 16 nearest by position in the chain, the 16 nearest by physical distance, and 32 random ones, giving 64 neighbours per amino acid, and it recomputes fresh neighbours at every denoising block as the structure moves. It also drops two expensive ingredients other models keep, the persistent pair features and the triangle multiplication that would push the cost back up toward the cube of the length. The result is a model that reasons about global structure through many overlapping local views rather than one giant all to all comparison.

Editing structures instead of retraining

The second idea is the one that makes salad flexible, and it is elegant. A denoising model turns noise into structure over many small steps. salad inserts two edit functions into that loop, one that edits the noisy input before the model sees it and one that edits the model’s output before the next step.

Because these edits run at every step, they can enforce arbitrary structural constraints on the fly, with no change to the trained weights. Want a symmetric protein? Symmetrize the input and the output, so the model is always working on a symmetric structure. Want to embed a functional motif? Replace the coordinates of certain residues with the motif’s coordinates before each step, so the growing backbone is forced to grow around it. The same frozen model that generates ordinary proteins becomes, through a few lines of editing, a motif scaffolder, a symmetry designer, or a multi state designer. This is the plug and play replacement the paper is really selling, and it is what lets a small team prototype design tasks the model never trained on.

Fast, small, and it scales

The efficiency claims are concrete. On a single RTX 3090 GPU, salad outran every model it was compared against on both time per design and time per denoising step, reaching up to sevenfold speedups over the fastest competitor and beating one widely used model by up to two orders of magnitude on large proteins.

Sit with the parameter count. salad produces designable 1000 residue proteins with roughly 8 million parameters, while a strong concurrent model, Proteina, uses about 200 million and does not reach 1000 residues at all. A twenty five fold smaller model that goes further is an unusual result in a field where scaling up is the reflex, and it is the direct payoff of the sparse architecture.

The quality holds up across sizes, with an honest wrinkle the paper is upfront about. Designability is measured as the fraction of generated backbones for which a designed sequence folds back to the target, using a self consistency distance below 2 angstroms and a confidence score above 70. In the 50 to 400 residue range, salad is comparable to the leading diffusion models and beats RFdiffusion, while slightly trailing Genie 2, which was trained on a much larger dataset. Past 400 residues the plain variant produces overly compact backbones and loses designability, but two other noise schedules fix this, holding self consistency below 2 angstroms up to 600 and 800 residues, and a shaped noise trick reaches 36.7 percent designability at a full 1000 residues, matching or improving the previous best for large proteins.

New tasks, no new training

The structure editing idea is validated across a spread of design tasks, and this is where the flexibility stops being a claim. On motif scaffolding, the task of building a protein around a functional fragment, salad was run two ways, one editing a general model to embed the motif and one using a motif conditioned model, and both did well on a standard benchmark.

Both salad approaches solved 23 of 24 single motif tasks and 5 of 6 multi motif tasks, generated more unique scaffolds than Genie 2, and dwarfed RFdiffusion’s 889. The editing route also extends to shapes the base model never saw, generating symmetric proteins by symmetrizing the noise, and repeat proteins with more repeats than any protein diffusion model had managed before, from fully helical to fully sheet topologies.

The most striking demonstration is multi state design, building one protein that can fold into two distinct shapes depending on conditions. Natural examples are rare, training data is scarce, and structure based generators were thought unsuitable for it. By coupling three denoising processes, one per state, salad tackled a benchmark that a previous method had struggled with.

At 2.9 percent full success, salad beat the previous method’s 0.05 percent by a wide margin, and even its weaker sequence level variant reached 0.32 percent, six times the earlier result. The comparison between the two salad rows also makes a clean point, that the structure editing is doing the work, since dropping it and conditioning only on secondary structure fell back to roughly the earlier method’s level. This is, to the authors’ knowledge, the first demonstration of multi state design with a protein backbone denoising model.

A useful side effect, synthetic training data

One result is worth pulling out because it points somewhere bigger. Because salad can be conditioned on random secondary structure to boost diversity, the authors used it to generate a synthetic dataset of 50,000 backbones, of which 81.4 percent were designable. Clustering these against the known protein database, more than 90 percent had no close match, meaning the model was inventing plausible folds that do not appear in nature’s catalogue, what the paper calls dark matter folds.

Training a fresh salad model on that synthetic set improved its one shot designability, letting it produce foldable backbones without the usual sequence design and structure prediction filtering loop, which cut the number of evaluations needed by a factor of eight. A generative model that can bootstrap its own better training data, from folds that do not yet exist, is a compounding kind of useful.

Where it falls short

The authors are candid about the limits, which is the right posture for a computational result. The clearest gap is validation. Every result here is a computational benchmark, with no wet lab experiments. Designability judged by AlphaFold or ESMFold has been shown to select experimentally viable designs, but it is not a perfect proxy, and structure predictors can be fooled by adversarial sequences. The paper argues, reasonably, that since salad was trained independently of ProteinMPNN and the structure predictors, it is unlikely to have learned to generate backbones that fool them, and that its pipeline matches ones already validated experimentally. Still, the honest reading is that these are strong in silico results awaiting laboratory confirmation.

The second limit is the training set. salad was trained on protein structures from the protein database with all small molecules, ions, water, and nucleic acids removed. That makes it less suited to enzyme design or small molecule binder design, where the ligand matters, although the authors note the architecture could likely be extended to include them. It also trails Genie 2 slightly on diversity, which they attribute to Genie 2 being trained on the much larger AlphaFold database rather than to any inherent limit, and suggest the gap could close by training salad on that larger set.

There is also the ordinary caveat of a compact result. The gains in designability at the very largest sizes rest on specific noise schedules and a shaped noise trick, so the headline 1000 residue number is a best case configuration rather than a universal one. None of this undercuts the contribution, which is efficiency and flexibility, but it keeps the scope honest.

Why the approach travels

Beyond proteins, salad carries two ideas that generalize. The first is that sparse, geometry aware attention over well chosen neighbours can replace dense pair attention in a structure generator without losing quality, which is a template for scaling any model that reasons over 3D structure, from other biomolecules to materials. The efficiency is not a trick specific to proteins, it is a general answer to the quadratic cost of comparing everything to everything.

The second is that editing a generative process step by step is a powerful alternative to retraining it. Rather than build a new conditioner and train it, you constrain the sampling loop directly, which turns one model into many. That pattern is showing up across generative science, from editing existing scaffolds to optimize peptide antibiotics to steering diffusion toward novel materials, and salad is a clean demonstration of it for protein backbones. Its specific contribution is to pair a genuinely efficient sparse architecture with a plug and play editing strategy, and to show that the combination reaches larger proteins and more design tasks than the heavier models it replaces.

Reference implementation in PyTorch

The code below is a runnable reconstruction of salad’s two core ideas, the sparse neighbour selection that makes attention cheap and the structure editing wrapper that lets one model take on new tasks, based on the paper’s description. A compact geometry aware attention block stands in for the full sparse invariant point attention, and a simple symmetrization edit shows how editing enforces a constraint at every denoising step. A smoke test runs a few denoising steps on a dummy backbone. Swap in the full attention block and trained weights for actual experiments.

# salad_reference.py
# Sparse neighbour selection and structure editing for protein denoising.
# Replace SparseBlock with full sparse invariant point attention for real runs.

import torch
import torch.nn as nn


def select_neighbours(x, k_seq=16, k_dist=16, k_rand=32):
    """Per residue neighbours: nearest by index, nearest by distance, random."""
    B, N, _ = x.shape
    idx = torch.arange(N, device=x.device)
    # nearest by position in the chain
    seq_d = (idx[:, None] - idx[None, :]).abs().float()
    seq_nn = seq_d.topk(k_seq, largest=False).indices        # (N, k_seq)
    # nearest by physical distance between CA atoms
    d = torch.cdist(x, x)                                    # (B, N, N)
    dist_nn = d.topk(k_dist, largest=False).indices        # (B, N, k_dist)
    # random neighbours, fresh each call
    rand_nn = torch.randint(0, N, (B, N, k_rand), device=x.device)
    seq_nn = seq_nn.unsqueeze(0).expand(B, -1, -1)
    return torch.cat([seq_nn, dist_nn, rand_nn], dim=-1)     # (B, N, K)


class SparseBlock(nn.Module):
    """Stand in for sparse invariant point attention over K neighbours."""
    def __init__(self, d=128):
        super().__init__()
        self.q = nn.Linear(d, d)
        self.k = nn.Linear(d, d)
        self.v = nn.Linear(d, d)
        self.pos = nn.Linear(d, 3)                          # position update
        self.norm = nn.LayerNorm(d)                          # pre normalization

    def forward(self, h, x, nb):
        B, N, d = h.shape
        K = nb.shape[-1]
        hn = self.norm(h)
        bi = torch.arange(B, device=h.device).view(B, 1, 1)
        hk = hn[bi, nb]                                      # (B, N, K, d) neighbour features
        q = self.q(hn).unsqueeze(2)                        # (B, N, 1, d)
        att = (q * self.k(hk)).sum(-1) / d ** 0.5          # (B, N, K) sparse scores
        att = att.softmax(-1).unsqueeze(-1)
        agg = (att * self.v(hk)).sum(2)                     # (B, N, d)
        h = h + agg
        x = x + self.pos(self.norm(h))                       # equivariant style position update
        return h, x


def symmetrize(x, folds=3):
    """Edit function: average repeat units to enforce cyclic symmetry."""
    B, N, _ = x.shape
    unit = N // folds
    x = x[:, :unit * folds].view(B, folds, unit, 3)
    mean_unit = x.mean(1, keepdim=True)                    # shared repeat unit
    ang = torch.arange(folds, device=x.device) * (2 * torch.pi / folds)
    rot = torch.stack([torch.stack([ang.cos(), -ang.sin(), torch.zeros_like(ang)], -1),
                       torch.stack([ang.sin(), ang.cos(), torch.zeros_like(ang)], -1),
                       torch.stack([torch.zeros_like(ang), torch.zeros_like(ang),
                                    torch.ones_like(ang)], -1)], -2)   # (folds, 3, 3)
    placed = torch.einsum("fij,bkuj->bfui", rot, mean_unit.expand(B, folds, unit, 3))
    return placed.reshape(B, unit * folds, 3)


class SaladDenoiser(nn.Module):
    def __init__(self, d=128, blocks=6):
        super().__init__()
        self.embed = nn.Linear(3, d)
        self.blocks = nn.ModuleList([SparseBlock(d) for _ in range(blocks)])
        self.head = nn.Linear(d, 3)

    def forward(self, x):
        h = self.embed(x)
        for blk in self.blocks:
            nb = select_neighbours(x)                        # fresh neighbours per block
            h, x = blk(h, x, nb)
        return x + self.head(h)                              # denoised coordinates


def sample(model, n=60, steps=10, edit=None):
    """Denoise from noise, applying an edit function each step, Eq 1."""
    x = torch.randn(1, n, 3) * 10.0
    for t in range(steps):
        if edit is not None:
            x = edit(x)                                      # edit_input
        x0 = model(x)
        if edit is not None:
            x0 = edit(x0)                                    # edit_output
        noise = (steps - t - 1) / steps
        x = x0 + noise * torch.randn_like(x0) * 10.0       # re add scaled noise
    return x


if __name__ == "__main__":
    model = SaladDenoiser()
    plain = sample(model, n=60)
    symm = sample(model, n=60, edit=lambda x: symmetrize(x, folds=3))
    print("plain backbone", tuple(plain.shape))
    print("symmetric backbone", tuple(symm.shape))
    npar = sum(p.numel() for p in model.parameters())
    print("parameters", round(npar / 1e6, 2), "M")

Conclusion

The core achievement of salad is to make protein backbone generation both cheaper and more flexible without giving up quality. By swapping dense pair attention for sparse attention over chosen neighbours, it generates designable proteins up to 1000 amino acids, a length where other diffusion models stall, and does so in seconds with a model a fraction of the size of its competitors. On the standard benchmarks it matches or beats the leading generators on designability and diversity while running far faster.

The conceptual contribution worth keeping is that you can adapt a generative model to new tasks by editing its process rather than retraining its weights. The structure editing loop turns one frozen denoiser into a motif scaffolder, a symmetry designer, and a multi state designer, and it delivered the first multi state design from a protein backbone denoising model, beating the prior best by a wide margin. That plug and play flexibility is what makes salad useful as a component in existing pipelines rather than yet another standalone model.

The efficiency lesson travels well beyond proteins. Sparse geometry aware attention over well chosen neighbours is a general answer to the quadratic cost of structure generators, and step by step editing is a general alternative to building and training new conditioners. Both ideas would carry to other biomolecules and to materials, and the synthetic data result hints at a further loop, where a model efficient enough to run at scale can generate its own better, more novel training data.

The honest limitations keep it grounded. The results are computational, without wet lab validation, the training set excludes the small molecules and nucleic acids that many real design tasks involve, and the largest size results lean on specific noise schedules. The authors name all of this plainly, and they are explicit that salad is meant to be an efficient, versatile backbone generator to slot into a design pipeline, not a finished all in one solution.

For anyone building generative models over structure, the practical message is compact. Do not attend to everything when a handful of neighbours will do, and do not retrain when you can edit the sampling loop instead. salad shows the combination reaches larger proteins and more design tasks than heavier models, on a fraction of the compute, and it ships as open code and parameters, so the ideas are there to build on.

Frequently asked questions

What is protein structure generation?

It is using a machine learning model to design a protein’s 3D backbone with a wanted shape or function, rather than searching for one in nature. The generated backbone is usually paired with a sequence designer such as ProteinMPNN and a structure predictor such as AlphaFold to pick which designs are worth making and testing in the lab.

What makes salad different from other protein generators?

salad replaces the dense pair attention that most protein diffusion models use with a sparse version that attends only to a fixed set of neighbours per residue, which cuts the computational cost from quadratic to near linear. This lets it generate proteins up to 1000 amino acids, well past the roughly 400 residue point where other models start to fail, and it runs in seconds with far fewer parameters.

What is structure editing and why does it matter?

Structure editing inserts two functions into the denoising loop, one that edits the noisy input and one that edits the model’s output at every step. Because these run during sampling, they can enforce constraints like symmetry or an embedded motif without changing the trained model. That lets one frozen model take on new design tasks, such as motif scaffolding, repeat proteins, and multi state design, without retraining.

How fast and how large can salad go?

On a single RTX 3090 GPU, salad generates a 1000 residue backbone in about 19 seconds, against more than 10 minutes for RFdiffusion, and it uses roughly 8 million parameters for designable 1000 residue proteins where a comparable concurrent model uses about 200 million and does not reach that length. With a shaped noise trick it reaches 36.7 percent designability at 1000 residues.

Has salad been tested in the lab?

No. All of the results are computational benchmarks using structure predictors to judge designability, with no wet lab experiments. The authors argue that because salad was trained independently of the sequence designer and structure predictors, and matches pipelines validated experimentally elsewhere, its designs are likely viable, but experimental confirmation is still needed.

Is the code available?

Yes. The work is a peer reviewed article in Nature Machine Intelligence, and the authors released the model code under an open license on GitHub along with the trained parameters, so the results can be reproduced and the models reused in other protein design pipelines.