What Protein Language Models Have Taught Me in a Year of Using Them

Quick history of how I got here

I started using protein language models in 2022, mostly out of curiosity, as a waystation between molecular property prediction and structure-conditioned featurization. By 2024 we had ESM-3 landing, and by mid-2025 every project I touched had at least one protein LM in the loop. This post is a retrospective on that year.

What I expected vs. what I got

Going into 2025 I expected two things from protein LMs:

1. That they would mostly replace hand-engineered features for protein-engineering tasks. Mostly true.
2. That they would crush sequence-only structure predictors like Rosetta and trRosetta. True, but not in the way I expected — they didn't beat them, they removed them from the toolkit.

What I did not expect:

• How catastrophic the inference-engineering gap turned out to be. ESM-2 650M inference on a single H100 is one thing; ESM-2 650M on a CPU is another thing entirely. We're 80% of our time in deployment, not training.
• That concatenated ESM-2 logits + a single linear head would still beat fine-tuned ESM-2 on small downstream tasks. Less surprising in hindsight, but I expected fine-tuning to win by more than it did.

ESM-2 in production

Our most-deployed model is still ESM-2 with the 650M variant. We use it for:

• Variant-effect scoring, as a frozen feature extractor feeding a tiny MLP head per assay. Surprisingly, the best head was a logistic regression with one hidden layer of 16 units. That's it.
• Log-likelihood ratio for protein engineering — natural log probability ratio of mutant vs. wild-type. We've used it to triage CRISPR screens of cancer targets.
• Embedding-space nearest neighbors as a quick duplicate detector on incoming sequences. FAISS over ESM-2 logits > sequence identity checks for a lot of wet-lab pipelines.

# Daily-driver ESM-2 loader
import torch, esm

model, alphabet = esm.pretrained.esm2_t33_650M_UR50D()
batch_converter = alphabet.get_batch_converter()
model.eval().cuda()

def embed(seq: str) -> torch.Tensor:
    # Returns mean-pooled per-residue logits, length 1280
    data = [("q", seq)]
    _, _, tokens = batch_converter(data)
    tokens = tokens.cuda()
    with torch.no_grad():
        out = model(tokens, repr_layers=[33], return_contacts=False)
    h = out["representations"][33]
    return h[<0, 1:1 + len(seq)].mean(0)

Structure predictors: where AlphaFold 3 won, where Boltz didn't

ESMFold is fast. We run 200 mini-proteins through it in an evening. But the precision bar is set by AlphaFold 3, which consistently beats ESMFold on multi-chain assemblies and on anything with bound cofactors.

We tried replacing AF3 with Boltz in two projects in 2025. Lessons:

1. Single-chain protein-ligand complexes: Boltz ~ AF3 within pLDDT error. Tied.
2. Antibody-antigen complexes: AF3 noticeably better. Boltz struggles with the CDRs.
3. Protein-RNA complexes: Boltz > AF3 for some families, about equal for others. We kept both.

The takeaway I share with collaborators: pick AF3 if you can afford the throughput. Pick Boltz if you need a hundred structures in an afternoon.

The "ESM-3 + downstream tasks" story, briefly

ESM-3 (Hayes et al., 2024) introduced a multi-track architecture: sequence, structure, and function tokens. We picked up the pretrained checkpoint in early 2025 and tried to use it as a drop-in replacement for ESM-2.

Honest result: for sequence-only inference, ESM-3 is no improvement over ESM-2 once you include the structure track. For tasks where you actually have a structure available (maybe from a homology model), ESM-3 wins handily. For tasks where you don't, don't pay the throughput cost.

Things I'm still trying to figure out

Three open questions in the lab right now:

1. How much protein engineering benefit comes from the pretraining objective vs. the scale? Ablation looks hard because you can't cleanly separate them in the released checkpoints.
2. Are PLM confidence scores transformable into probabilities for property prediction? We have a small project on post-hoc calibration. Results so far: barely, and only with extra data per domain.
3. What's the analogue of "prompt engineering" for proteins? Embedding-context concatenation works surprisingly well. Sequence-level magic prompts, less so.

What I'd recommend to a new lab

1. Start with ESM-2 650M logits + a tiny per-task head. Stop when that's no longer competitive.
2. If you're predicting complexes, run AF3 in parallel with a baseline (e.g. AF2-multimer). Don't trust the confident score, trust the difference between the two.
3. If you need throughput on single proteins, try Boltz-1x or ESMFold. They are not state-of-the-art on benchmarks; they're state-of-the-art on practical throughput for 2025 budgets.
4. Treat PLM internals as opaque for now. The attention-head-interpretation literature is half of the field right now, but interpretability hooks for protein LMs are still loose.