Generative modeling

Language models matter less than search algorithms for difficult protein design tasks

Diego del Alamo

20 Mar 2026 — 1 min read

Two papers from the last week show how search algorithms improve the performance of deep learning-based protein design when design restraints are available. The first compared AbLang-2, ESM-2, and other models, finding middling success rates and low variance when used as-is^[1]. However, a noticeable performance boost was observed when augmenting AbLang-2 with beam search:

Pasted image 20260316152952.png

The second, focused on structure-based generative modeling, finds that diverse search algorithms speed up the rate at which diverse solutions are sampled in hard binder design problems^[2]:

Pasted image 20260318080351.png

In both cases, the implementations being tested - Gibbs sampling, beam search, Monte Carlo tree search, Feynman-Kac steering - rely on complete rollouts of the design process. A takeaway is that the base model being used probably matters much less than the inclusion (and composition) of potentials, as well as the choice of search algorithms.

References

McCarter, C., Bhattacharya, N., Ober, S. W., & Elliott, H. (2026). How to make the most of your masked language model for protein engineering (Version 1). arXiv. https://doi.org/10.48550/ARXIV.2603.10302 ↩︎
Didi, K., Zhang, Z., Zhou, G., Reidenbach, D., Cao, Z., Cha, S., Geffner, T., Dallago, C., Tang, J., Bronstein, M. M., Steinegger, M., Kucukbenli, E., Vahdat, A., & Kreis, K. (2026). Scaling atomistic protein binder design with generative pretraining and test-time compute. In The Fourteenth International Conference on Learning Representations. https://openreview.net/forum?id=qmCpJtFZra ↩︎

Flow matching and diffusion perform comparably on biomolecular structure prediction

Flow matching and diffusion perform comparably on biomolecular structure prediction[1]. This is, to my knowledge, the only head-to-head comparison of these two approaches for protein structure modeling or design. References 1. Gong, C., Chen, X., Zhang, Y., Song, Y., Zhou, H., & Xiao, W. (2025). Protenix-Mini: Efficient Structure Predictor

Not all high-fitness sequences have plausible evolutionary paths from lower-fitness starting points via sequential introduction of mutations

I first saw this precise idea articulated by Weinreich et al. in their aptly named "Darwinian evolution can follow only very few mutational paths to fitter proteins"[1]. Many mutation combinations don't have additive effects on every aspect governing protein fitness (expression, stability, function, etc.), and

Conformational entropy could still matter in miniprotein binder design

Antibody V-regions improve their affinity for targets by both creating more high-energy interactions and reducing the conformational entropy of their antigen-binding loops[1][2]. Entropy's importance in antibody-antigen affinity seems obvious, given that loop residues largely mediate binding[3][4]. But for de novo-designed miniprotein binders, which often

Glutamate- and lysine-rich designs are susceptible to expression failure resulting from adenosine-rich sequences

High rates of glutamate and lysine introduction are a staple of structure-based sequence design by ProteinMPNN and related models, regardless of who trains them[1]. Recently, an analysis of the Bits in Bio competition showed that high glutamate/lysine content is predictive of expression failure[2]: The authors traced this

References

Read more

Flow matching and diffusion perform comparably on biomolecular structure prediction

Not all high-fitness sequences have plausible evolutionary paths from lower-fitness starting points via sequential introduction of mutations

Conformational entropy could still matter in miniprotein binder design

Glutamate- and lysine-rich designs are susceptible to expression failure resulting from adenosine-rich sequences