Machine Learning for Ranking in Search Systems: A Scoping Review

The exponential growth of digital information has made machine learning (ML)-based ranking a crucial mechanism for modern search systems, enabling them to manage complex queries and deliver highly relevant results across various domains. This scoping review aims to systematically characterize the current state of machine learning for ranking in search systems. Following PRISMA-ScR guidelines, a comprehensive literature search was conducted across the IEEE Xplore and Lens.org databases for peer-reviewed studies published between 2023 and 2026, selecting 78 eligible papers for data extraction and thematic synthesis. Key findings reveal a clear paradigm shift, with transformer-based architectures and Large Language Models (LLMs) dominating the recent literature, particularly at the re-ranking stage. In contrast, neural ranking and classical learning-to-rank (LTR) approaches remain less frequent. Evaluation mostly relies on standardized public benchmarks like MS MARCO and BEIR, utilizing offline ranking metrics (e.g., nDCG, MRR), though efficiency and generation quality metrics are increasingly reported for LLMs. General and web search remain the primary application domains, along with emerging generative search pipelines. Critical challenges persist regarding high inference costs, latency, domain generalization, data bias, and the "black-box" nature of these models. This review provides significant value for both science and practice by mapping the rapid structural evolution of the field, establishing a clear taxonomy of current methodologies, and highlighting critical research gaps. For practitioners and researchers alike, it offers a foundational roadmap for addressing scalability and interpretability constraints, guiding the future development of efficient, transparent ranking models for production-ready search systems.