Bouneffouf, Djallel, Feraud, Raphael · ArXiv.org · 2025
Bandit algorithms and Large Language Models (LLMs) have emerged as powerful tools in artificial intelligence, each addressing distinct yet complementary challenges in decision-making and natural language processing. This survey explores the synergistic potential between these two fields, highlighting how bandit algorithms can enhance the performance of LLMs and how LLMs, in turn, can provide novel insights for improving bandit-based decision-making. We first examine the role of bandit algorithms in optimizing LLM fine-tuning, prompt engineering, and adaptive response generation, focusing on their ability to balance exploration and exploitation in large-scale learning tasks. Subsequently, we explore how LLMs can augment bandit algorithms through advanced contextual understanding, dynamic adaptation, and improved policy selection using natural language reasoning. By providing a comprehensive review of existing research and identifying key challenges and opportunities, this survey aims to bridge the gap between bandit algorithms and LLMs, paving the way for innovative applications and interdisciplinary research in AI.
Cited by 0Open access
Djallel Bouneffouf, Raphaël Féraud · Proceedings of the AAAI Conference on Artificial Intelligence · 2026
Bandit algorithms and Large Language Models (LLMs) have emerged as powerful tools in artificial intelligence, each addressing distinct yet complementary challenges in decision-making and natural language processing. This survey explores the synergistic potential between these two fields, highlighting how bandit algorithms can enhance the performance of LLMs and how LLMs, in turn, can provide novel insights for improving bandit-based decision-making. We first examine the role of bandit algorithms in optimizing LLM fine-tuning, prompt engineering, and adaptive response generation, focusing on their ability to balance exploration and exploitation in large-scale learning tasks. Subsequently, we explore how LLMs can augment bandit algorithms through advanced contextual understanding, dynamic adaptation, and improved policy selection using natural language reasoning. By providing a comprehensive review of existing research and identifying key challenges and opportunities, this survey aims to bridge the gap between bandit algorithms and LLMs, paving the way for innovative applications and interdisciplinary research in AI.
Cited by 0Open access
Sai Zhang · Applied and Computational Engineering · 2024
Nowadays, with the development of the Internet, recommending personalized information for every user becomes economically valuable. How to properly recommend to users the content they prefer is important. Mostly, bandit strategies are useful for this problem. As one of the most time-saving algorithms, Minimax Optimal Strategy (MOSS) performs well in most data sets. However, the traditional MOSS algorithm applies a conservative strategy in order to minimize the cumulative regrets in the worst situations. This strategy will over-explore the bandit models and spend too much time exploring every arm at the initial stage. This means the MOSS strategy cannot fully take advantage of the known information and then the accuracy and efficiency will be lower. In real life, people have a lot of Internet information to look through and they will not stay on one website or one advertisement for a long time. So, designing an efficient algorithm that performs well in all situations and can make good use of the known knowledge to do quick exploitation becomes so important. Therefore, this research aims to develop an advanced MOSS algorithm that can properly explore each of the bandit arms and improve the performance of the exploitation phase. By achieving this goal, this research adds an enhancement factor on the confidence bound part to speed up the exploration process. The research bases on the MovieLens data set and the data set is divided into 18 categories (arms) according to its genre attribute. Then apply this new advanced MOSS strategy to the designed data set and compare its performance with normal MOSS and UCB algorithms. Results display that the advanced MOSS strategys selection provides higher total rewards and lower average cumulative regrets than others with the advantage of time-saving. This shows the improvement of the advanced MOSS strategy in balancing the exploitation and exploration phases.
Cited by 0Open access
Yilin Ge · Applied and Computational Engineering · 2024
In the rapidly evolving financial markets, traditional investment strategies are increasingly being replaced by advanced algorithms. Among these, the Upper Confidence Bound (UCB) and TS algorithms, which originate from the multi-armed bandit framework, have demonstrated significant potential across various sectors. This study utilizes historical stock data from companies during the period from 2019 to 2024 to evaluate the performance of these algorithms in long-term investment strategies. The research aims to identify how these algorithms manage investment risks and maximize returns under varying market conditions. The results reveal distinct behaviors of UCB and TS in the financial markets: UCB performs better in markets with lower volatility, while TS exhibits stronger adaptability in high-volatility environments, thereby enhancing potential returns. Through a detailed comparison of UCB and TS in long-term investment scenarios, this study provides valuable insights into the strategic application of these algorithms. It offers investment strategy guidance for investors and financial strategists and supplies effective information for economic decision-making in relevant fields.
Cited by 0Open access
Konrad-Felix Krentz, Marius Corici · 2021
The tunneling protocol WireGuard outperforms its main competitors OpenVPN and IPsec in terms of throughput and latencies. These improvements are due to WireGuard's use of faster crypto primitives, as well as to the implementation of WireGuard as a Linux kernel module that uses multithreading and advanced locking strategies. Independently of the WireGuard project, Lukaszewski et al. demonstrated improvements in end-to-end goodput when tunneling protocols exploit alternative communication paths. In this poster, we combine these two research directions by proposing multipath extensions for WireGuard. Our extensions involve additions to the WireGuard header, which enable obtaining real-time statistics on the performance of each path. Further, these real-time path performance statistics enable a self-adaptive selection of paths. As a proof of concept, we adapted the WireGuard Linux kernel module accordingly and prototyped four example path schedulers, two of which adopt multi-armed bandit algorithms.
Cited by 7
Jingyu Li · ITM Web of Conferences · 2025
In the technological revolution of the 21st century, autonomous driving technology is rapidly changing transportation modes, and path planning, as a key component, relies heavily on advanced algorithm optimization. The Multi-Arm Bandit (MAB) algorithm may become an efficient decision optimization tool in autonomous driving path planning. Because it can continuously experiment, learn, and quickly determine the optimal strategy to maximize profits under resource constraints. When applied to autonomous driving, the MAB algorithm may be able to demonstrate its advantages. In complex traffic environments, it dynamically adjusts strategies to adapt to constantly changing road conditions, plans safe and efficient driving paths, and quickly responds to unexpected situations to ensure driving safety. Compared with other algorithms, the learning and adaptability of MAB algorithm makes it particularly suitable for the dynamics and unpredictability of real-world driving scenarios. However, the practical application of MAB algorithm in autonomous driving faces challenges, including accurately evaluating path efficiency, efficiently processing large amounts of traffic data, and ensuring the stability and reliability of the algorithm. Further in-depth research and exploration are crucial for fully utilizing the advantages of MAB algorithm in path planning and promoting the sustainable development and enhancement of autonomous driving technology.
Cited by 1Open access
Sherief Hashima, Ahmad Gendia, Kohei Hatano et al. · IEEE Open Journal of Vehicular Technology · 2025
The convergence of sixth-generation (6G) networks with unmanned aerial vehicles (UAVs) and satellites is poised to introduce substantial improvements to the landscape of wireless communication, paving the way for a unified and uninterrupted space-air-ground-sea network that ensures comprehensive global connectivity. At the heart of this transformative paradigm lies artificial intelligence (AI), which drives innovation across diverse sectors by enhancing decision-making autonomy, enabling real-time data processing, and optimizing network performance and coverage. This survey paper explores AI-enabled UAV-satellite communications for 6G applications, focusing on its challenges, potential, and future. This new system combines the strengths of 6G networks, UAVs (advanced drones), and satellites. It opens up new possibilities in precision agriculture, disaster management, enhanced telecommunication services, and remote sensing. Despite its promise, this field faces complex challenges. These include spectrum management, security risks, regulatory barriers, and integrating AI operations seamlessly. This paper comprehensively analyzes these challenges, offering innovative solutions and outlining future research directions to unlock the complete capabilities of 6G-enabled UAV-satellite communications. Furthermore, it includes a case study demonstrating the effectiveness of multi-armed bandit (MAB) algorithms in optimizing resource allocation and decision-making processes for UAV-low Earth orbit (LEO) satellite communication scenarios, showcasing significant improvements in network performance. This work lays the foundation for a new generation of ultra-connected, data-driven applications that will redefine global connectivity and technological advancement by addressing these critical aspects.
Cited by 9Open access
Xiang Zhang, Keying Wu, Chengqi Zhang et al. · Biomedical Signal Processing and Control · 2025
Cited by 7
Kai Wang, Lily Xu, Aparna Taneja et al. · Proceedings of the AAAI Conference on Artificial Intelligence · 2023
Restless multi-armed bandits (RMABs) extend multi-armed bandits to allow for stateful arms, where the state of each arm evolves restlessly with different transitions depending on whether that arm is pulled. Solving RMABs requires information on transition dynamics, which are often unknown upfront. To plan in RMAB settings with unknown transitions, we propose the first online learning algorithm based on the Whittle index policy, using an upper confidence bound (UCB) approach to learn transition dynamics. Specifically, we estimate confidence bounds of the transition probabilities and formulate a bilinear program to compute optimistic Whittle indices using these estimates. Our algorithm, UCWhittle, achieves sublinear O(H \sqrt{T log T}) frequentist regret to solve RMABs with unknown transitions in T episodes with a constant horizon H. Empirically, we demonstrate that UCWhittle leverages the structure of RMABs and the Whittle index policy solution to achieve better performance than existing online learning baselines across three domains, including one constructed from a real-world maternal and childcare dataset.
Cited by 8Open access
Jose A. Ayala‐Romero, Andrés García‐Saavedra, Xavier Costa‐Pérez · Proceedings of the AAAI Conference on Artificial Intelligence · 2024
Recent advances in learning techniques have garnered attention for their applicability to a diverse range of real-world sequential decision-making problems. Yet, many practical applications have critical constraints for operation in real environments. Most learning solutions often neglect the risk of failing to meet these constraints, hindering their implementation in real-world contexts. In this paper, we propose a risk-aware decision-making framework for contextual bandit problems, accommodating constraints and continuous action spaces. Our approach employs an actor multi-critic architecture, with each critic characterizing the distribution of performance and constraint metrics. Our framework is designed to cater to various risk levels, effectively balancing constraint satisfaction against performance. To demonstrate the effectiveness of our approach, we first compare it against state-of-the-art baseline methods in a synthetic environment, highlighting the impact of intrinsic environmental noise across different risk configurations. Finally, we evaluate our framework in a real-world use case involving a 5G mobile network where only our approach satisfies consistently the system constraint (a signal processing reliability target) with a small performance toll (8.5% increase in power consumption).
Cited by 1Open access
Manoharan Aswath, Stamberger Jeannie, Yuanyuan Yu et al. · Figshare · 2011
For each of the 104 observedbird species the charts compare the number of questions required for with the number of questions required by one ofseven algorithm settings: 2000–2005, and 2005_4. Bars above the zeroline indicate that the biased runs required more questions than . Bars below zero indicate that outperformed . The ordinate scale is number of questions required beyond (positive bars) or saved against (negative bars). The top panel compares against the worst performing biased setting for each species. The middle panel compares against the best performing setting, and the bottom panel compares against the mean of all seven biased settings for each species. Note that the panels have different ordinate scale ranges (5→-4, 1→-5, and 3→-5).<b>Copyright information:</b>Taken from "Optimizations for the EcoPod field identification tool"http://www.biomedcentral.com/1471-2105/9/150BMC Bioinformatics 2008;9():150-150.Published online 17 Mar 2008PMCID:PMC2322985.
Cited by 0Open access
Wei Qian, Ching‐Kang Ing, Ji Liu · Figshare · 2023
This article studies an important sequential decision making problem known as the multi-armed stochastic bandit problem with covariates. Under a linear bandit framework with high-dimensional covariates, we propose a general multi-stage arm allocation algorithm that integrates both arm elimination and randomized assignment strategies. By employing a class of high-dimensional regression methods for coefficient estimation, the proposed algorithm is shown to have near optimal finite-time regret performance under a new study scope that requires neither a margin condition nor a reward gap condition for competitive arms. Based on the synergistically verified benefit of the margin, our algorithm exhibits adaptive performance that automatically adapts to the margin and gap conditions, and attains optimal regret rates simultaneously for both study scopes, without or with the margin, up to a logarithmic factor. Besides the desirable regret performance, the proposed algorithm simultaneously generates useful coefficient estimation output for competitive arms and is shown to achieve both estimation consistency and variable selection consistency. Promising empirical performance is demonstrated through extensive simulation and two real data evaluation examples. Supplementary materials for this article are available online.
Cited by 0Open access
Angeliki Giannou, Emmanouil Vasileios Vlatakis-Gkaragkounis, Panayotis Mertikopoulos · HAL (Le Centre pour la Communication Scientifique Directe) · 2021
International audience
Cited by 1Open access
Álvaro Labarca Silva · 2024
En los ultimos años, el Aprendizaje Reforzado (RL por sus siglas en inglés) ha mostrado un gran potencial en recomendaciones basadas en sesión. Modelos secuenciales que usan RL han alcanzado resultados de estado del arte en la tarea de predicción del siguiente ítem (NIP por sus siglas en inglés). Este resultado es intrigante, ya que la tarea NIP solo evalúa qué tan bien el sistema puede correctamente recomendar el siguiente ítem al usuario, mientras que el objetivo de RL es encontrar una política que optimiza la recompensa en el largo plazo - a veces a costa de un desempeño a corto plazo sub-óptimo. Entonces, ¿Cómo puede RL mejorar el desempeño del sistema en métricas de corto plazo? Este artículo investiga esta pregunta explorando con objetivos de aprendizaje a través de un proxy, que identificamos como objetivos que los modelos de RL podrían estar siguiendo y de esta forma explicar la ganancia en desempeño. Encontramos que RL - al ser usado como pérdida auxiliar - promueve el aprendizaje de embeddings que capturan información acerca de ítems con los que el usuario interactuó previamente. Luego, reemplazamos el objetivo RL con una pérdida auxiliar directa diseñada para predecir el número de ítems con los que el usuario ha interactuado. Esta substitución resulta en una mejora de rendimiento comparable a la de RL. Estos resultados abren el camino para mejorar el desempeño y
Cited by 0
Julien Aubert, Louis Köhler, Luc Lehéricy et al. · ArXiv.org · 2025
Learning for animals or humans is the process that leads to behaviors better adapted to the environment. This process highly depends on the individual that learns and is usually observed only through the individual's actions. This article presents ways to use this individual behavioral data to find the model that best explains how the individual learns. We propose two model selection methods: a general hold-out procedure and an AIC-type criterion, both adapted to non-stationary dependent data. We provide theoretical error bounds for these methods that are close to those of the standard i.i.d. case. To compare these approaches, we apply them to contextual bandit models and illustrate their use on both synthetic and experimental learning data in a human categorization task.
Cited by 0Open access
Shang Lu, Kohei Hatano, Shuji Kijima et al. · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · 2025
This work introduces the dueling dice problem, which is a variant of the multi-armed dueling bandit problem. A die is a set of m arms in this problem, and the goal is to find the best set of m arms from n arms (m ≤ n) by an iteration of dueling dice. In a round, the learner arbitrarily chooses two dice α ⊆ [n] and β ⊆ [n] and lets them duel, where she roles dice α and β, observes a pair of arms i ∈ α and j ∈ β, and receives a probabilistic result Xi,j ∈ {0,1}. This paper investigates the sample complexity of an identification of the Condorcet winner die, and gives an upper bound O(nh-2(log log h-1 + log nm2γ-1)m log m) where h is a gap parameter and γ is an error parameter. Our problem is closely related to the dueling teams problem by Cohen et al. 2021. We assume a total order of the strength over arms similarly to Cohen et al. 2021, which ensures the existence of the Condorcet winner die, but we do not assume a total order of the strength over dice unlike Cohen et al. 2021.
Cited by 0Open access
Lin Yang, Mohammad Hajiesmaili, Sadegh Talebi et al. · Research at the University of Copenhagen (University of Copenhagen) · 2020
This paper studies adversarial bandits with corruptions.In the basic adversarial bandit setting, the reward of arms is predetermined by an adversary who is oblivious to the learner's policy.In this paper, we consider an extended setting in which an attacker sits in-between the environment and the learner, and is endowed with a limited budget to corrupt the reward of the selected arm.We have two main results.First, we derive a lower bound on the regret of any bandit algorithm that is aware of the budget of the attacker.Also, for budget-agnostic algorithms, we characterize an impossibility result demonstrating that even when the attacker has a sublinear budget, i.e., a budget growing sublinearly with time horizon T , they fail to achieve a sublinear regret.Second, we propose , a bandit algorithm that incorporates a biased estimator and a robustness parameter to deal with corruption.We characterize the regret of and show that for the case of a known corruption budget, the regret of is tight.
Cited by 0Open access
Lütfi Kerem Senel, Besnik Fetahu, Davis Yoshida et al. · 2024
Lütfi Kerem Senel, Besnik Fetahu, Davis Yoshida, Zhiyu Chen, Giuseppe Castellucci, Nikhita Vedula, Jason Ingyu Choi, Shervin Malmasi. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.
Cited by 3Open access
Yue Wu, Mingyu Zhao, Ziyu Lyu et al. · SSRN Electronic Journal · 2024
Cited by 0Open access
Tong Zhang · Cambridge University Press eBooks · 2023
In the standard multiarmed bandit problem, one observes a fixed number of arms. To achieve optimal regret bounds, one estimates confidence intervals of the arms by counting. In the contextual bandit problem, one observes side information for each arm, which can be used as features for more accurate confidence interval estimation. This chapter studies contextual bandit problems with both linear and nonlinear models
Cited by 0