Search 250 million research papers

Reinforcement Learning in Robotics: A Survey

Jens Kober, Jan Peters · Adaptation, learning, and optimization · 2012

Cited by 144

Reinforcement Learning Approaches in Social Robotics

Neziha Akalın · MDPI (MDPI AG) · 2021

This article surveys reinforcement learning approaches in social robotics. Reinforcement learning is a framework for decision-making problems in which an agent interacts through trial-and-error with its environment to discover an optimal behavior. Since interaction is a key component in both reinforcement learning and social robotics, it can be a well-suited approach for real-world interactions with physically embodied social robots. The scope of the paper is focused particularly on studies that include social physical robots and real-world human-robot interactions with users. We present a thorough analysis of reinforcement learning approaches in social robotics. In addition to a survey, we categorize existent reinforcement learning approaches based on the used method and the design of the reward mechanisms. Moreover, since communication capability is a prominent feature of social robots, we discuss and group the papers based on the communication medium used for reward formulation. Considering the importance of designing the reward function, we also provide a categorization of the papers based on the nature of the reward. This categorization includes three major themes: interactive reinforcement learning, intrinsically motivated methods, and task performance-driven methods. The benefits and challenges of reinforcement learning in social robotics, evaluation methods of the papers regarding whether or not they use subjective and algorithmic measures, a discussion in the view of real-world reinforcement learning challenges and proposed solutions, the points that remain to be explored, including the approaches that have thus far received less attention is also given in the paper. Thus, this paper aims to become a starting point for researchers interested in using and applying reinforcement learning methods in this particular research field.

Cited by 125Open access

Reinforcement Learning in Robotics: A Survey

Kober, Jens, J. Andrew Bagnell, Peters, Jan · KiltHub Repository · 2018

Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between model based and model-free as well as between value function-based and policy search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research.

Cited by 0Open access

Modulartransfer reinforcement learning in industrial robotics

Bitter, Christian · Bergische Universität Wuppertal · 2025

Diese Dissertation trägt wesentlich zum Bereich des modularen Transfer Reinforcement Learning (TRL) im Kontext der Industrierobotik bei. Sie befasst sich systematisch mit der Problematik der Dateneffizienz und Transferierbarkeit von Deep Reinforcement Learning (DRL) in der Industrierobotik. Da monolithische end-to-end DRL-Ansätze hohe Datenanforderungen haben und es an Interpretierbarkeit und Übertragbarkeit mangelt, wird in der Dissertation eine modulare Entscheidungspipeline vorgeschlagen und validiert. Diese Pipeline unterteilt die Robotersteuerung in vier Phasen, die verschiedene Aspekte des Entscheidungsprozesses abdecken: Wahrnehmung, Planung, Ausführung und Steuerung. Die Beiträge dieser Dissertation sind nach diesen vier Phasen gegliedert.Die Arbeit befasst sich zunächst mit der Anpassung von diskretem DRL für Robotikanwendungen, indem ein asynchroner DRL-Ansatz und kontinuierliche, glatte Aktionsräume mittels Bézier-Kurven entwickelt werden. Dies ermöglicht eine ruckfreie Robotersteuerung und reduziert die mechanische Belastung sowie die Ausführungszeit. Darüber hinaus wird die Vorhersage zukünftiger visueller Zustände mit generativen Modellen erforscht. Für das Wahrnehmungsmodul wird in dieser Arbeit eine semi-überwachte Lernstrategie eingeführt, die Autoencodermodelle verwendet, um hochdimensionale Bildzustände zu komprimieren, um kombiniert mit Domänenwissen in Form von Segmentierungskarten latente Repräsentationen zu extrahieren, welche von einem DRL-Agenten genutzt werden können. Das Modul wird vollständig in der Simulation mit Domain Randomization trainiert und benötigt nur ein einziges annotiertes reales Bild, um einen Zero-Shot-Transfer in die reale Welt zu erreichen.Darüber hinaus werden wesentliche Beiträge zur Strukturausnutzung in Roboterbewegungen für TRL erbracht. Es wird gezeigt, dass hierarchisches Reinforcement Learning (HRL) modulare, aufgabenunabhängige Verhaltensstrategien auf unteren Ebenen erlernt, die sich auf neue Aufgaben übertragen lassen. Zusätzlich wird das Konzept der „Montage durch Demontage“ im Kontext von TRL beleuchtet. Basierend auf der Erkenntnis, dass das Erlernen einer Demontageaufgabe aufgrund einer stärkeren Eingrenzung meist einfacher ist als die eigentliche Montageaufgabe, wird in dieser Arbeit eine Strategie vorgeschlagen, bei der zunächst die Demontageaufgabe gelöst und anschließend die invertierten Trajektorien zum Vortraining eines Montageagenten verwendet werden. Die Ergebnisse zeigen, dass mit dieser Strategie eine Montageaufgabe erfolgreich gelöst wird, bei der ein direkter DRL-Ansatz versagt. Schließlich wird in dieser Arbeit ein roboterübergreifender Imitationsansatz vorgestellt, bei dem Trajektorien zwischen Robotern unterschiedlicher Morphologie unter Verwendung expliziter Vorwärts- und Rückwärtskinematiken und Embodiment-Metriken übertragen werden. Es wird gezeigt, dass die übertragenen Demonstrationen für ein effektives Vortraining genutzt werden können.Diese Beiträge werden anhand von drei repräsentativen Fallbeispielen evaluiert: dem akademischen Wire-Loop-Spiel, einem visuell gesteuerten Objekt-Picking-Szenario und einer industriellen Montageaufgabe für Flugzeug-Clips. Insgesamt zeigen die Ergebnisse, dass modulares TRL eine effizientere, leistungsfähigere und besser wartbare Automatisierung in der Industrierobotik ermöglicht. Die Dissertation schließt mit einer kritischen Würdigung, die aus den Ergebnissen Schlüsse für zukünftige Forschungsansätze hinsichtlich modularer und transferierbarer KI in industriellen Robotikanwendungen zieht.

Cited by 0Open access

APPLICATIONS OF REINFORCEMENT LEARNING IN ROBOTICS

Mushtaq Khan · Computer Science Bulletin · 2018

Reinforcement Learning (RL) has emerged as a transformative approach in the field of robotics, enabling robots to learn from interactions with their environment to achieve complex tasks. This article explores the applications of RL in various domains of robotics, including motion control, robotic manipulation, navigation, and human-robot interaction. The integration of RL with advancements in neural networks has significantly improved robots' adaptability, decision-making, and autonomy. Despite the challenges related to sample efficiency, safety, and real-world implementation, RL continues to show immense potential in addressing complex robotics problems. The article concludes with a discussion on future directions and the role of RL in shaping autonomous robotic systems.

Cited by 0Open access

Reinforcement learning in robotics: A survey

Jens Kober, J. Andrew Bagnell, Jan Peters · The International Journal of Robotics Research · 2013

Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between model-based and model-free as well as between value-function-based and policy-search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research.

Cited by 2,977Open access

Reinforcement Learning in Robotics: Challenges and Applications

O. P. Singh, Vishnu Kumar Barodiya, Abhishek Paridwal et al. · Türk bilgisayar ve matematik eğitimi dergisi · 2020

Reinforcement Learning (RL) is at the vanguard of robotics revolution, allowing machines to learn and make choices in complicated environments. This paper explores the symbiotic dating between RL and robotics, focusing at the challenges and programs that shape this interdisciplinary area. The dialogue makes a speciality of the fundamental standards of RL and its integration into robotics, elucidating the specific demanding situations encountered on this merger. The research covers the space among simulation-primarily based schooling and real-world applicability, navigating hardware obstacles, and addressing safety concerns, organising a comprehensive view of the challenges encountered when deploying RL for robotic systems. This paper also offers insights into the various programs of RL in robotics, inclusive of self sustaining navigation, item manipulation, healthcare, and business automation. It investigates how RL algorithms assist robots navigate complex environments, gain item manipulation dexterity, and contribute to healthcare improvements and industrial optimization. Case studies exhibit the software of RL in robotics by means of highlighting a hit implementations and demonstrating the transformative capability of this aggregate. Finally, this paper outlines future instructions, paving the way for persisted innovation and emphasizing the importance of bridging the theoretical advancements and real-international deployment in RL-driven robotics.

Cited by 1Open access

Techniques for batch reinforcement learning in robotics

Cunha, João Alexandre da Silva Costa e · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · 2015

This thesis addresses the Batch Reinforcement Learning methods in Robotics. This sub-class of Reinforcement Learning has shown promising results and has been the focus of recent research. Three contributions are proposed that aim to extend the state-of-art methods allowing for a faster and more stable learning process, such as required for learning in Robotics. The Q-learning update-rule is widely applied, since it allows to learn without the presence of a model of the environment. However, this update-rule is transition-based and does not take advantage of the underlying episodic structure of collected batch of interactions. The Q-Batch update-rule is proposed in this thesis, to process experiencies along the trajectories collected in the interaction phase. This allows a faster propagation of obtained rewards and penalties, resulting in faster and more robust learning. Non-parametric function approximations are explored, such as Gaussian Processes. This type of approximators allows to encode prior knowledge about the latent function, in the form of kernels, providing a higher level of exibility and accuracy. The application of Gaussian Processes in Batch Reinforcement Learning presented a higher performance in learning tasks than other function approximations used in the literature. Lastly, in order to extract more information from the experiences collected by the agent, model-learning techniques are incorporated to learn the system dynamics. In this way, it is possible to augment the set of collected experiences with experiences generated through planning using the learned models. Experiments were carried out mainly in simulation, with some tests carried out in a physical robotic platform. The obtained results show that the proposed approaches are able to outperform the classical Fitted Q Iteration.

Cited by 0Open access

Reinforcement Learning for Tracking Control in Robotics

YudhaPrawira Pane · 2015

For a robot manipulator, an accurate reference tracking capability is one of the most important performance factor. This is especially the case for a robot which is deployed for an industrial use such as welding and laser cutting. Recently, an emerging application of 3D printing also attracts researchers to use robotic manipulators in hope of gaining a higher dimension in the printing process as well as a larger workspace. The conventional method for controlling robot manipulator involves a model based feed-forward controller to eliminate the nonlinearities such as gravity and the coriolis term, combined with a feedback controller to compensate the residual error. This method, however, has a drawback in obtaining a good model of the robot. Furthermore, a robot's physical properties are subject to change over time due to, for instance, degraded gearboxes, etc. Such a problem will require the robot to re-perform a system identification which is often unacceptable due to the time it takes and the possibly dangerous movement the robot must execute. Therefore, this class of controller may not be the best option for such condition. This thesis addresses the aforementioned problem from the point of view of model-free, learning controller which improves the tracking performance safely in an online fashion. The thesis exploits the reinforcement learning (RL) technique which uses the nature of trial and errors to correct the tracking error. Improvement to the control performance is done by feeding an additive compensation signal given by a learned policy function to the robot's nominal controller. Two different compensation methods are proposed in the thesis. Both methods are realized by the so called actor-critic (AC) algorithm in order to cope with the continuous state of a robot arm. The first method compensates the control input of the nominal controller while the second method modifies the reference trajectory. The methods are implemented on a real 6 DoFs 3D printing robotic setup. For the test, three different reference tracking tasks are designed and executed. Furthermore, to provide a comparative study, two conventional control methods namely model predictive control (MPC) and iterative learning control (ILC) are implemented as well. Their results are then analysed and compared to that of the proposed RL-based controllers. The experimental result shows that the RL-based controllers improve the performance of the nominal control significantly. It is also shown that the performance of the learning controller is close to that of the model-based control method. For a circular, smooth trajectory, the experiment even shows that the RL-based controller is able to outmatch both MPC and ILC performance.

Cited by 0

Comprehensive Review of Robotics Operating System-Based Reinforcement Learning in Robotics

Mohammed Aljamal, Sarosh Patel, Ausif Mahmood · Applied Sciences · 2025

Common challenges in the area of robotics include issues such as sensor modeling, dynamic operating environments, and limited on-broad computational resources. To improve decision making, robots need a dependable framework to facilitate communication between different modules and the optimal action for real-world applications. The Robotics Operating System (ROS) and Reinforcement Learning (RL) are two promising approaches that help accomplish precise control, seamless integration of sensors-actuators, and exhibit learned behavior. The ROS enables seamless communication between heterogeneous components, while RL focuses on learning optimal behaviors through trial-and-error scenarios. Combining the ROS and RL offers superior decision making, improved perception, enhanced automation, and reliability. This work focuses on investigating ROS-based RL applications across various domains, aiming to enhance understanding through comprehensive discussion, analysis, and summarization. We base our evaluation on the application area, type of RL algorithm used, and degree of ROS–RL integration. Additionally, we provide summary of seminal works that define the current state of the art, along with GitHub repositories and resources for research purposes. Based on the review of successfully implemented projects, we make recommendations highlighting the advantages and limitations of RL techniques for specific applications and environments. The ultimate goal of this work is to advance the robotics field by providing a comprehensive overview of the recent important works that incorporate both the ROS and RL, thereby improving the adaptability of these emerging techniques.

Cited by 10Open access

Safe Policy Optimization for Reinforcement Learning in Robotics

Hao Wang, Zhen Kan · Elsevier eBooks · 2024

Cited by 0

Deep Reinforcement Learning in Robotics and Autonomous Systems

Uma Yadav, Shweta V. Bondre, Bhakti Thakre · 2024

Deep reinforcement learning (DRL) has begun as a powerful paradigm for enabling intelligent behavior in robotics and autonomous systems. A potential method for autonomously learning complicated behaviors from cursory sensor data is deep reinforcement learning (RL). Deep RL has shown potential in empowering somatic robots to acquire difficult abilities in the actual world despite a substantial chunk of research focusing on solicitations in audiovisual games and virtual control, which do not link with the restrictions of erudition in existent situations. Actual-world robotics, which is closely related to how individuals learn as personified agents in the actual world—provides an interesting arena for assessing such algorithms. Numerous difficulties arise while learning to see and move in the actual environment; some of these difficulties are simpler to solve than others, and some of these difficulties are frequently overlooked in RL research that solely considers virtual domains. In this chapter, we give many case revisions employing mechanical deep RL. We examine often-recognized deep RL issues and how these issues have been addressed in these studies, building on these case examples. We also give a brief summary of other problems that still need to be solved, numerous of which are specific to the real-world computing environment and are not frequently the subject of conventional RL research. This chapter provides a widespread outline of the solicitation of DRL techniques to address the challenges and complexities inherent in robotic tasks. The chapter also highlights the importance of transfer learning, sim-to-real transfer, and safety considerations in deploying DRL agents in real-world environments. Finally, we outline future directions and the ongoing challenges that researchers and practitioners face in harnessing the potential of DRL for shaping the future of robotics and autonomous systems.

Cited by 2

Reinforcement Learning for Solving Control Problems in Robotics

Askhat Diveev, Elena Sofronova, S.V. Konstantinov et al. · 2023

The use of reinforcement learning technology for the optimal control problem solution is considered. To solve the optimal control problem an evolutionary algorithm is used that finds control to ensure the movements of a control object along different trajectories with approximately the same values of the quality criterion. Additional conditions for passing the trajectory in the neighbourhood of given areas of the state space are included in the quality criterion. To build a stabilization system for the movement of an object along a given trajectory, machine learning control by symbolic regression is used. An example of solving the optimal control problem for a quadcopter is given.

Cited by 2Open access

Data-Efficient Reinforcement and Transfer Learning in Robotics

Xi Chen · KTH Publication Database DiVA (KTH Royal Institute of Technology) · 2020

In the past few years, deep reinforcement learning (RL) has shown great potential in learning action selection policies for solving different tasks.Despite its impressive success in games, several challenges remain, such as designing appropriate reward functions, collecting large amounts of interactive data, and dealing with unseen cases, which make it difficult to apply RL algorithms to real-world robotics tasks. The ability of data-efficient learning and rapid adaptation to novel cases is essential for an RL agent to solve real-world problems. In this thesis, we discuss algorithms to address the challenges in RL by reusing past experiences gained while learning other tasks to improve the efficiency of learning new tasks.Instead of learning directly from the target task, which is complicated and sometimes unavailable during training, we propose first learning from relevant tasks that contain valuable information about the target environment, and reuse the obtained solutions in solving the target task.We follow two approaches to achieve knowledge sharing between tasks. In the first approach, we model the problem as a transfer learning problem and learn to minimize the distance between the representations found based on the training and the target data, such that the learned solution can be applied to the target task using a small amount of data from the target environment.In the second approach, we formulate it as a meta-learning problem and obtain a model that is explicitly trained for rapid adaptation using a small amount of data. At test time, we can learn quickly on top of the trained model in a few iterations when facing a new task. We demonstrate the effectiveness of the proposed frameworks by evaluating the methods on a number of real and simulated robotic tasks, including robot navigation, motion control, and manipulation. We show how these methods can be applied to challenging tasks with high-dimensional state/action spaces, limited data, sparse rewards, and requiring diverse skills.

Cited by 0Open access

Deep reinforcement learning in robotics and dialog systems

Rui Zhao · Electronic Theses of LMU Munich (Ludwig-Maximilians-Universität München) · 2020

Cited by 0Open access

Safe Reinforcement Learning to Make Decisions in Robotics

Gu, Shangding, Knoll, Alois 1961-, Technische Universität München · 2023

Reinforcement Learning (RL) has demonstrated remarkable efficacy. Nevertheless, the widespread adoption of RL techniques remains predominantly confined to simulated environments, primarily due to safety concerns. To handle the inherent limitations of RL, safe RL has emerged as a critical area, particularly in robotics. To handle the above challenges, we propose multifaceted approaches for robot RL: efficiency to ensure safety, multi-functionality to satisfy multi-objective requirements, cooperation to work with other robots, and robustness to carry out tasks in uncertain environments.

Cited by 0

REINFORCEMENT LEARNING: APPLICATIONS IN ROBOTICS AND AUTONOMOUS SYSTEMS

Muhammad Ali Khan · Computer Science Bulletin · 2023

Reinforcement learning (RL) has emerged as a powerful machine learning technique for enabling intelligent agents to make decisions through interactions with their environment. This paper explores the applications of RL in robotics and autonomous systems, providing insights into how RL algorithms enable robots and autonomous systems to perform tasks autonomously in dynamic and uncertain environments. The study covers key advancements, challenges, and future trends in the integration of RL within autonomous systems, highlighting the potential to revolutionize industries such as manufacturing, healthcare, transportation, and security. Through a review of various RL models and case studies, this paper demonstrates the applicability of RL techniques to real-world problems, emphasizing their practical applications and performance in robotic systems. Finally, the paper outlines potential research directions and the evolution of RL in intelligent robotic and autonomous systems.

Cited by 0Open access

DEEP REINFORCEMENT LEARNING FOR ADAPTIVE CONTROL IN ROBOTICS

Discover Archive (Vanderbilt University) · 2022

Cited by 0

Reinforcement Learning in Robotics: Unlocking Applications and Advancements

MD. Faiyaz Ahmed, Anil Kumar Tulluri, N. B. Prakash Tiruveedula · BENTHAM SCIENCE PUBLISHERS eBooks · 2025

The intersection of Reinforcement Learning (RL) and robotics has ushered in a new era of autonomy and intelligence, fundamentally transforming robotic capabilities across various fields. This chapter embarks on a comprehensive journey through RL applications in robotics, highlighting pivotal advancements, persistent challenges, and promising future directions. RL's impact on robotics lies in its ability to enable machines to learn and adapt from experience, navigating complex environments, manipulating objects, and interacting seamlessly with humans. Through iterative interactions with their surroundings, RL-driven robots dynamically refine their behaviors, circumventing the need for cumbersome task-specific programming. This flexibility enhances their versatility and makes them invaluable in industries such as manufacturing, healthcare, and logistics, where tasks involve non-linearities, uncertainties, and dynamic constraints. RL empowers robots to develop personalized and context-aware capabilities, adjusting behaviors in response to real-time feedback and environmental cues. This adaptability is crucial in assistive robotics, where robots must engage with humans safely and intuitively, accounting for individual preferences and situational contexts. Additionally, RL-driven optimizations in resource utilization, energy efficiency, and task execution bolster system performance and costeffectiveness. However, the widespread adoption of RL in robotics presents challenges, such as developing efficient exploration strategies and ensuring robustness against sensor noise and environmental variability. Moreover, the ethical and safety implications of deploying RL-powered robots in real-world settings demand careful consideration and interdisciplinary collaboration. This chapter offers a holistic examination of RL's transformative potential in robotics, underscoring its role in unlocking new frontiers of autonomy, intelligence, and adaptability. By elucidating key applications, challenges, and future avenues, it aims to inspire researchers, practitioners, and enthusiasts to harness RL's transformative power in shaping the future of robotics.

Cited by 0

Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

Lukas Brunke, Melissa Greeff, Adam W. Hall et al. · Annual Review of Control Robotics and Autonomous Systems · 2022

The last half decade has seen a steep rise in the number of contributions on safe learning methods for real-world robotic deployments from both the control and reinforcement learning communities. This article provides a concise but holistic review of the recent advances made in using machine learning to achieve safe decision-making under uncertainties, with a focus on unifying the language and frameworks used in control theory and reinforcement learning research. It includes learning-based control approaches that safely improve performance by learning the uncertain dynamics, reinforcement learning approaches that encourage safety or robustness, and methods that can formally certify the safety of a learned control policy. As data- and learning-based robot control methods continue to gain traction, researchers must understand when and how to best leverage them in real-world scenarios where safety is imperative, such as when operating in close proximityto humans. We highlight some of the open challenges that will drive the field of robot learning in the coming years, and emphasize the need for realistic physics-based benchmarks to facilitate fair comparisons between control and reinforcement learning approaches.

Cited by 603Open access