File Download

There are no files associated with this item.

  Links for fulltext
     (May Require Subscription)
Supplementary

Article: Zero-shot sim-to-real transfer of reinforcement learning framework for robotics manipulation with demonstration and force feedback

TitleZero-shot sim-to-real transfer of reinforcement learning framework for robotics manipulation with demonstration and force feedback
Authors
Keywordsdigital twin
reinforcement learning
sim-to-real transfer
Issue Date7-Sep-2022
PublisherCambridge University Press
Citation
Robotica, 2022, v. 41, n. 3, p. 1015-1024 How to Cite?
Abstract

In the field of robot reinforcement learning (RL), the reality gap has always been a problem that restricts the robustness and generalization of algorithms. We propose Simulation Twin (SimTwin) : a deep RL framework that can help directly transfer the model from simulation to reality without any real-world training. SimTwin consists of a RL module and an adaptive correct module. We train the policy using the soft actor-critic algorithm only in a simulator with demonstration and domain randomization. In the adaptive correct module, we design and train a neural network to simulate the human error correction process using force feedback. Subsequently, we combine the above two modules through digital twin to control real-world robots, correct simulator parameters by comparing the difference between simulator and reality automatically, and then generalize the correct action through the trained policy network without additional training. We demonstrate the proposed method in an open cabinet task; the experiments show that our framework can reduce the reality gap without any real-world training.


Persistent Identifierhttp://hdl.handle.net/10722/337637
ISSN
2023 Impact Factor: 1.9
2023 SCImago Journal Rankings: 0.579
ISI Accession Number ID

 

DC FieldValueLanguage
dc.contributor.authorChen, YP-
dc.contributor.authorZeng, C-
dc.contributor.authorWang, ZP-
dc.contributor.authorLu, P-
dc.contributor.authorYang, CG -
dc.date.accessioned2024-03-11T10:22:42Z-
dc.date.available2024-03-11T10:22:42Z-
dc.date.issued2022-09-07-
dc.identifier.citationRobotica, 2022, v. 41, n. 3, p. 1015-1024-
dc.identifier.issn0263-5747-
dc.identifier.urihttp://hdl.handle.net/10722/337637-
dc.description.abstract<p>In the field of robot reinforcement learning (RL), the reality gap has always been a problem that restricts the robustness and generalization of algorithms. We propose Simulation Twin (SimTwin) : a deep RL framework that can help directly transfer the model from simulation to reality without any real-world training. SimTwin consists of a RL module and an adaptive correct module. We train the policy using the soft actor-critic algorithm only in a simulator with demonstration and domain randomization. In the adaptive correct module, we design and train a neural network to simulate the human error correction process using force feedback. Subsequently, we combine the above two modules through digital twin to control real-world robots, correct simulator parameters by comparing the difference between simulator and reality automatically, and then generalize the correct action through the trained policy network without additional training. We demonstrate the proposed method in an open cabinet task; the experiments show that our framework can reduce the reality gap without any real-world training.</p>-
dc.languageeng-
dc.publisherCambridge University Press-
dc.relation.ispartofRobotica-
dc.rightsThis work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.-
dc.subjectdigital twin-
dc.subjectreinforcement learning-
dc.subjectsim-to-real transfer-
dc.titleZero-shot sim-to-real transfer of reinforcement learning framework for robotics manipulation with demonstration and force feedback-
dc.typeArticle-
dc.identifier.doi10.1017/S0263574722001230-
dc.identifier.scopuseid_2-s2.0-85148014080-
dc.identifier.volume41-
dc.identifier.issue3-
dc.identifier.spage1015-
dc.identifier.epage1024-
dc.identifier.eissn1469-8668-
dc.identifier.isiWOS:000850705000001-
dc.publisher.placeNEW YORK-
dc.identifier.issnl0263-5747-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats