I am a Ph.D. candidate at
the Autonomous Learning Robots (ALR) Lab of
the Karlsruhe Institute of Technology (KIT),
under the supervision of Prof. Gerhard Neumann
and T.T. Prof. Rudolf Lioutikov.
My research focuses on advancing the learning and representation of robotic movement trajectories,
with a particular emphasis on their applications in imitation and reinforcement learning.
I aim to improve model capacity, learning efficiency, and interpretability by leveraging
state-of-the-art algorithms and tools.
In addition to my research, I serve as a teaching assistant at KIT, contributing to courses
Cognitive Systems (SS20, SS21), Machine Learning (WS21/22, SS23, SS24),
and partially in Deep Reinforcement Learning (WS21/22). I also mentor and supervise
several Bachelor’s and Master’s research projects, seminars, and theses.
I was born and raised in Beijing, China and obtained my Bachelor’s degree in
Mechanical Engineering at
the University of Science and Technology of China.
Later, I moved to Germany to pursue my Master’s degree in Computer-Aided Mechanical Engineering at
the RWTH Aachen University.
During my studies, I gained practical experience as a software engineering intern at Kärcher in Stuttgart,
focusing on autonomous robotic cleaners.
For my Master’s thesis, I joined the Max Planck Institute for Intelligent Systems
(MPIIS), where I collaborated with my thesis supervisor Prof. Jan Peters
and my current Ph.D. advisor Prof. Gerhard Neumann.
News
Dec. 2024, I presented my co-authored work in NeurIPS 2024 in Vancouver, Canada.
Aug. 2024, one co-authored paper got accepted by CoRL 2024.
May 2024, I presented my PhD work in ICLR 2024 in Vienna, Austria.
Publications
TOP-ERL: Transformer-based Off-Policy Episodic Reinforcement Learning (Under Review)
Ge Li, Dong Tian, Hongyi Zhou, Xinkai Jiang, Rudolf Lioutikov, Gerhard Neumann.
See:
arxiv
We propose a novel online off-policy RL methodology that utilizes a transformer-based critic to learn values of action sequences.
Variational Distillation of Diffusion Policies into Mixture of Experts
Hongyi Zhou, Denis Blessing, Ge Li, Onur Celik, Gerhard Neumann, Rudolf Lioutikov, NeurIPS, 2024.
See:
arxiv |
NeurIPS |
GitHub
We introduce Variational Diffusion Distillation (VDD), a novel method for distilling denoising diffusion policies into a Mixture of Experts (MoE).
MaIL: Improving Imitation Learning with Selective State Space Models
Xiaogang Jia, Qian Wang, Atalay Donat, Bowen Xing, Ge Li, Hongyi Zhou, Onur Celik, Denis Blessing, Rudolf Lioutikov, Gerhard Neumann, CoRL, 2024.
See:
arxiv |
OpenReview |
GitHub
We introduce Mamba Imitation Learning (MaIL), a novel imitation learning architecture that offers a computationally efficient alternative to state-of-the-art Transformer policies.
Open the Black Box: Step-based Policy Updates for Temporally-Correlated Episodic Reinforcement Learning.
Ge Li, Hongyi Zhou, Dominik Roth, Serge Thilges,
Fabian Otto, Rudolf Lioutikov, Gerhard Neumann, ICLR, 2024.
See:
arxiv |
OpenReview |
GitHub
We propose a novel RL framework that integrates step-based information into the
policy updates of Episodic RL, while preserving the broad exploration scope,
movement correlation modeling and trajectory smoothness.
MP3: Movement Primitive-Based (Re-) Planning Policies
Hongyi Zhou, Fabian Otto, Onur Celik, Ge Li, Rudolf Lioutikov, Gerhard Neumann,
in Conference on Robot Learning Workshop on Learning Effective Abstractions for Planning, 2023
See:
arxiv |
Website |
Poster
We enable a new Episodic RL framework that allows trajectory replanning in deep
RL, which allows the agent to react with changing goal and dynamic perturbation.
ProDMPs: A Unified Perspective on Dynamic and Probabilistic Movement Primitives.
Ge Li, Zeqi Jin, Michael Volpp, Fabian Otto, Rudolf Lioutikov and Gerhard Neumann, in IEEE Robotics and Automation Letters (RAL), 2023.
See:
Paper |
Poster |
GitHub |
YouTube
We unified the Dynamic Movement Primitives and the Probabilistic Movement
Primitives into one model, and achieved smooth trajectory generation,
goal-attractor convergence, correlation analysis, non-linear conditioning, and online
re-planing in one framework.
Specializing Versatile Skill Libraries using Local Mixture of Experts
Onur Celik, Dongzhuoran Zhou, Ge Li, Philipp Becker, Gerhard Neumann,
in Conference on Robot Learning, 2021.
See:
Paper |
OpenReview |
YouTube
We developed a mixture of experts RL framework to learn versatile skills
given the same task, such as forehand and backhand strikes in playing table tennis. Our
method is able to assign policy experts to their corresponding context domains
and automatically add or delete these experts when necessary.