Chi Zhang

I am currently a PhD student at the ATARI Lab at the Technical University of Munich. My research focuses on enhancing the capabilities of concurrent robotic systems by combining optimal control and learning-based methods. I will be working with hardware platforms, including Unitree Go2, Aliengo, and G1.

I obtained my master's degree in Mechatronics and Robotics (MuR) from the School of Engineering and Design at the Technical University of Munich. I earned my bachelor's degree in Mechanical Engineering in 2019 from the Department of Mechanical Engineering at the China University of Mining and Technology.

I completed my master's thesis in 2023 at the Robotics Innovation Center (RIC) at the German Research Center for Artificial Intelligence (DFKI) under the supervision of Felix Wiebe, Akhil Sathuluri, Prof. Dr. Shivesh Kumar, Prof. Dr. Frank Kirchner, and Prof. Dr. Markus Zimmermann.

In 2022, I worked as a research assistant in the Robot Systems Group at the Laboratory for Product Development and Lightweight Design (LPL) at the School of Engineering and Design of the Technical University of Munich.

Email  /  GitHub  /  LinkedIn

Reinforcement learning based control of an underactuated double pendulum system

In this project, reinforcement learning-based control was attempted on an underactuated double pendulum system to perform the tasks of swinging up and stabilization. Depending on the joint being controlled, the systems are referred to as the ‘pendubot’ and the ‘acrobot.’ A combined controller consisting of an RL agent trained with the Soft Actor-Critic (SAC) algorithm for the swing-up task and a Linear Quadratic Regulator (LQR) for stabilization is employed. A 3-stage reward function was designed to guide the agent into the Region of Attraction (RoA) of the LQR controller, where the system achieves asymptotic stability toward the goal state and maintains stability over an extended duration.

Validation in an ideal environment, without real-world features such as friction, torque responsiveness, measurement noise, and signal delay, was successful. During the real hardware phase, a noisy simulation was established to mimic these real-world conditions before transferring the controller to real hardware. To address the sim-to-real gap, a noisy training process based on domain randomization was implemented to enhance robustness. The SAC+LQR controller, which demonstrated a 40% success rate on a real pendubot setup, was evaluated in terms of performance and robustness based on results from both simulations and real-world tests, with quantitative metrics reflected on the respective leaderboards.

The results were presented at IJCAI 2023 to participate in the RealAIGym competition. Details can be found in the pre-print.

Tmotor AK-series motor driver based on CAN and Linux

In this project, a software driver for Tmotor AK-series motors is designed based on the CAN protocol and a Linux environment. The AK-series motors are lightweight, quasi-direct electric motors with an inbuilt MIT Mini-Cheetah controller, making them suitable for modern robotics applications such as exoskeletons, quadrupeds, and humanoids.

This software driver utilizes the standard CAN communication protocol and employs ROS for node management. The entire driver is written in Python and implemented in a Linux environment. Middle-level control functionalities, including motor enable/disable and feedback control of position, velocity, and torque, have been implemented. Additionally, some high-level control functions, such as trajectory planning and tracking, have been tested on a self-designed testbench and a 7-DOF robotic arm.

Numerical Co-optimization for Robot Design and Control

In this project, two topics of optimization in the field of robotics have been discussed, being workspace optimization and control optimization.

In the workspace optimization, a five-bar parallel structure and a cable-driven musculoskeletal robot are presented as examples of configuration optimization. And in the control optimization part, a trajectory optimization task has been attempted given the initial and goal point. Eventually, a bi-level optimization method combining workspace and control optimization is introduced for a 2R-robot design.

Tour into the picture

In this project, we provide an interactive application, which enables the user to generate animations from a single 2D image. This application provides users with the following functions in the form of GUI: select an image, select the foreground object, select the vanishing point and inner rectangle, as well as change the perspective.

Firstly, once the user has selected foreground objects, it’s possible to distinguish foreground objects from the background. Secondly, the user adds a vanishing point and inner rectangle (rear wall) for the scene. Consequently, we can extract 5 regions from the picture. Based on the collection of polygons from background and foreground objects a simple scene model will be constructed.

Simulation of an autonomous system for medicine production based on Siemens PLC

In this project, our team set up a simulation to automate the material transfer process in the vaccine production chain.

Functions like manual on & off, auto-running, error interruption & resume, status representation, and emergency-stop have been realized using the Siemens PLC platform.

Multiple programming methods have been attempted, including FBD(Function Block Diagram), LD(Ladder Diagram), IL(Instruction List), SCL(Structured Control Language), SFC(Sequential Function Chart).