Visuo-Tactile Slip Detection for Teleoperated Manipulation

My current research at the Intelligent Autonomous Systems group (TU Darmstadt) investigates how robots can feel objects slipping out of their grasp. I built a custom teststand that induces controlled slip with a linear actuator while recording four sensing modalities simultaneously: a GelSight Mini visuo-tactile camera, a six-axis force/torque sensor (ReSense HEX21), an ADXL355 accelerometer, and webcam-based ArUco marker tracking.

On the software side, the system runs on ROS2 Jazzy across a Raspberry Pi 5 (hardware control) and a laptop (high-bandwidth sensor processing). A branching neural network fuses CNN-encoded tactile images with force/torque and IMU time-series through an LSTM to regress slip velocity, the basis for slip-aware grip force control in teleoperation.

Technologies: Python, ROS2, PyTorch, GelSight, force/torque sensing, CAD, Docker