Tool Contact Force Estimation for Robot Drawing

In this research project a pen was mounted to a Franka robot arm through a GelSight visuo-tactile sensor, and the force at the pen tip was estimated purely from the sensor’s images. I developed several computer vision approaches to infer the contact force (stereo correspondence, motion flow, and marker-based methods) as well as a neural network-based estimator.

Technologies: ROS, computer vision (stereo, motion flow, SIMD), PyTorch