3D Scanning and Measuring with Visuo-Tactile Sensors

Using a GelSight Mini visuo-tactile sensor, this project reconstructs a unified 3D surface from multiple overlapping sensor presses. Each capture produces an RGB image and a neural-network-inferred depth map; individual captures are registered with SIFT keypoint matching and fused into a single colored point cloud, exportable as a triangulated mesh. Euro and cent coins served as test objects.

Beyond the stitching pipeline itself, the project provides tools for live 3D visualization, snapshot capture, and inspecting SIFT correspondences between captures, along with a driver layer for the GelSight device and its depth reconstruction.

Technologies: computer vision, 3D point cloud analysis, image stitching, PyTorch