MiTaS: Multi-Resolution Tactile Imitation Learning for Contact-Rich Robotic Manipulation

TL;DR

Contact-rich manipulation needs more than vision. MiTaS fuses multiple tactile modalities at different temporal resolutions and trains a flow-matching policy that outperforms vision-only and standard visuo-tactile baselines on five real-world tasks.

MiTaS multi-sensor setup: RGB camera, GelSight Mini, and Evetac tactile sensors on a robotic gripper

Abstract

Touch sensing is beneficial for solving a wide variety of manipulation tasks. While there exists a wide range of tactile sensors with different properties, exploiting the fusion of multiple heterogeneous tactile sensors to improve manipulation learning remains underexplored. We present Multi-Resolution Tactile Sensing (MiTaS), a representation framework that leverages multiple tactile sensors operating at different temporal resolutions in order to solve complex contact-rich manipulation tasks. We propose an effective architecture using modality-specific convolutional stems and transformer-based fusion that effectively integrates information from an RGB camera stream, a vision-based GelSight Mini sensor and a high-frequency event-based Evetac sensor. This multisensory representation then conditions a flow-matching policy for solving downstream tasks. Our systematic study across five contact-rich manipulation tasks demonstrates the effectiveness of multi-resolution tactile features in imitation learning wherein MiTaS achieves an average success rate of 80 %, while vision-only (35.5 %) and visuotactile (55.5 %, 63 %) baselines cannot solve the task reliably.

MiTaS Architecture

Overview of the MiTaS architecture. Modality-specific CNN and 3D-CNN stems encode vision, GelSight, and Evetac into sensor tokens. Sensor fusion turns those tokens into the condition for a flow-matching policy with adaptive layer norm, cross-attention, and self-attention. The policy does not receive the robot state and predicts the action from sensor readings alone.

Real-World Robot Evaluations

Gear Assembly

Lamp Installation

Key in Lock

Board Wiping

Lightbulb Connection

Sensor Attention Analysis

Sensor attention over timesteps during lamp installation: vision for reaching, Evetac during insertion, GelSight during screwing

Attention weights during lamp installation for Vision, GelSight Mini, and Evetac over task time. Vision is highest during reaching, Evetac during insertion, and GelSight Mini during screwing.

Experimental Results

Success rates: MiTaS outperforms Sparsh-X, ViT-CNN, and ViT on gear assembly, board wiping, lamp installation, key in lock, and lightbulb connection

MiTaS (Ours) outperforms Sparsh-X and the vision-only ViT-CNN and ViT baselines on all five contact-rich tasks, reaching 85% on gear assembly, 95% on board wiping, 83% on lamp installation, 75% on key in lock, and 63% on lightbulb connection.

BibTeX

@inproceedings{krohn2026mitas,
      title={Multi-Resolution Tactile Imitation Learning for Contact-Rich Robotic Manipulation},
      author={Rickmer Krohn and Erik Helmut and Niklas Funk and Jan Peters and Vignesh Prasad and Georgia Chalvatzaki},
      booktitle={10th Annual Conference on Robot Learning},
      year={2026},
      url={https://openreview.net/forum?id=b2gSINMX3R}
}

Acknowledgments

This research is funded by the German Research Foundation (DFG) Emmy Noether Programme (CH 2676/1-1), the EU’s Horizon Europe project ARISE (Grant no.: 101135959), the German Federal Ministry of Education and Research (BMBF) project “RiG” (Grant no.: 16ME1001) and the European Research Council (ERC) project “SIREN” (Grant No.: 101163933). The authors gratefully acknowledge the scientific support and HPC resources provided by the Erlangen National High Performance Computing Center (NHR@FAU) of the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU). The hardware is funded by the German Research Foundation (DFG).