Research / Perception

Perception & State Estimation

Embodied systems must turn noisy observations into a state representation that is useful for action. In robots, perception is therefore tightly coupled to control, planning and learning.

What the problem contains

Robot perception can combine RGB or depth cameras, force and tactile sensing, joint encoders, inertial measurements and other signals. State estimation concerns the robot's belief about pose, objects, contacts, velocities and task-relevant conditions.

Why embodiment changes the problem

A physical robot does not observe the world from a fixed viewpoint. Motion changes the camera view; contact creates new information; occlusion and sensor noise are unavoidable. Useful representations therefore need to remain actionable under changing geometry and dynamics.

Connection to embodied AI

Modern robot-learning systems increasingly combine perception with language, demonstrations and action prediction. This makes the boundary between “seeing” and “acting” less rigid: the useful representation is often the one that supports the next physical decision.

Questions worth tracking