In brief: Robot manipulation is the process of changing an object's position, orientation or physical state through controlled contact. A robot may grasp and place a part, push an object, open a drawer, turn a handle, insert a connector or use a tool. Reliable manipulation combines perception, state estimation, task and motion planning, an appropriate end effector, force-aware control and feedback from the physical interaction.
What is robot manipulation?
Manipulation is broader than picking up an object. It includes any purposeful interaction in which a robot applies forces or motion to an object or its environment. Examples include grasping, pushing, sliding, reorienting, fastening, opening, pouring, assembling and bimanual coordination. The task may be geometric, such as moving a block to a target pose, or functional, such as inserting a plug until an electrical connection is made.
Unlike free-space movement, manipulation is shaped by contact. Friction, compliance, object mass, geometry, uncertainty and the location of contact points all influence what will happen when the robot acts. A plan that looks valid in a camera image can fail if the object is heavier than expected, the gripper touches an edge, a surface slips or a fixture is slightly misaligned.
The manipulation pipeline
A practical system usually coordinates several capabilities. They may be separate modules or learned together in a policy, but the underlying questions remain similar:
- Perception: detect the object, estimate its pose and geometry, identify relevant surfaces and understand the surrounding scene. RGB, depth, stereo and other sensors provide complementary evidence.
- State estimation: combine observations with robot kinematics and sensor readings to estimate the robot configuration, object pose, velocity and uncertainty. The estimate should be updated as the scene changes.
- Task planning: choose the sequence of subgoals. A pick-and-place task, for example, may require approaching, grasping, lifting, transporting, aligning and releasing.
- Grasp or contact selection: choose where and how to make contact, considering reachability, stability, collision risk, object constraints and the available end effector.
- Motion generation: calculate a feasible path and trajectory while respecting joint limits, obstacles, velocity and acceleration limits, and the geometry of the workspace.
- Execution and feedback: track the motion, monitor contact and task progress, then correct, retry or re-plan when observations differ from expectations.
The loop matters: a robot should not assume that an action succeeded merely because a command was issued. It should observe the result, check whether the intended state was reached and decide what to do next.
Grasping and end effectors
The end effector is the part of the robot that makes the task contact. A parallel-jaw gripper is often effective for objects with accessible opposing surfaces. Suction tools can handle suitable flat or non-porous surfaces. Dexterous hands provide more contact options for reorientation and tool use, but add degrees of freedom, calibration demands and control complexity. Bimanual systems can stabilize an object with one arm while the other performs a precise operation.
A grasp is not good simply because the fingers can close around an object. The system must consider whether the contact arrangement can resist expected disturbances, whether the object can be lifted without slipping, whether the wrist can reach the required pose and whether the grasp blocks later steps. Grasp planning therefore connects object geometry to force, reachability and task intent.
Some tasks do not require a full grasp. Pushing, pivoting, sliding and fixture-assisted manipulation deliberately use environmental contact. In these cases, the planner must reason about contact transitions and the forces that keep the object moving along the intended path.
Contact sensing and compliant control
Vision can estimate where an object is, but it may not reveal exactly when contact occurs or how much force is being applied. Joint encoders and motor-current estimates provide proprioceptive information; wrist force/torque sensors measure interaction forces; tactile sensors can provide local information about pressure, contact location or slip. Not every robot has all of these sensors, so a system must be designed around the signals actually available.
Rigid position tracking is useful when the environment is predictable. Contact-rich tasks often benefit from compliance: impedance or admittance control can regulate the relationship between motion, force and the environment. This helps a robot respond to small alignment errors rather than pushing harder against an obstruction. Compliance is not a substitute for safety limits; force, speed and workspace constraints still need explicit treatment.
Planning and control are different layers
Task planning decides what should happen; motion planning decides how the robot can move to make it happen. Inverse kinematics maps a desired end-effector pose to feasible joint configurations. Collision checking rejects paths that intersect the robot's surroundings. Trajectory generation adds timing and smoothness, while a low-level controller tracks commands and reacts to disturbances.
For manipulation, a collision-free path is not always a complete solution. The robot may need to establish contact, maintain a stable grasp, follow a surface, insert a part or release an object without disturbing nearby items. Receding-horizon planning and feedback control can repeatedly update the next action using fresh observations. The appropriate controller depends on the robot, task, sensor latency and physical properties of the interaction.
Robot motion planning vs robotic manipulation
Robot motion planning and robotic manipulation are related, but they describe different parts of a task. Motion planning finds a feasible path for the robot or its end effector through the workspace while respecting kinematics, joint limits and collision constraints. Manipulation deliberately changes an object's pose or physical state through controlled interaction. A manipulation task therefore adds questions about grasp or contact choice, object motion, friction, force limits and whether the intended object-level result was achieved.
- Free-space motion: move an arm to a reachable pose while avoiding obstacles, without intentionally changing an object's state.
- Prehensile manipulation: grasp an object and control it through lifting, carrying, reorienting or placing.
- Non-prehensile manipulation: push, slide, pivot or guide an object without maintaining a grasp, using contact with the robot or environment.
- In-hand manipulation: move or rotate an object relative to the gripper by changing finger contacts or using controlled rolling and slip.
The categories can overlap within one task. A pick-and-place sequence uses free-space motion to approach and transport, but the grasp, lift, placement and release are manipulation phases. Planning systems often combine these decisions as task-and-motion planning: the task layer chooses object-level actions, while the motion layer checks whether the robot can execute them safely and feasibly. For a broad treatment of manipulation, see Mason's survey of robotic manipulation and the survey of manipulation in contact.
Learning-based manipulation
Learning can improve perception, grasp selection, motion generation or the policy that maps observations to actions. Imitation learning uses demonstrations from teleoperation or a human operator. Reinforcement learning optimizes behavior through trial and error in a defined environment. Sequence models can predict several actions at once, while receding-horizon execution keeps the system responsive to new observations.
The ALOHA and Action Chunking with Transformers (ACT) study demonstrated fine-grained bimanual tasks learned from real demonstrations on a comparatively low-cost platform. Diffusion Policy models visuomotor behavior as a conditional diffusion process; its paper evaluates the approach across multiple manipulation tasks and benchmarks. These are examples of learning methods, not guarantees that a policy will transfer unchanged to another robot or factory.
Learning systems still need dependable state estimates, suitable action spaces, enough representative data and a way to detect failure. A model trained on a fixed camera and one gripper may struggle when the camera moves, the object changes or the target robot has different joint limits.
Why diverse robot data matters
Manipulation policies are shaped by the tasks, objects, environments and robot bodies represented in their training data. The DROID dataset describes 76,000 demonstration trajectories collected across varied scenes and tasks. The Open X-Embodiment project combines demonstrations from multiple robot platforms and studies cross-robot learning. These efforts illustrate why diversity can matter: a policy exposed to more than one table, object arrangement or robot may learn representations that are less tied to a single setup.
Dataset size alone is not enough. Collection protocols, calibration, action definitions, task labels, failure examples and licensing all affect whether data can be reused. Evaluation should document which embodiments and environments were actually tested rather than treating a large training set as proof of generalization.
Manipulation is an embodiment problem
A skill is not independent of the body that performs it. Reach, payload, joint range, hand geometry, number of fingers, sensor placement, actuator bandwidth and control frequency all shape the available actions. A grasp pose that is reachable for a six-degree-of-freedom arm may be impossible for a compact humanoid hand. A bimanual strategy may depend on the spacing and coordination of both arms.
Moving a policy between robots therefore requires more than copying model weights. The system may need coordinate-frame conversion, action retargeting, kinematic calibration, adaptation data, a new controller or a different grasp representation. This is one reason manipulation connects closely to embodied AI, humanoid robot control and robot teleoperation.
How to evaluate a manipulation system
Report performance over repeated trials and state the conditions. Useful measures include task success rate, grasp stability, completion time, recovery after a disturbance, collision or safety events, peak contact force, energy use and the number of operator interventions. For learning systems, also test new objects, positions, lighting conditions and environments. For cross-embodiment claims, report results separately for each robot and describe any adaptation performed.
A benchmark is only interpretable when its task definition, object set, reset procedure, sensors, control frequency, allowed retries and failure criteria are clear. A single successful video is a demonstration, not a reliability estimate. Simulation can make tests repeatable, but real hardware validation remains necessary because friction, compliance, sensing and actuator behavior may differ from the model. See the sim-to-real robotics guide for the transfer problem.
Related platform: For a repeatable simulation of a common pick-and-place task, explore the HumanoidBehavior Benchmark workspace. It supports measured MuJoCo runs with explicit seeds and policy versions, and stores results for inspection. Simulation results help compare controlled experiments; they do not replace validation on the target robot.
Practical design checklist
- Define the task outcome and the conditions that count as success.
- Measure object geometry, mass range, friction and relevant tolerances.
- Choose an end effector and sensors that match the contact requirements.
- Check reachability, collision constraints and safe force or speed limits.
- Use feedback to verify grasp, placement and task completion.
- Test repeated trials, failure recovery and variation in objects and scenes.
- Document the robot embodiment, software version, data and evaluation setup.
Research references
- Khazatsky et al., “DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset” (2024).
- Zhao et al., “Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware” (2023).
- Chi et al., “Diffusion Policy: Visuomotor Policy Learning via Action Diffusion” (2023; revised 2024).
- Open X-Embodiment Collaboration, “Open X-Embodiment: Robotic Learning Datasets and RT-X Models” (2023).
- Fu et al., “Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation” (2024).
- Mason, “Toward Robotic Manipulation”, Annual Review of Control, Robotics, and Autonomous Systems (2018).
- Suomalainen et al., “A Survey of Robot Manipulation in Contact” (2022).
Explore the field
Continue through the research map, robot model directory, robot platform directory, robotics company directory and robot manipulation overview.