UMI Gripper Demonstrations
Handheld-gripper demonstrations of household tasks: six synchronised cameras, gripper trajectories and subtask labels.
Demo combines the main head view (left) with one camera on each gripper (right). Delivered as six separate camera streams plus trajectories.
Overview
UMI (Universal Manipulation Interface) data is recorded with handheld grippers instead of a robot: a person performs the task holding instrumented grippers, so every demonstration already has a robot's end-effector, wrist-camera viewpoint and action space. That makes it more directly usable for robot policies than video of bare hands.
Bimanual recordings are delivered in LeRobot v2.1 format: six synchronised 30 fps camera streams (two head views and two cameras on each gripper) plus per-frame state and action trajectories for both arms and grippers in Parquet.
The collection holds 1,320,332 recordings and 70,680 valid hours across bedrooms, living rooms, kitchens, bathrooms, studies, balconies, entryways and dining rooms. Every recording is segmented into subtasks (27.2 million in total, about 20 per recording), each with an action label and start/end timestamps.
Environments
- Kitchens
- Living rooms
- Bedrooms
- Bathrooms
- Studies and home offices
- Balconies, entryways and dining rooms
Built for
- Diffusion-policy and VLA training on gripper actions
- Bimanual and long-horizon household manipulation
- Cross-embodiment pretraining with wrist-camera views
- Subtask segmentation and task-progress models
Scenarios
PreviewWashing Cherry Tomatoes
Opening a cabinet, scooping cleaning powder onto cherry tomatoes, filling the bowl at the sink and rinsing them. Both grippers work together across 14 annotated subtasks.
PreviewFitting a Sofa Cover
Lifting, shaking out and pulling a patterned cover over a sofa: long-horizon handling of large deformable cloth, with 49 annotated subtasks over 4.6 minutes.
PreviewClearing and Wiping a Kitchen Counter
Moving bottles and jars from around the stove to the sink side, then wiping the stove and counter with a cloth. 20 annotated subtasks over 2.5 minutes.
PreviewClearing Tableware to the Sink
Collecting chopsticks, spoons, bowls and plates from a dining table into a draining basket and carrying them to the kitchen sink. 24 annotated subtasks over 4.2 minutes.
PreviewPutting Away Small Clothes
Picking socks and small garments off a bed and placing them into the compartments of a corner cabinet. 40 annotated subtasks over 4.1 minutes.
PreviewRinsing Utensils at the Sink
Turning on the tap, rinsing ladles and strainers from a basin under running water and setting them on a draining rack. 24 annotated subtasks over 3.2 minutes.
PreviewShoe Arrangement
Picking shoes and slippers up from the floor and lining them up in pairs beside the bed.
Capture specifications
Bimanual recordings: two head views at 1408×1280 and four gripper-camera views at 640×480, all 30 fps, plus per-frame state and action trajectories. Shoe-arrangement recordings: wrist and chest views at 1920×1536. Demo clips on this page combine two or three views and are downscaled and muted.
| Scenario | Resolution | FPS | Sample length | Device | Extras |
|---|---|---|---|---|---|
| Washing Cherry Tomatoes | 1408×1280 head · 640×480 wrist | 30 | 1m 04s | Bimanual UMI grippers, 6 cameras | H.264 · metadata |
| Fitting a Sofa Cover | 1408×1280 head · 640×480 wrist | 30 | 4m 36s | Bimanual UMI grippers, 6 cameras | H.264 · metadata |
| Clearing and Wiping a Kitchen Counter | 1408×1280 head · 640×480 wrist | 30 | 2m 33s | Bimanual UMI grippers, 6 cameras | H.264 · metadata |
| Clearing Tableware to the Sink | 1408×1280 head · 640×480 wrist | 30 | 4m 13s | Bimanual UMI grippers, 6 cameras | H.264 · metadata |
| Putting Away Small Clothes | 1408×1280 head · 640×480 wrist | 30 | 4m 04s | Bimanual UMI grippers, 6 cameras | H.264 · metadata |
| Rinsing Utensils at the Sink | 1408×1280 head · 640×480 wrist | 30 | 3m 10s | Bimanual UMI grippers, 6 cameras | H.264 · metadata |
| Shoe Arrangement | 1920×1536 per view | 30 | 26s | Handheld UMI gripper, wrist and chest cameras | H.264 |
Licensing & availability
| Inventory | 70,680 hours · indicative — exact hours confirmed in writing in your quote |
|---|---|
| Non-exclusive licence | Yes — default for all in-stock hours |
| Exclusive licence | On request, subject to existing licences |
| Commercial model training | Included |
| Custom top-up collection | Available when you need more than stock or a new scenario |
| Annotation | Included: subtask segmentation with action labels and start/end timestamps, plus scene, object and action tags per subtask. Custom label specs quoted. |
| Privacy | Samples de-identified before publication; delivery screening per licence terms |
| Delivery | LeRobot v2.1 (MP4 per camera, Parquet trajectories, JSON annotations), to S3 / GCS / Azure or physical drives |
FAQ
What is UMI data?
UMI stands for Universal Manipulation Interface: demonstrations recorded with handheld grippers that match a robot's gripper, with cameras on the grippers and the head. The recordings carry the gripper's trajectory and opening, so they map onto robot actions more directly than video of bare hands.
How is UMI data priced?
Per project. The price depends on hours, scenes and tasks, which camera streams you need, and exclusivity. Tell us what you need and we send a quote with the confirmed hours.
What format is it delivered in?
Bimanual recordings come in LeRobot v2.1 format: one MP4 per camera, Parquet files with per-frame state and action trajectories, and JSON annotations with subtask segments, action labels and timestamps.
Can I see the scene, object and action breakdown?
Yes. We share episode counts and hours per scene (bedroom, kitchen, living room and more), per object category and per action type on request.
