ProductPackageCartonContainerTool
Products, packages, containers and toolsPRODUCT · PACKAGE · CONTAINER · TOOL
Retail × Embodied Intelligence
Retail Reality Is the Training Ground for Embodied Intelligence
REAL STORES · REAL TASKS · REAL INTERACTIONS
20,000+Data Hours
100+Capture Devices
20,000+SKUs Covered
02 / Field Evidence
Captured in Real Work,
Not Staged for a Demo
Collected in live stores, on real shelves, with real products. First-person operators move through everyday tasks while the data preserves natural light, occlusion, reflections and customer traffic.
Enter
Live context
Live context
Record
Continuous task
Continuous task
Verify
Task outcome
Task outcome
Trace
Dataset version
Dataset version
The environment stays complexShelves, lighting, traffic, occlusion and real routes remain intact.
The task stays continuousContext, outcome, failure and recovery stay connected.
03 / Per-Site Collection Output
Real Work.Measurable Output.
Cumulative hours reveal the data output of each collection site.
- Average cumulative hours per site
- ≈1,813hours
- Highest cumulative hours at one site
- 2,969.20hours
TOP 8
Leading sites by cumulative collection hours
04 / Retail Task Graph
Retail Task Graph
Real store videos are decomposed into reusable action chains — replenishment, processing, checkout and backroom fulfilment — then grounded in observable events.
Root
Retail Task GraphVIDEO-GROUNDED ASSET
Work / Video workflows
Shelf replenishmentSnacks · Beer · Daily stock
Fresh merchandisingFace · Align · Restore
Bakery prepFrozen dough · Arrange · Load
Deli processingPrep · Portion · Box
Checkout & paymentScan · Collect · Bag
Backroom pickingVerify · Pick · Pack
Generalized Task
Move Large Load / RestockBackroom trolley to shelf
Face / Align / RestoreReturn the shelf to standard
Arrange / LoadBakery preparation
Unload / Box / LabelBakery output and packing
Cut / Weigh / SealFresh preparation chain
Portion / Box / WrapDeli processing and packing
Scan / Collect / BagContinuous human-device interaction
Pick / Verify / PackTerminal-guided fulfilment
Event Leaf
Move rear stock forward and alignFresh shelf facing
Cut fruit → Weigh → PackFresh preparation
Durian → Weigh → Open → SealFresh preparation
Arrange dough → Load rackBakery preparation
Unload → Box → LabelBakery output
Scan → Collect → BagCheckout
Portion deli food → Seal boxDeli processing
Verify terminal list → Pick → PackBackroom fulfilment
● New tasks & events keep growing →
05 / Data Acquisition Pyramid
A Data Pyramid for Embodied Pretraining and Scene Fine-Tuning
Real first- and third-person video preserves field context. Simulation and synthesis expand scale and long-tail coverage. UMI bridges human manipulation, while teleoperation aligns robot actions with task outcomes.
01 / REALFirst- / Third-Person CaptureLive-store video, task context and continuous work.
02 / SIMSimulation & Synthetic DataScene diversity, failure cases and long-tail pretraining.
03 / UMIUMI DataPortable capture of fine manipulation and object interaction.
04 / TELEOPTeleoperation DataRobot actions, states and task outcomes aligned.
DATA PYRAMID
Teleoperation Data
UMI Data
Simulation & Synthetic Data
First- / Third-Person Capture
06 / Object Universe × Physical Properties
20,000+ SKUs.
A Taxonomy of Physical Reality.
20,000+Interacted SKU Coverage
Flexible & deformablePouches · Film · Soft packs · Dough
Rigid & case-packedCartons · Bottles · Hard packs · Trays
Fragile & force-sensitiveBaked goods · Eggs · Glass · Produce
Irregular & naturalLeafy greens · Roots · Produce · Bread
07 / Scene Network × Human–Robot Flywheel
A Store Network Where Human–Robot Collaboration Compounds into Data
Scene coverage creates the validation base. Demonstration, deployment, intervention and event return keep the model moving forward.
China Retail Scene Network
Dmall customer scene networkMap approval No. GS(2023)2767 · City lights are illustrative and pending validation
Real scene
Structured data
Model training
Store deployment
HUMAN × ROBOTHuman judgment→Robot execution→Event return↺