Nvidia releases groot n1, a 2 billion parameter hierarchical foundation model for humanoid robots.
2
Its really interesting how they were able to use generative video in this dataset to create extra executions. I've pretty vocally been a skeptic of this working in the past. But i think this recipe will be hard for many people to apply...
3
Since from the groot paper you can see they actually need to fine tune on their environment first! I really wonder if this actually helps with generalization. It allows them to basically create a bunch more in-distribution trajectories w/ language labels.
4
Results numbers in simulation and in the real world. I like the focus on reproducible robot benchmarks like robocasa here - compare:
5
Gemini robotics, when comparing against a pi0 baseline, are all on real world tasks. Kind of a shame because real world tasks, it's hard to see exactly how much variation there is, it's hard to assess results and what they actually mean. But...
6
Google does have results comparing to other papers, first of all for pointing:
7
And second of all, for Aloha 2 sim. Weird that there's no comparison against pi0 here but there are at other points in this paper. So where are we?
8
Basically, we now have three open (or at least publicly available) robot foundation models: pi0, Gemini robotics, and groot n1, and *i have no idea at all which to use for anything.* we need chatbot arena or mmlu for robotics real bad.