OpenAI + Figure conversations with humans, on end-to-end neural...

@adcock_brett
Brett Adcock@adcock_brett
19 views Mar 15, 2024 ~1 min read
Advertisement
1
OpenAI + Figure

conversations with humans, on end-to-end neural networks:

→ OpenAI is providing visual reasoning & language understanding
→ Figure's neural networks are delivering fast, low level, dexterous robot actions

(thread below)
2:34
2
Here's some background:

Two weeks ago, we announced Figure + OpenAI are joining forces to push the boundaries of robot learning

Together we are developing next generation AI models for our humanoid robots

It’s been 13 days & we're excited to share today's update
Media image
3
@OpenAI @Figure_robot The video is showing end-to-end neural networks

There is no teleop

Also, this was filmed at 1.0x speed and shot continuously

As you can see from the video, there’s been a dramatic speed-up of the robot, we are starting to approach human speed
4
@OpenAI @Figure_robot Figure’s onboard cameras feed into a large vision-language model (VLM) trained by OpenAI

Figure's neural nets also take images in at 10hz through cameras on the robot

The neural net is then outputting 24 degree of freedom actions at 200hz
5
@OpenAI @Figure_robot In addition to building leading AI, Figure has also vertically integrated basically everything

We have hardcore engineers designing:

- Motors
- Firmware
- Thermals
- Electronics
- Middleware OS
- Battery Systems
- Actuator Sensors
- Mechanical & Structures
0:14
6
@OpenAI @Figure_robot AI work shown here was led by @coreylynch and Toki Migimatsu, big congrats!

Our goal is to train a world model to operate humanoid robots at the billion-unit level. Follow our journey: @Figure_robot

We are scaling up our AI team, join us: figure.ai/careers
7
@OpenAI @Figure_robot @coreylynch good technical deep dive by the AI lead @coreylynch
@coreylynch
Corey Lynch@coreylynch
We are now having full conversations with Figure 01, thanks to our partnership with OpenAI.

Our robot can:
- describe its visual experience
- plan future actions
- reflect on its memory
- explain its reasoning verbally
Technical deep-dive 🧵:
Actions
What You Can Do
  • Export as PDF or Markdown
  • Batch Export to Notion
  • Bookmark & Highlight
  • Screenshot Tweet
Create Free Account

Includes 7-day Premium trial

Advertisement