It was a huge week of AI and robotics news. So I summarized...

@adcock_brett
Brett Adcock@adcock_brett
22 views Sep 29, 2024 ~2 min read
Advertisement
1
It was a huge week of AI and robotics news.

So I summarized everything announced by OpenAI, Google DeepMind, Meta, Tencent, Stability AI, and more.

Here's everything you need to know and how to make sense out of it:
2
OpenAI rolled out an enhanced Advanced Voice Mode to all ChatGPT Plus and Teams subscribers this week

Recommend trying if you haven't yet

They added Custom Instructions, Memory, and five new 'nature-inspired' voices
0:44
3
Meta announced Orion: "the world’s most advanced AR glasses ever made"

On top of the AR, the glasses use Meta AI's multimodal capabilities to understand everything you see and have advanced features like voice control and hand tracking
1:01
4
Google announced the rollout of two new production-ready Gemini AI models: Gemini-1.5-Pro-002 and Gemini-1.5-Flash-002

Best part of the announcement was 50% reduced price on 1.5 Pro, and 2x higher rate limits on Flash/3x higher on 1.5 Pro

@rowancheung
Rowan Cheung@rowancheung
Exclusive: Google just released two upgraded Gemini 1.5 models, 1.5-Pro-002 and 1.5-Flash-002 — achieving new, state-of-the-art performance across math.

I sat down with Logan Kilpatrick (@OfficialLoganK) to discuss the models, AI agents, AGI, and more.

Timestamps:

00:00 Intro
01:01 Google rolls out two new Gemini models
2:18 What makes the new models so unique
3:40 Math improvements
8:20 Examples of Gemini 1.5 in the real world
10:19 Future problems AI could solve
12:54 Advantages of Gemini 1.5
16:50 Where beginners can get started
19:01 Advice to thrive in the new age of AI
22:02 Turning notes into podcasts
26:30 AI agents: Definitions and what they can do
31:22 What’s the final form for agents?
33:26 Proactive AI agent systems
36:00 Context windows in the agent era
41:01 AGI’s definition in Logan’s eyes
42:05 The current bottlenecks towards AGI
5
Google DeepMind unveiled AlphaChip, an AI system that designs chips by using reinforcement learning

→ AI design chips, which train more advanced AI, which designs even better chips

Superhuman chip layouts can now be built in hours rather than months
@JeffDean
Jeff Dean@JeffDean
Welcome, AlphaChip!

Today, we are sharing some exciting updates on our work published in @Nature in 2021 on using reinforcement learning for ASIC chip floorplanning and layout. We’re also naming this work AlphaChip.

Since we first published this work, our use of this approach internally has grown significantly. It has now been used for multiple generations of TPU chips (TPU v5e, TPU v5p, and Trillium), with AlphaChip placing an increasing number of blocks and with larger wirelength reductions vs. human experts from generation to generation:

AlphaChip has also been used with excellent results for other chips across Alphabet, including Google’s Axion chip, an Arm-based general-purpose data center CPU.

In 2022, as a companion to the Nature paper, we open-sourced the code for the AlphaChip algorithms described in the Nature paper (see link below). Since then, external researchers could use this repository to pre-train on a variety of chip blocks and then apply the pre-trained model to new blocks, as was done and described in our original paper.

Today we’re also releasing a pre-trained AlphaChip checkpoint for the open source release that makes it easier for external users to get started using AlphaChip for their own chip designs.

Original Nature paper w/ wonderful joint first authors @Azaliamirh + @annadgoldie, and @mnyazgan, @joesmemory, @ESonghori, @ShenWangURC, @xylophi, @efjohnson, @pathomkar, @Azade_na, @PakJiwoo, Andy Tong, @kavyasrinivas23, @willhang_, @emretuncer, @quocleix, @JamesLaudon, @rh00, Roger Carpenter, and myself):
nature.com/articles/s4158… (PDF: rdcu.be/cmedX)

Today’s Addendum to the paper published in Nature: nature.com/articles/s4158… (same authors)

AlphaChip blog post: deepmind.google/discover/blog/…

Open source release: github.com/google-researc…

Pre-trained checkpoint: github.com/google-researc…

Three things we have observed in the external community are described in the Nature Addendum: (1) not doing any pre-training (circumventing the learning aspects of our method by removing its ability to learn from prior experience) (2) not training to convergence (standard practice in ML methods), and (3) using fewer computational resources than described in our Nature paper (using fewer resources is likely to harm performance, or require running for considerably longer to achieve the same performance).

Pre-training the model for it to learn the craft of chip layout and to be able to generalize to new designs is an important part of our method. The pre-training process requires some effort to perform, since one has to find representative blocks and then run a lengthy computational process to pre-train the model to be good at placing those blocks. To avoid external users having to perform this process and make it easier for the external community to use AlphaChip, today we are releasing an AlphaChip model checkpoint pre-trained on 20 TPU blocks. This will enable users to get good zero-shot performance and faster convergence for novel blocks right out of the box. (For best results, however, we continue to recommend that developers pre-train on their own in-distribution blocks, and we provide a tutorial on how to perform pre-training with our open-source repository: see the Addendum).

Many organizations have used AlphaChip as a building block for their own chip design efforts. For example, MediaTek, one of the top chip design companies in the world, extended AlphaChip to accelerate development of their most advanced chips (e.g. the Dimensity Flagship 5G used in Samsung mobile phones), while improving power, performance and chip area.

We’re very excited about the increasing impact of AlphaChip internally and externally, and we look forward to continued work in this space to make custom higher performance, more efficient, and more capable chips dramatically easier to design and build.
Media image
6
Meta announced Llama 3.2, the latest iteration of the open source model family

Updates include two medium-sized vision LLMs (11B and 90B) and two lightweight, text-only models (1B and 3B) that fit onto mobile devices and are SOTA for many on-device use cases
Media image
Media image
7
Meta AI, the consumer chatbot, is also now multimodal, meaning it can 'see' images

But Meta AI is also now allowing users to edit photos using AI, for, e.g. removing an object, adding a hat, changing backgrounds, etc.

Hilarious use case:

@diegocabezas01
Diego | AI 🚀 - e/acc@diegocabezas01
Meta AI Llama 3.2 can edit selected parts of the image
Media image
Media image
8
Tencent Robotics released their wheeled home service robot called "The Five"

The wheeled legs turn into retractable feet for climbing stairs, and it can fold itself for better storage
1:12
9
Google upgraded NotebookLM/Audio Overviews, adding support for YouTube videos and audio files

Audio Overviews turns notes, PDFs, Google Docs, and more into AI-generated podcasts

Here’s a wild example:
4:03
10
James Cameron joined the board of directors at Stability AI

The film director sees the convergence of generative AI and CGI as “the next wave” in visual media creation

Hollywood is starting to pay attention to AI progress
@StabilityAI
Stability AI@StabilityAI
Today, our CEO, @premakkaraju, announced that legendary filmmaker, technology innovator, and visual effects pioneer, James Cameron, has joined the Stability AI Board of Directors.

Cameron’s addition represents a significant step forward in our mission to transform visual media. His artist-centric perspective, paired with his business and technical acumen, will be invaluable as we continue to build a full stack AI pipeline that unlocks new opportunities for creators to tell stories in ways once unimaginable.

Read more here: stability.ai/news/james-cam…
Media image
11
EA demoed a new AI concept for user-generated video game content.

It works using 3D assets, code, gameplay hours, telemetry events, EA-trained custom models to remix games and asset libraries in real time.
4:14
12
We're hiring engineers @Figure_robot:

→ Manufacturing
→ AI Roles (many)
→ Electrical Engineers
→ Embedded / Firmware
→ Systems Integration & Test

See all open roles and apply here:
figure.ai/careers

@adcock_brett
Brett Adcock@adcock_brett
We're building robots that can learn and reason like humans in real-world scenarios

Looking for hardcore engineers:
→ Manufacturing
→ AI Roles (many)
→ Electrical Engineers
→ Embedded / Firmware
→ Systems Integration & Test
Media image
13
@Figure_robot That's it for this week's AI and Robotics breakdown.

I share the latest research every week, so follow me @adcock_brett for more.

If you found this valuable, consider a like/retweet to spread the word.
@adcock_brett
Brett Adcock@adcock_brett
It was a huge week of AI and robotics news.

So I summarized everything announced by OpenAI, Google DeepMind, Meta, Tencent, Stability AI, and more.

Here's everything you need to know and how to make sense out of it:
Actions
What You Can Do
  • Export as PDF or Markdown
  • Batch Export to Notion
  • Bookmark & Highlight
  • Screenshot Tweet
Create Free Account

Includes 7-day Premium trial

Advertisement