Figure AI Robots 🤖 // OpenAI Leaks GPT-5 🤫 // Scaling Language Models 📈

18/03/2024 14 min

Listen "Figure AI Robots 🤖 // OpenAI Leaks GPT-5 🤫 // Scaling Language Models 📈"

Episode Synopsis

Figure, a leading AI robotics company, is making significant advancements in creating robots that can perceive their environment, make decisions, and take action, all in a way that aligns with human expectations.
OpenAI may have accidentally leaked details about a new AI model called GPT-4.5 Turbo, which could level the playing field with Google's AI model Gemini.
Two papers explore the development and evaluation of large language models (LLMs) for code-related tasks, and propose simple and scalable strategies to continually pre-train LLMs to save on compute.
Another paper investigates scaling in the over-trained regime and relates language model perplexity to downstream task performance via a power law, providing useful insights into how language models can be scaled and evaluated more effectively.
Contact:  [email protected]
Timestamps:
00:34 Introduction
01:28 Figure AI
03:02 Did OpenAI just accidentally leak the next big ChatGPT upgrade?
04:48 Gradio's Grog
05:47 Fake sponsor
07:37 Simple and Scalable Strategies to Continually Pre-train Large Language Models
09:31 LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
11:11 Language models scale reliably with over-training and on downstream tasks
13:07 Outro

More episodes of the podcast GPT Reviews