Listen "Figure AI Robots 🤖 // OpenAI Leaks GPT-5 🤫 // Scaling Language Models 📈"
Episode Synopsis
Figure, a leading AI robotics company, is making significant advancements in creating robots that can perceive their environment, make decisions, and take action, all in a way that aligns with human expectations.
OpenAI may have accidentally leaked details about a new AI model called GPT-4.5 Turbo, which could level the playing field with Google's AI model Gemini.
Two papers explore the development and evaluation of large language models (LLMs) for code-related tasks, and propose simple and scalable strategies to continually pre-train LLMs to save on compute.
Another paper investigates scaling in the over-trained regime and relates language model perplexity to downstream task performance via a power law, providing useful insights into how language models can be scaled and evaluated more effectively.
Contact: [email protected]
Timestamps:
00:34 Introduction
01:28 Figure AI
03:02 Did OpenAI just accidentally leak the next big ChatGPT upgrade?
04:48 Gradio's Grog
05:47 Fake sponsor
07:37 Simple and Scalable Strategies to Continually Pre-train Large Language Models
09:31 LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
11:11 Language models scale reliably with over-training and on downstream tasks
13:07 Outro
OpenAI may have accidentally leaked details about a new AI model called GPT-4.5 Turbo, which could level the playing field with Google's AI model Gemini.
Two papers explore the development and evaluation of large language models (LLMs) for code-related tasks, and propose simple and scalable strategies to continually pre-train LLMs to save on compute.
Another paper investigates scaling in the over-trained regime and relates language model perplexity to downstream task performance via a power law, providing useful insights into how language models can be scaled and evaluated more effectively.
Contact: [email protected]
Timestamps:
00:34 Introduction
01:28 Figure AI
03:02 Did OpenAI just accidentally leak the next big ChatGPT upgrade?
04:48 Gradio's Grog
05:47 Fake sponsor
07:37 Simple and Scalable Strategies to Continually Pre-train Large Language Models
09:31 LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
11:11 Language models scale reliably with over-training and on downstream tasks
13:07 Outro
More episodes of the podcast GPT Reviews
OpenAI's 'Strawberry' AI 🚀 // World's Fastest AI Inference ⚡ // Photo-realistic 3D Avatars 🎨
28/08/2024
Grok-2's Speed & Accuracy 🚀 // OpenAI's Transparency Push 🗳️ // LlamaDuo for Local LLMs 🔄
27/08/2024
Amazon Cloud Chief Spicy Takes 🚀 // Zuckerberg's AI Vision 📈 // Multimodal Models for Safety 🔒
23/08/2024
Grok-2 Beta Release 🚀 // Apple's $1,000 Home Robot 🏡 // ChemVLM Breakthrough in Chemistry 🔬
15/08/2024
Gemini Live AI Assistant 📱 // OpenAI’s Coding Benchmark ✅ // LongWriter’s 10K Word Generation ✍️
14/08/2024
ZARZA We are Zarza, the prestigious firm behind major projects in information technology.