OpenAI's Nuclear Threat Team ☢️ // Demystifying CLIP Data 🧩 // Trillion-Parameter Model Compression 📏

27/10/2023 18 min

Listen "OpenAI's Nuclear Threat Team ☢️ // Demystifying CLIP Data 🧩 // Trillion-Parameter Model Compression 📏"

Episode Synopsis

OpenAI's new team tackling nuclear threats. The demystification of CLIP data from Meta Research and the Data Provenance Explorer, a tool that provides transparency and accountability in AI datasets. The team also delves into research papers on detecting pretraining data, quantized Transformers, and compression of trillion-parameter models.
Contact:  [email protected]
Timestamps:
00:34 Introduction
02:25 OpenAI forms team to study ‘catastrophic’ AI risks, including nuclear threats
04:34 Demistifying CLIP data from Meta Research
06:27 Data Provenance Explorer
08:33 Fake sponsor
11:11 Detecting Pretraining Data from Large Language Models
12:44 LLM-FP4: 4-Bit Floating-Point Quantized Transformers
14:28 QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models
16:34 Outro

More episodes of the podcast GPT Reviews