Listen "#65 – Katja Grace on Slowing Down AI and Whether the X-Risk Case Holds Up"
Episode Synopsis
Katja Grace is a researcher and writer. She runs AI Impacts, a research project trying to incrementally answer decision-relevant questions about the future of artificial intelligence (AI). Katja blogs primarily at worldspiritsockpuppet, and indirectly at Meteuphoric, Worldly Positions, LessWrong and the EA Forum.
We discuss:
What is AI Impacts working on?
Counterarguments to the basic AI x-risk case
Reasons to doubt that superhuman AI systems will be strongly goal-directed
Reasons to doubt that if goal-directed superhuman AI systems are built, their goals will be bad by human lights
Aren't deep learning systems fairly good at understanding our 'true' intentions?
Reasons to doubt that (misaligned) superhuman AI would overpower humanity
The case for slowing down AI
Is AI really an arms race?
Are there examples from history of valuable technologies being limited or slowed down?
What does Katja think about the recent open letter on pausing giant AI experiments?
Why read George Saunders?
Key links:
World Spirit Sock Puppet (Katja's main blog)
Counterarguments to the basic AI x-risk case
Let's think about slowing down AI
We don't trade with ants
Thank You, Esther Forbes (George Saunders)
You can see more links and a full transcript at hearthisidea.com/episodes/grace.
We discuss:
What is AI Impacts working on?
Counterarguments to the basic AI x-risk case
Reasons to doubt that superhuman AI systems will be strongly goal-directed
Reasons to doubt that if goal-directed superhuman AI systems are built, their goals will be bad by human lights
Aren't deep learning systems fairly good at understanding our 'true' intentions?
Reasons to doubt that (misaligned) superhuman AI would overpower humanity
The case for slowing down AI
Is AI really an arms race?
Are there examples from history of valuable technologies being limited or slowed down?
What does Katja think about the recent open letter on pausing giant AI experiments?
Why read George Saunders?
Key links:
World Spirit Sock Puppet (Katja's main blog)
Counterarguments to the basic AI x-risk case
Let's think about slowing down AI
We don't trade with ants
Thank You, Esther Forbes (George Saunders)
You can see more links and a full transcript at hearthisidea.com/episodes/grace.
More episodes of the podcast Hear This Idea
#84 – Dean Spears on the Case for People
01/11/2025
#83 – Max Smeets on Barriers To Cyberweapons
13/03/2025
#80 – Dan Williams on How Persuasion Works
26/10/2024
#78 – Jacob Trefethen on Global Health R&D
08/09/2024
#77 – Elizabeth Seger on Open Sourcing AI
25/07/2024
#76 – Joe Carlsmith on Scheming AI
16/03/2024
ZARZA We are Zarza, the prestigious firm behind major projects in information technology.