This is a crosspost from our report website for Why LLMs Aren't Scientists Yet: Lessons from Four Autonomous Research Attempts. This report details the work behind our LLM-written paper "The Consistency Confound: Why Stronger Alignment Can Break Black-Box Jailbreak Detection" accepted at Agents4Science 2025, the first scientific conference requiring AI as primary author, where it passed both AI and human review. TL;DR We built 6 AI agents using Gemini 2.5 Pro and Claude Code, mapped to stages of the scientific workflow from idea to hypothesis generation, experiment execution, evaluation and paper writing. We tested our agents on 4 research ideas across ML sub-domains such as Multi-Agent RL, World Models, and AI Safety. 3 ideas failed during implementation or evaluation. Only 1 succeeded and was published at Agents4Science 2025. We document 6 recurring failure modes: bias toward training data, implementation drift under pressure, memory/context degradation, overexcitement that declares success despite obvious failures, and gaps in domain intelligence and scientific taste. We also derive 4 design principles for more robust AI scientist systems, discuss the limitations of training and evaluation data for future autonomous science, and release all prompts, artifacts, and outputs at github.com/Lossfunk/ai-scientist-artefacts-v1. Problem Definition and System Overview We [...] ---Outline:(00:37) TL;DR(01:44) Problem Definition and System Overview(03:55) Our Agents4Science 2025 Submission(06:25) Observed Failure Modes and Mitigation(06:40) 1. Bias on Training Data(07:25) 2. Implementation Drift(08:04) 3. Memory and Context Issues(08:55) 4. Overexcitement and Eureka Instinct(09:50) 5. & 6. Lack of Domain Intelligence and Scientific Taste(10:44) Design Takeaways for AI Scientist Systems(10:57) 1. Start Abstract, Ground Later(11:15) 2. Verify Everything(11:35) 3. Plan for Failure and Recovery(11:57) 4. Log Everything(12:14) Limitations and Discussion ---
First published:
January 8th, 2026

Source:
https://www.lesswrong.com/posts/y7TpjDtKFcJSGzunm/why-llms-aren-t-scientists-yet
---
Narrated by TYPE III AUDIO.
---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ver todos los episodios