This work was partially done by an automated research scaffold developed at Redwood Research. For this project, all of the experiment ideas were designed by a human and a human wrote the write up. The AI mostly just executed on the experiment ideas. We think this project is slightly below the level of rigor of a mid-MATS research update, and the research scaffold was not very helpful for this project. More discussion of AI usage is in the Appendix. 💻 Codebase If we want to train a classifier that distinguishes whether a passage is code or prose, we can do so by gathering samples of code and of prose, and training the classifier to distinguish between the two classes. Unfortunately, this might not work if the data hides a spurious correlation. If all the code is in Spanish and all of the prose is in English, then the classifier might learn to predict Spanish vs. English instead of code vs. prose. We find that this happens in practice: when we fine-tune an LLM to classify between Spanish code and English prose and evaluate on Spanish prose or English code, it generalizes to predicting the language rather than the domain. [...] ---Outline:(04:45) The setup(08:13) Measuring feature strength(12:49) Activation differences and feature strength(15:21) Explicit prompting(17:17) Diagonal vs antidiagonal pairs(18:47) Conclusion(20:20) Appendix(20:24) AI involvement(22:17) The 37 features(23:57) The ranking is robust across measurements(30:30) Intensity moves the fine-tune, not the probe(32:13) Safety features in Qwen3.6-27B(33:05) Counterexamples and training on a third cell(34:46) Near ties often produce degenerate fine-tunes(35:24) Related work The original text contained 4 footnotes which were omitted from this narration. ---
First published:
August 11th, 2026

Source:
https://www.lesswrong.com/posts/qpJYNjQ6wdWRxbykL/measuring-spurious-correlations-with-feature-strength
---
Narrated by TYPE III AUDIO.
---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ver todos los episodios