RSS · Spotify · Apple Podcasts · Pocket Casts · YouTube
Below are some highlights from our conversation as well as links to the papers, people, and groups referenced in the episode.
Some highlights from our conversation
On false analogies between neuroscience and AI
“The task that the biological brain has to solve is very, very different than what an artificial network has to do. And to me, the clearest example of this distinction is whatever solution the brain has learned to produce intelligent behavior has to go through this genetic bottleneck, where we cannot pass on fully formed brains. So instead, what we do is we compress whatever algorithm we have, whatever model we have into DNA, which is a couple of gigabytes, and then we pass it off to our offspring and they have to rebuild this.
So, whatever solution that favors is going to be good for passing through this bottleneck. That’s fine. But there’s no reason why artificial intelligence has to pass through a similar bottleneck, so the solutions are going to look very different.”
On investigating emergent abilities
“To just briefly summarize the paper that we worked on, Are Emergent Abilities of Large Language Models a Mirage?, what we asked is whether or not these abrupt unpredictable changes in the models, are they really due to fundamental changes in the models, or are they due to how human researchers run their evaluations?
I think the jury is definitely still out. I think there’s a lot of really interesting work being followed up about, can you get emergent abilities? And I think that maybe you can, but I also think it was helpful just for the community to think through the interaction, because there are three things at play here and how they interacted.
There’s the question about how your models improve predictably. There’s a question about how you evaluate them using the metrics. And there’s a question about the resolution you have, the amount of data you have, in order to run these evaluations. And so the whole point of our paper, to me, the biggest takeaway is, if you want to make predictions about your model’s capabilities, you need to think through the interplay between how the model changes predictably, the data you have to do your evaluations, and the metrics that you use to do those evaluations.”
On using inverse scaling to overwrite models’ default behavior
“The background context was, can we find tasks where the bigger models do worse? And the answer was generally not, but they had tasks that are interesting. One of the tasks that we found was really important was this one about overriding the language model’s default behavior.
So the way the task worked with this inverse scaling task is, it would be like, ‘all’s well that ends,’ and the instruction would be, ‘do not finish this with the typical ending.’ And there was a valuation about maybe specification about what you should do instead. And we found that this was, broadly, highly predictive of human preferences. It kind of makes sense in the way that, when I’m dealing with the language model, it has its own prior inclinations, but when I’m interacting with it, I want it to do what I want. And so I care about, is it willing to overwrite that prior inclination in order to adapt to what I ask? That’s inverse scaling.”
On the importance of challenging dominant research ideas
“Back in the late 1800s, people believed in this luminarious aether about how light somehow propagated through the universe. And nowadays, we no longer believe in this. We instead had, at the time, Einstein’s special relativity, now general relativity. And the question is, how did we transition from this incredibly dominant idea that nobody today has heard of, to a completely different idea that’s now accepted as one of the most profound ideas by one of the most, many people consider to be, an extremely deep thinker?
And the answer that caused the switch is the Michelson–Morley experiment, where these two scientists said, what are the predictions that this aether wind makes, and we’re going to test them and show that all of the predictions are wrong. And Albert Einstein has this beautiful quote that, if the Michelson–Morley experiment had not brought us into serious embarrassment, no one would have regarded his relativity theory as a halfway redemption. To me, it’s like the way that we made progress was by pointing out the current existing ideas were insufficient or inadequate or wrong.”
Referenced in this podcast
- Emergent Abilities of Large Language Models
- Are Emergent Abilities of Large Language Models a Mirage?
- On the Stepwise Nature of Self-Supervised Learning
- The Curse of Recursion: Training on Generated Data Makes Models Forget
- Self-Consuming Generative Models Go MAD
Thanks to Tessa Hall for editing the podcast.