Stanford Professor Tatsu Hashimoto on AI Biases and Improving LLM Performance
Ep. 11, Unsupervised Learning
On Unsupervised Learning we sat down with Prof. Tatsu Hashimoto to provide an academic perspective on the LLM space. See the full episode below!
Our discussion with Professor Hashimoto, who runs the lab behind the famous Alpaca and AlpacaFarm projects, spanned from talking about the impact of different components within the LLM training pipeline, to the role of academia and everything in between. You can listen to the full conversation on Spotify, Apple, and YouTube, or read our highlights below!
⚡ Highlight 1: No free lunch in model performance
We’ve seen companies like Cerebras and MosaicML (recently acquired by Databricks) showcase their ability to train compute-efficient LLMs. We discussed key factors when determining the next wave of high-quality, low cost models.
“When you finetune a really good base model, things feel easy because with a little bit of data, you're going a long way. And that's because in some sense, Meta has paid for the compute and data cost of making these models good. If you want to take a 13 billion parameter model and make it cover all the different things that a 65 billion parameter model is going to do, you're going to be paying tremendous data and compute costs.”
Check out our piece explaining the rapid decline in LLM train costs and why Databricks bought MosaicML here!
⚡ Highlight 2: What really matters when training an LLM
When LLMs started to gain traction within the AI academic community, many researchers found it difficult to understand the impact different parts of the LLM training pipeline had on the model’s performance. Prof. Hashimoto’s AlpacaFarm provides researchers the capability to simulate the LLM training process.
“Every piece is important, but if we prioritize which ones are most important, the base LM is really important in the sense that that's where a lot of the knowledge is coming from. We're not giving it enough instruct data that’s really changing the underlying knowledge score of the model, the base LM is handling a lot of that. The instruction following data is the next most important because that makes the model do roughly the right thing. Then reinforcement learning, it turns out, does have a substantial impact. It's not zero, but the way in which it has impact is subtle. We found that it's able to do things like control the length of the answer, or control how often it uses lists or not.”
⚡ Highlight 3: The role of academia in the LLM space
As companies like OpenAI, Google, and Facebook lead the way in developing next generation LLMs, there are still huge opportunities for academics to contribute to the space.
“I think one of the advantages of academia is that our incentives are very different from the incentives of a company…I think the work that we're doing trying to figure out what kinds of opinions are in these LLMs for better public discourse, and thinking about what kinds of opinions these LLMs should reflect…sort of like stepping back and saying, okay, can we make it possible for many other people to get in on the kinds of language model research so we can have a much better diversity of opinions.”
Check out Prof. Hashimoto’s publication Whose Opinions Do Language Models Reflect? and corresponding Opinion QA repo!