AI for the Next Era | Greylock

Authors

Reid Hoffman

Sam Altman

Tags

Top Highlights

• "I'm a big believer that the only real driver of human progress and economic growth over the long term is the societal structure that enables scientific progress, and then scientific progress itself."

• I think language models are going to go just much, much further than people think, and we’re very excited to see what happens there. I think it’s what a lot of people say about running out of compute, running out of data. That’s all true. But I think there’s so much algorithmic progress to come that we’re going to have a very exciting time.

• I doubt we’ll still be using the transformers in five years. I hope we’re not. I hope we find something way better. But the transformers obviously have been remarkable. So I think it’s important to always look for where I am going to find the next totally new paradigm.

• what I think will happen is there’ll be a whole new set of startups that take an existing very large model of the future and tune it, which is not just fine tuning, all of the things you can do.

• I think if I had time to do something else, I would be so excited to go after a bio company right now. I think you can just do amazing things there.

• the alignment problem is: how do we build AGI that does what is in the best interest of humanity? How do we make sure that humanity gets to determine the future of humanity?

• right now, if you use GPT whatever, it’s stuck in the time that it was trained. And the more you use it, it doesn’t get any better and all of that. I think we’ll get that changed. So I’m very excited about all of that.

• I think [AI] is going to just seep in everywhere. My basic model of the next decade is that the marginal cost of intelligence and the marginal cost of energy are going to trend rapidly towards zero, surprisingly far.

• The synthetic bio companies that I’ve seen that have been most interesting are the ones that find a way to make the cycle time super fast. And that benefits an AI that’s giving you a lot of good ideas, but you’ve still got to test them, which is where things are right now.

• I’m a huge believer in startups that the thing you want is low costs and fast cycle times. And if you have those, you can then compete as a startup against the big incumbents.

• "I don't think all the deep biological things will be changed by AI. I think we will still really care about interaction with other people. I think the stuff that people cared about 50,000 years ago is more likely to be the stuff that people care about 100 years from now than 100 years ago."

• I don’t think we’ll still be doing prompt engineering in five years. And this’ll be integrated everywhere. Either with text or voice, depending on the context, you will just interface in language and get the computer to do whatever you want.

• I think the fundamental interface will be natural language.

• the artist will still do the best with image generation but not because they figured out to add this one magic word at the end of it. Because they were just able to articulate it with a creative eye that I don’t have.

• for me, AGI is basically the equivalent of a median human that you could hire as a coworker. And then they could do anything that you’d be happy with a remote coworker doing just behind a computer, which includes learning how to go be a doctor, learning how to go be a very competent coder.

• What is the new social contract? My guess is that the things that we’ll have to figure out are how we think about fairly distributing wealth, access to AGI systems, which will be the commodity of the realm, and governance, how we collectively decide what they can do, what they don’t do, things like that.

• in some sense, the startups will train their own models, just not from the beginning. They will take base models that are hugely trained with a gigantic amount of compute and data, and then they will train on top of those to create the model for each vertical. So in some sense, they are training their own models, just not from scratch. But they’re doing the 1% of training that really matters for whatever this use case is going to be.

• That will be a significant change to the way that technological development, scientific development happens. So those are the two that I think are huge now and lead to just an acceleration of progress.

• And I think all these questions about new knowledge generation (how do we really advance humanity?) I think there will be systems that can help us with that.

• “If you just think about that alone as a way to unlock the applications people will be able to build, that would be a huge victory for all of us and just a massive step forward and a genuine technological revolution,” says Altman. “I think that these powerful models will be one of the genuine new technological platforms, which we haven’t really had since mobile. And there’s always an explosion of new companies right after.”