Video transcripts

Lesson 1

Part 1.

Teddy: So, I read through the first lesson, and my main takeaway was that ChatGPT and large language models are, at their core, text prediction and autocorrect, but on a much more powerful scale. They don't necessarily engage in logical reasoning, they just put words in an order that makes sense to us.

But, for a program that does that, and that doesn't engage in logical reasoning, how is that so good at holding a conversation and seeming so human and being able to write code and things like that?

Carl: Let me lay out some of the pieces I think are important here. So, these are enormous models. They're machine learning models on a scale we've never worked with before.

These are models with a trillion parameters that have been trained on many terabytes of text. So, they're very, very big models. We don't really have a good understanding of what big models can or can't do. Another important piece is that they're using this new architecture called the transformer architecture that has a bunch of technical features that allows these machines to do more than just predicting the next word.

They're kind of thinking about the broader context. Thinking is the wrong term there, but they're taking into account the broader context of the conversation, of what they're saying, of what you've said.

Then another really, sort of third really important piece is the way these things have been trained to actually respond to humans.

If we just had an autocomplete machine, it wouldn't know what does a dialogue look like? What does a question look like? What does a request look like? It would just try to fill out text like if it was just writing a story.

But these systems have been trained through this process of reinforcement learning with human feedback where they've had people spend a long time asking them questions. They give an answer and they say, "no, that's not, you're continuing my question. That's not what I want. I want you to answer the question."

And so these systems have learned through reinforcement learning, another machine learning approach, to give humans the answers that they want to questions. They've learned what humans mean by a question and how humans use a dialogue. And so that's a really, really important piece.

And then the fourth piece I would say is basically that human language is remarkably predictable. It seems like there's an infinite variety of things I can say, but 99.99% of the time I actually continue along quite a well-trodden path with what it is that I'm saying. So I'm much easier to predict than I think I am.

Jevin: Exactly. And one thing that I would add when it comes to that feedback from the usage data is that once you've released these to the world, like OpenAI did and like Google has done with Gemini and a lot of these LLMs, once you've done that, then you've opened up the door to all sorts of new kinds of feedback to learn and then become better at.

Carl: Because you've got millions or even billions of people coming and using your system every day. And you can learn from the questions they're asking it and from whether they seem satisfied with the answers or whether they sort of ask again in a new way. So you're continuing to use reinforcement learning on the increasing numbers of people coming to your platform.

Jevin: That's why maybe some of these tech companies have released it, even knowing that many of the results are not so good because they're in competition when it comes to the usage data with all the other tech companies that are getting a bunch of usage data.

And I think that's really the sort of gold that is out there in this massive race that's going on between these big tech companies and the LLMs that they're developing.

Part 2.

Teddy: Why do you stop yourself from using words like understand and thinking?

Carl: Well, because I constantly fall into the trap of thinking of these large language models as intelligences. For the first 51 years of my life, if I could have a conversation with something, that was an intelligent thing. It was a human, actually, that had thoughts and feelings and sentience and all of that. And all of a sudden, now I live in a world where I can have a conversation with something that for all the world seems to be human and do all of this, and it's not.

It's not intelligent. It doesn't understand anything. It doesn't think. It doesn't have feelings. But I forget that over and over and over again when I talk to these systems.

The systems are, in fact, designed to make you forget that. They speak to you in the first person. They're set up as a chatbot, where you kind of have a conversation that goes back and forth. And when you forget that, you fall into all of these traps of assuming that the system can do things that it can't, that it knows things it can't, then you end up misusing it, being confused, misunderstanding what it can do, and sort of not doing a good job of interacting with an LLM.

Jevin, what about you?

Jevin: It's so hard not to. We constantly check ourselves, we've been talking about this for multiple years, and we talk to students about this. We talk to the public about it. We talk to colleagues that push us even harder than ourselves to not anthropomorphize these things.

But it's incredibly hard because they've been optimized to communicate at human and sort of beyond human level. But when we do anthropomorphize them too much, we become susceptible, and we can fall into traps that we'll talk about later in the course. And sometimes there's high consequence for that. There's research out there that shows when you anthropomorphize, you become more susceptible to the manipulation, the ability for these bots to persuade, to make you sort of change your mind, et cetera, et cetera. (Although changing our mind sometimes can be good.)

The main idea here is that we are up against a very effective communicator, as good as any humans that we've talked to in many ways. And so we have to be on the lookout.

Carl: You and some colleagues wrote what I think is a really important paper that sort of frames these systems as anthropoglossic systems—anthropoglossic meaning "human speaking". So they're systems that are designed to speak like humans, and that does make us vulnerable to all kinds of assumptions, false assumptions about what they are and what they're capable of doing.

Jevin: Yeah, exactly. So our colleagues, Kai Reamer and Sandra Peter, we've had these conversations about how a lot of the attention right now is on their cognitive abilities, these LLMs.

But less attention has been on their communicative abilities, and it's that ability that is their biggest strength and also potentially the biggest threat, I think, to humanity.

Carl: Are they con men? I mean, con men are really good communicators.

Jevin: Yeah, it's a little bit like that. They are great communicators, they can jump into any conversation, they can pretend to know what they're talking about, and they do that. I would say that there are some similarities between a con man and these LLMs.

Teddy: Should I trust them?

Carl: No.

I think you can use them to suggest things to try, suggest questions to ask, suggest things to research, but you can't trust them to give you correct answers.

Lesson 2

Teddy: I am in high school and I write for the paper and we did a story called ChatGP-Teach where we looked at what teachers were using it for. Ninety percent of the teachers at my school use ChatGTP as a teaching resource. My history teacher used it to create all of his slideshows and all of his teaching presentation and then he didn't, you know, add anything else on above on top of that and at some point, you know, read off the slide and be like, oh no, that isn't right.

But then also my English teacher would create reading quizzes for us to see whether or not we read the last few chapters of our book and she would use ChatGTP to make those.

How am I supposed to know? Because I don't feel like it's my responsibility to then go research it and double check what my teachers are teaching. What can I do to make sure that I know what is real for my teachers because I trust them? And when do I have to call BS on?

Carl: I think you're hitting on one of the really important dangers of LLMs and this is people being lazy and using them because it is not your job to know when your teachers are full of shit.

I mean, to some certain degree, if they're making ridiculous arguments, it is your job to see through those and see that their arguments don't hold logically. But if they're putting facts up on PowerPoint slides that aren't true, it's not your job to be having to scrutinize that and fact check the basic details of the material that's being presented to you.

And this is what's so pernicious about this is that they may, you know, think that these systems are better than they are. It's certainly saving them some time, but it leaves you in essentially a hopeless position where you're really struggling to fact check what they're saying. And fact checking is harder than ever because you try to do it online and you go to Google and Google gives you an answer from Gemini that's also wrong. What can you do? Learn how to fact check and we can talk about that, but you shouldn't have to.

Jevin: Let me add something to that.

It may save the teacher some time, but in the end, if we care about getting to the truth, it certainly doesn't save you time because now you feel obligated to check what the teacher has shown you. And in many ways you have to now because somewhere in there, there's probably some bullshit. There is this baseline bullshit that comes from LLMs now, and I don't see that changing anytime soon.

So the net effort might be even greater now that we're in LLMs to get at the truth. So we all save a little bit of time here and there, but in the end, we all have to work that much harder to do the fact checking.

Carl: I'll go one further. It's really irresponsible to teach that way. I mean, it is lazy to use the PowerPoint slides that the textbook company gives you that you could use if you're lazy, but at least those have been vetted by the author.

Jevin: Yes, at least they've been vetted.

Carl: And they're written by a human and they're right. Doing it with an LLM is just inexcusable.

Teddy: I think you bring up this point of net effort, but I would also argue, I don't think most people care. I don't think my history teacher necessarily cares that him cutting a corner, you know, it saves him twenty minutes but adds an hour for all fifteen of his students.
I'm not sure he cares.

Carl: This is the fundamental, like, you know, what we'd call public goods problem or common pool resource problem of LLMs: it saves the person who does it twenty minutes, costs everyone else way more than twenty minutes collectively, but that's not the problem with the person who uses it. And so everyone ends up using it. The information ecosystem ends up getting degraded.

Jevin: It is like the classic sort of common goods problem. And it reminds me when I was a kid, there was a huge national campaign to clean up our highways because there was trash everywhere. And ultimately it is the responsibility of each individual that's driving.

And somehow you have to set this norm and that can come through public service campaigns that can come through, you know, lessons at schools. It can come through, you know, the signs that you see on the highway now. And I think we're at that moment now with information that we have to set this norm or at least make people aware of the cost that is there for everyone else, even though you're saving that twenty minutes.

Carl: People did change their norms about the environment. You don't throw your beer can out the window anymore while you're driving down the highway. Wee learned not to do that.

Jevin: And there's a little bit of throwing out that beer right now with these LLMs because that's what people are doing to save themselves time.