March 22, 2025

Lesson 9

The original version of the search chapter did not delve into retrieval-augmented generation (RAG). We've added a paragraph about how poorly sourcing works under RAG with the major commercial LLMs, based on a Columbia Review of Journalism report.

More recent models can also pull information from web searches and use these search queries to provide sources. At present, however, this process suffers from the same fabrications and confident false assertions that characterize LLMs more generally. One study found that commercial LLMs cite incorrect or non-existent sources from 37% (Perplexity.ai) to 94% (Grok) of the time.

March 9, 2025

Lesson 8
Stefan Ciobaca pointed out that it's not advisable to run any code that you don't understand because it could do harm to your system or even incorporate malware. The more critical your system and the more important it is to protect data privacy, the more cautious one should be about using an LLM as a coding assistant. We've modified our coding example accordingly.

Request help with writing computer code to read the information in a file.

Read the code and make sure you understand it*; then cCompile the code and see if your program imports the data successfully.

*We think it's risky to run LLM-derived code that you don't understand. It could erase important data, introduce malware, or cause other forms of harm.

March 7, 2025

Lesson 3
Karen Hausdoerffer had a comment that we thought was so insightful that it deserved mention in our discussion sci-fi visions of artificial intelligence and how they differ from current large language models.

And another key difference: think about any artificial intelligence from any science fiction world you like. 2001: A Space Odyssey, Star Wars, Wall-E — we have always imagined artificial intelligences as single sentient beings, rather than all-pervasive information systems that subtly infiltrate human efforts to make sense of world.

February 23, 2025

Lesson 18
We were struck by by Bluesky discussions of news outlets using LLMs to write their stories and then attributing quotations to people who never said those things. In light of this, we felt it was important to add some discussion of how it's not just propaganda that drives misinformation; profit motives do as well and can have a similar effect on the electorate.

Sometimes the motive may be profit rather than propaganda, but the consequences can be similar.

Low-quality news stories written by LLM are further undermining our trust in the media. Small-time scammers and once-storied magazines alike have used LLMs to write vapid and often false stories that get published as news. They also cross lines that even disreputable outlets have traditionally avoided. For example, a Texas A&M professor and public figure recently reported that AI-authored news stories are referencing non-existent interviews and attributing quotations to her that she never said.

It is unsurprising that an LLM that generates likely strings of text and has no underlying factual model would do this. But as the internet becomes increasingly littered with fabricated content which subsequently populates search results and is used to train future generations of AI models, we risk losing any sort of anchoring to truth. That, we feel, poses an existential risk to democracy.

February 13, 2025

Lesson 1
The original version of the paragraph at right claimed that LLMs don't engage in logical reasoning. While we believe this is accurate with regard to the basic models, a number of readers pointed out that Chain-of-Thought models engage in processes that one could argue are indeed forms of logical reasoning. We could try to carve out a definition of logical reasoning that excludes chain of thought models, but that's not a hill are eager to die on. Suffice to say that if they reason logically, they do so differently than humans do.

By thinking about what LLMs do, we can better understand what they don’t do. They don’t engage in logical reasoning. They don't reason the way that people do. They don’t have any sort of embodied understanding of the world. They don’t even have a fundamental sense of truth and falsehood.