Each row represents a different model. The three bottom rows are Llama models from Meta. And as you can see, Llama 3.1 70B—a ...

Specifically, the paper estimates that Llama 3.1 70B has memorized 42 percent of the first Harry Potter book well enough to reproduce 50-token excerpts at least half the time. (I’ll unpack how this ...