In the spirit of Feynman this superb blog post, by none other than Stephen Wolfram, gives a lucid explanation of what is going on under the hood of the latest tech phenomenon.
The short answer is “it’s maths”.
“But in the end, the remarkable thing is that all these operations—individually as simple as they are—can somehow together manage to do such a good “human-like” job of generating text. It has to be emphasized again that (at least so far as we know) there’s no “ultimate theoretical reason” why anything like this should work. And in fact, as we’ll discuss, I think we have to view this as a—potentially surprising—scientific discovery: that somehow in a neural net like ChatGPT’s it’s possible to capture the essence of what human brains manage to do in generating language.”
