The latest 2023 report is worth a flick (all 160 slides).
This graph, for example, shows the largest Nvidia H100 chip clusters – interesting to see TSLA there, who also run the 4th largest A100 cluster in the world.
Or see Slide 76 which suggests that Nvidia’s advantage (the use of its chips in academic papers) continues to increase.
Interview between Goldman Chair/CEO David Solomon and former CEO/Chair of Google on the future of Generative AI is worth a read.
“In general, the disruption occurs first in the industries that have the most amount of money and the least amount of regulation.”
Pairs nicely with this analysis of the latest batch of Y-combinator companies that are using AI/ML startups (139 in total!) and what areas they are working on.
It is a bit technical but left me with a feeling that though LLMs are a big breakthrough, they have big limitations.
Models beyond the autoregressive LLM that start to mimic some of the planning and reasoning required to rival human intelligence are a lot more complicated with not-so-neat solutions.
In the spirit of Feynman this superb blog post, by none other than Stephen Wolfram, gives a lucid explanation of what is going on under the hood of the latest tech phenomenon.
The short answer is “it’s maths”.
“But in the end, the remarkable thing is that all these operations—individually as simple as they are—can somehow together manage to do such a good “human-like” job of generating text. It has to be emphasized again that (at least so far as we know) there’s no “ultimate theoretical reason” why anything like this should work. And in fact, as we’ll discuss, I think we have to view this as a—potentially surprising—scientific discovery: that somehow in a neural net like ChatGPT’s it’s possible to capture the essence of what human brains manage to do in generating language.”
This dataset tracks the flow of talent in AI around the world.
The chart shows the top 25 institutions for AI research.
“The United States has a large lead over all other countries in top-tier AI research, with nearly 60% of top-tier researchers working for American universities and companies. The US lead is built on attracting international talent, with more than two-thirds of the top-tier AI researchers working in the United States having received undergraduate degrees in other countries.”
