Everyone is trying to figure out what AI means for GPU demand.
It’s hard as the true picture is muddied by providers investing in their own customers, demand-pull forward, and strategic buying ahead of having a real use case (see Saudi, UAE, U.K.)
Confounding all this is Meta releasing Llama 2 for almost free, followed most recently by its coding version (by far the most useful application of AI so far).
This matters because training is a lot more GPU-intensive than inference. Free models mean less training needed. This specifically matters for Nvidia’s H100 chip (which by the way weigh over 30 kgs!).
Qualcomm actually thinks processing might happen right in our phones (they of course would benefit most from this).
“Eventually, a lot of the AI processing will move over to the device for several use cases. The advantages of doing it on the device are very straightforward. Cost, of course, is a massive advantage. It’s — in some ways, it’s sunk cost. You bought the device. It’s sitting there in your pocket. It could be processing at the same time when it’s sitting there. So that’s the first one. Second is latency. You don’t have to go back to the cloud, privacy and security, there’s data that’s user-specific that doesn’t need to go to the cloud when you’re running it on the device. But beyond all of these, we see a different set of use cases playing out on the device.” Qualcomm CFO Akash Palkhiwala (via The Transcript).
Data for the UK suggests we have hit saturation point.
“The growth of SVoD household penetration slowed in 2022, and this continued into early 2023 as the rising cost of living, combined with SVoD service price rises, put greater strain on household budgets.“
Podcast advertising leads to pretty good returns for brands.
“After conducting a study with 250 advertisers and marketers, it says two-thirds (67%) of podcast ad buyers say that every $1 spent on podcasts returns between $4 and $6 for their brands.“
Yet SPOT is struggling to capture this – why?
This blog post covers a lot of reasons. For example:
“[What’s] most misunderstood about Spotify is Spotify doesn’t get to monetize all the podcast content that they have. So in the most recent quarterly earnings report, they say that they had 5 million podcasts on their platform, but 99.9% of those podcasts, Spotify does not get to monetize.“
If you haven’t come across it already, read and marvel at this article by a Wired journalist who took a tour of a TSMC semiconductor fab.
“Every six months, just one of TSMC’s 13 foundries—the redoubtable Fab 18 in Tainan—carves and etches a quintillion transistors for Apple. In the form of these miniature masterpieces, which sit atop microchips, the semiconductor industry churns out more objects in a year than have ever been produced in all the other factories in all the other industries in the history of the world.” (h/t The Diff)
In the spirit of Feynman this superb blog post, by none other than Stephen Wolfram, gives a lucid explanation of what is going on under the hood of the latest tech phenomenon.
The short answer is “it’s maths”.
“But in the end, the remarkable thing is that all these operations—individually as simple as they are—can somehow together manage to do such a good “human-like” job of generating text. It has to be emphasized again that (at least so far as we know) there’s no “ultimate theoretical reason” why anything like this should work. And in fact, as we’ll discuss, I think we have to view this as a—potentially surprising—scientific discovery: that somehow in a neural net like ChatGPT’s it’s possible to capture the essence of what human brains manage to do in generating language.”
The argument that AI is unlikely to be a winner for the middle-ground companies.
Why? “was a feature not a product” – in other words value will either accrue to core AI platforms (e.g. Open AI) or to incumbent software tools with distribution who will just add AI features.
“Adobe will own the AI-based image editing market Office & Google Docs will own the AI-based writing market Salesforce will be the best AI-enabled CRM Shopify the best AI optimization and customer support Zoom the best AI meeting summaries … all with a few API calls“
