Training vs. Inference and GPU Demand

  • Everyone is trying to figure out what AI means for GPU demand.
  • It’s hard as the true picture is muddied by providers investing in their own customers, demand-pull forward, and strategic buying ahead of having a real use case (see Saudi, UAE, U.K.)
  • Confounding all this is Meta releasing Llama 2 for almost free, followed most recently by its coding version (by far the most useful application of AI so far).
  • This matters because training is a lot more GPU-intensive than inference. Free models mean less training needed. This specifically matters for Nvidia’s H100 chip (which by the way weigh over 30 kgs!).
  • Qualcomm actually thinks processing might happen right in our phones (they of course would benefit most from this).
  • Eventually, a lot of the AI processing will move over to the device for several use cases. The advantages of doing it on the device are very straightforward. Cost, of course, is a massive advantage. It’s — in some ways, it’s sunk cost. You bought the device. It’s sitting there in your pocket. It could be processing at the same time when it’s sitting there. So that’s the first one. Second is latency. You don’t have to go back to the cloud, privacy and security, there’s data that’s user-specific that doesn’t need to go to the cloud when you’re running it on the device. But beyond all of these, we see a different set of use cases playing out on the device.” Qualcomm CFO Akash Palkhiwala (via The Transcript).

On-Demand Audio Milestone

  • At the end of 2015, a scant seven and one-half years ago, the margin between linear listening and on-demand listening was 38 percentage points. But drop by drop, quarter by quarter and year by year, the margin was erased, and now on-demand leads.
  • “linear” audio refers to – radio over the air, radio streams, Pandora’s free radio service, satellite radio, etc. and “on-demand” audio – paid streaming, podcasts, owned music, etc.

The Rate Cycle and Bank Net Interest margins

  • Excellent analysis of how banks got themselves in a bad position this rate-rising cycle.
  • So much to learn – capitulation led banks into buying MBS that suffer from negative convexity “trapping” (by rising rates extending weighted average lives) banks into low-yielding assets, not enough capital to take losses, held-to-maturity accounting choice requiring marking everything to market (“tainting the book”), the head fake from a higher portion of floating assets pulling forward net interest margin gains.
  • The result – a worrying state of affairs with funding rates having a long way to go hurting earnings for a long time. This isn’t good for the credit outlook either.

Bundling and Unbundling

  • Are said to be the two business models in media.
  • Has the time for bundling come?
  • Disney is planning to charge $35 for its standalone ESPN service.
  • As a result, The Information now calculates that after all the prices rises for streaming a permutation of the main “channels” starts to cost $90 per month … the price of Comcast’s cable package.

AI Semis

  • Three excellent questions about the future of semiconductors and AI.
  • First, is AI additive to the semis market? Will purchases of AI chips increase overall demand for semis, or will they merely replace purchases of other chips?
  • How will the market for AI Inference semis play out? How big will it be and who will win share?
  • Can Nvidia be displaced from the market dominance it currently enjoys?
  • Some view on answers here.

Podcast Business and Spotify

  • Podcast advertising leads to pretty good returns for brands.
  • After conducting a study with 250 advertisers and marketers, it says two-thirds (67%) of podcast ad buyers say that every $1 spent on podcasts returns between $4 and $6 for their brands.
  • Yet SPOT is struggling to capture this – why?
  • This blog post covers a lot of reasons. For example:
  • “[What’s] most misunderstood about Spotify is Spotify doesn’t get to monetize all the podcast content that they have. So in the most recent quarterly earnings report, they say that they had 5 million podcasts on their platform, but 99.9% of those podcasts, Spotify does not get to monetize.
WordPress Cookie Notice by Real Cookie Banner