Rendered at 20:16:25 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
stult 7 minutes ago [-]
I have generally been bullish on data centers independent of how AI demand/load evolves. People will find a way to use that compute, even if it isn't the precise use we expect. As scale increases and the price per computation comes down, we will be able to brute force solutions to problems that otherwise would be intractable or prohibitively expensive to solve. And there are effectively an infinite set of those problems.
This tracks the evolution of how we use cloud computing and GPUs over the last 20 years. Cloud computing was originally just about saving companies from needing to maintain their own server racks, but has unlocked previously unserviced demand by allowing people to build an app or service and scale it to meet rapidly rising use without needing to invest a ton upfront in hardware. Suddenly a hobbyist could spin something up in their spare time that previously required many thousands of dollars of investment. Or when someone wants to run a single large scale computation, they can do it without needing to waste capital on maintaining idling servers to meet an occasional demand spike, like when a company I used to work at moved from running atmospheric calculations on a server in a closet to the cloud and were able to achieve double digit accuracy increases with the increased scale, while spending less overall on batch computing jobs.
GPUs, as the name implies, were created for graphics, and primarily for gaming graphics, but then more or less accidentally ended up enabling the present AI boom, which depends on a scale of computation that would have been impossible with older CPU architectures. Maybe someone at some point predicted this, but I think for the vast majority of people, it was extremely surprising that a niche gaming product would enable an industrial revolution level technological leap forward.
LLMs are just one way that increased compute scale unlocks seemingly magical results, but they are far from the only example and I have no doubt that there are many unknown examples remaining to be discovered yet.
XorNot 5 minutes ago [-]
The eventual utility of the Internet did not stop a lot of people losing a lot of money in the dotcom crash.
bryanlarsen 33 minutes ago [-]
Any data center that can remain profitable selling open source tokens at commodity prices will be fine. Any data center that relies on OpenAI/Anthropic level token prices and margins might be in trouble. After clearing their debts through bankruptcy, they'd likely be quite profitable selling open source tokens at commodity prices.
nomel 24 minutes ago [-]
I think there's still some low-hanging fruit with thin clients and colocated-to-AI applications. I don't think the math for beefy personal computers is going to hold up, for most use cases.
bix6 14 minutes ago [-]
Quite profitable at commodity prices? I don’t buy that. And all of them are building with debt / equity that expects high token prices?
bryanlarsen 11 minutes ago [-]
After bankruptcy they likely have no debt, so can out-compete those that didn't go through bankruptcy. It's the bankruptcy that makes them profitable -- it's a common pattern in nascent commodity industries.
nemomarx 3 minutes ago [-]
Bad news for wall Street though if they have to go bankrupt first?
I mean the physical hardware will be fine but the owners and investors should be worried then right
Apes 16 minutes ago [-]
If it costs you more to generate the tokens that the market is willing to pay for those tokens, then not even bankruptcy will save any of the costs invested in one of these datacenters.
If the cutting edge OpenAI token prices are $80 per 1M token, and the open source tokens are $1 per 1M token, that's a huge gap of "this will never be able to make money under any scenario if the bubble bursts" that will catch a lot of these new datacenters. No one will run a datacenter that costs $5 per 1M token to sell at $1 per 1M token even if the debts are cleared.
alfalfasprout 11 minutes ago [-]
assuming demand remains elevated and growing, maybe. But spend on AI is pretty stratospheric right now... companies are already starting to clamp down on spend. This makes you really wonder if there will be sufficient demand at current commodity prices for eg; OSS models to justify all these data centers.
bryanlarsen 4 minutes ago [-]
Yes, I believe so. Using a sibling commenter's number of frontier models being 80X the price of commodity models, I think companies who switch will spend a minority of their savings to increase their token usage and only pocket the majority of the savings.
Apes 35 minutes ago [-]
I'll believe it the day SpaceX - the AI company that is mostly making money selling datacenter compute using gas turbines for energy - takes a single day 90% or greater drop in stock price.
Zigurd 29 minutes ago [-]
Currently nobody knows when the first big financial crisis is fully locked in. For example, if OpenAI can't close another round, and they default on their contracts with Oracle, there's your sign. Until then it looks like everybody is enjoying the communal hallucination.
TacticalCoder 20 minutes ago [-]
> Currently nobody knows when the first big financial crisis is fully locked in.
What do you mean the "first" big? 1929? 2001? 2008?
Do you mean 1929 wasn't a big financial crisis and that, this time, we'll have the first "real" big financial crisis?
I'm confused.
hattmall 14 minutes ago [-]
They mean the roughly 25 year tech sector run that's now culminating in the irrational exuberance of AI. There have been some hits already and the result of those is market cap consolidation of the largest companies. Not sure what the number is but the the 10 largest companies make up a huge percent of the entire S&P and most have extreme exposure to the same risks. The sector has been boosted by the NVIDIA circular financing as well but at some point there is a limit. Although they are angling for a pre-bailout with all the AI is going to kill humanity fear mongering. The only savior for the sector will be the government, the question is if the government steps in before or after an organic collapse.
Zigurd 17 minutes ago [-]
I mean the first AI financial crisis at an AI company. I did provide an example. First, the investor cash has to dry up, then the AI data centers have to start getting itchy about what those multi billion dollar contracts are actually worth.
Dependance 28 minutes ago [-]
Be careful what you wish for. The repercussions might be titanical.
Apes 25 minutes ago [-]
We're already seeing repercussions from an economy that has been retooled not to actually produce anything of value, but to produce more air to fill up the largest economic bubble in the history of the world.
National debt through the roof, inflation through the roof, PHD and research programs gutted, non-ai startups dead and unfunded for the last 4 years. These are just a few things that have been sacrificed on the altar of this bubble - there's far more I haven't recounted.
We're already in a widespread long term economic collapse, but the delusion just hasn't broken yet.
hdhdjdif 13 minutes ago [-]
hmmm havent you heard of agi? rokos basilisk? its vewy vewy dangerous
lil-lugger 32 minutes ago [-]
Is there anyone serious who thinks that the future is local models anyway? All computers used to be the size of rooms like these data centers and then they got smaller and faster until the home computer came. Is that not a possibility down the line as we improve efficiency of the models and increase compute?
Zetaphor 18 minutes ago [-]
I have a shoebox sized computer (Framework Desktop) running Qwen 3.8 Flash Next. It has completely replaced my use of proprietary models in my personal life. 6 months ago I would have told you this was impossible. Based on the current trajectory I expect 6 months from now I'll have a Mythos class model at home. The best part is not having to concern myself with token cost has unlocked all kinds of experimentation and use cases. I have been pushing over a billion tokens per week for multiple weeks now, all for the $52/year it costs to keep this machine running 24/7
1q12h 11 minutes ago [-]
Great news for Omarchy and the merchants at Spotify! Democratize now! Replace all software engineers!
Seriously, do you believe the garbage you wrote?
Zetaphor 51 seconds ago [-]
What does Omarchy or Spotify have to do with anything I just said?
I am a software engineer, and using this software in my personal and professional work has lead me to a very different conclusion, but everyone is entitled to their opinion.
captainbland 12 minutes ago [-]
They scale with compute so even if today's frontier model equivalents work on future desktop hardware then the big servers will still have bigger and better models.
Zigurd 24 minutes ago [-]
AI budgets and AI pricing have too much squish in them currently. Frontier models are being sold at a loss, and AI budgets are experimental. And there is still a whiff of FOMO in the air.
Also, Google and Facebook are still spending like drunken sailors. Nobody has stubbed their toe on hard limitations yet. So yes of course people will figure out how to optimize the cost of AI in their products. Just probably not this year.
johnbellone 7 minutes ago [-]
Google and Facebook have profitable businesses.
stult 24 minutes ago [-]
Do you mean is there anyone who thinks that the future is NOT local models anyway? Since that seems to be the gist of your other points
Analemma_ 24 minutes ago [-]
I have no strong opinion on whether the endgame of AI services is local or in-cloud, but I think your historical analogy is pretty suspect: it's true that computers got smaller and faster, but it's also true that most people have shifted most of their workloads from local and on-prem to datacenters since the turn of the century. Why would AI be an exception?
lenerdenator 16 minutes ago [-]
Like everything else, "it depends".
There will be people who want to host things on-device. At some point, you could probably do most day-to-day tasks with a Siri-like agent, so you don't necessarily need it to be on a datacenter rack somewhere.
More complex tasks being run quickly opens up a choice: insanely beefy individual devices, on-prem hosting, or cloud hosting, whether that be some data center running FOSS models, or ones from people like Anthropic or OpenAI.
Beefy hardware for individual users? Not cost-effective. Could have people share that hardware by putting it in a data center. Do you want to operate that data center? For proven business cases, sure, why not? If you're still working out what your scale will be, maybe you ask the Googles, Amazons, or Microsofts of the world to rent you the hardware so you don't have wasted or too little capacity.
The real question is, how much value is there in a few companies that talk about how their eventual goal is to create AGI as opposed to just giving you enough intelligence to augment your current workers?
The answer is "probably not enough to justify more than one company having a valuation of over a trillion dollars, and that's generous".
latchkey 23 minutes ago [-]
Home computers are still nowhere close to as powerful as a $500k+ 10kW server full of 1.5TB of HBM GPU compute.
Zetaphor 12 minutes ago [-]
You don't need a data center to get real work done. Not every task requires "PhD level intelligence"
latchkey 8 minutes ago [-]
That's the usual response, along with "you can't compete with free". But, look at how much money the frontier models are printing, it is obvious that the bell curve of usefulness is still centered around them.
Let's also not forget that even the open models are not getting smaller, they are getting larger. Of course, you can distill them down into something that will fit on smaller compute, but at the end of the day, the data centers of compute, still play a huge role.
bobthepanda 19 minutes ago [-]
One of these companies wanted an IPO valuing them at over $50B. To put in perspective what kind of crazy number that is, the entirety of Visa raised a valuation of $34B; and this company's parent, Softbank, had an IPO valuation of $64B.
This tracks the evolution of how we use cloud computing and GPUs over the last 20 years. Cloud computing was originally just about saving companies from needing to maintain their own server racks, but has unlocked previously unserviced demand by allowing people to build an app or service and scale it to meet rapidly rising use without needing to invest a ton upfront in hardware. Suddenly a hobbyist could spin something up in their spare time that previously required many thousands of dollars of investment. Or when someone wants to run a single large scale computation, they can do it without needing to waste capital on maintaining idling servers to meet an occasional demand spike, like when a company I used to work at moved from running atmospheric calculations on a server in a closet to the cloud and were able to achieve double digit accuracy increases with the increased scale, while spending less overall on batch computing jobs.
GPUs, as the name implies, were created for graphics, and primarily for gaming graphics, but then more or less accidentally ended up enabling the present AI boom, which depends on a scale of computation that would have been impossible with older CPU architectures. Maybe someone at some point predicted this, but I think for the vast majority of people, it was extremely surprising that a niche gaming product would enable an industrial revolution level technological leap forward.
LLMs are just one way that increased compute scale unlocks seemingly magical results, but they are far from the only example and I have no doubt that there are many unknown examples remaining to be discovered yet.
I mean the physical hardware will be fine but the owners and investors should be worried then right
If the cutting edge OpenAI token prices are $80 per 1M token, and the open source tokens are $1 per 1M token, that's a huge gap of "this will never be able to make money under any scenario if the bubble bursts" that will catch a lot of these new datacenters. No one will run a datacenter that costs $5 per 1M token to sell at $1 per 1M token even if the debts are cleared.
What do you mean the "first" big? 1929? 2001? 2008?
Do you mean 1929 wasn't a big financial crisis and that, this time, we'll have the first "real" big financial crisis?
I'm confused.
National debt through the roof, inflation through the roof, PHD and research programs gutted, non-ai startups dead and unfunded for the last 4 years. These are just a few things that have been sacrificed on the altar of this bubble - there's far more I haven't recounted.
We're already in a widespread long term economic collapse, but the delusion just hasn't broken yet.
Seriously, do you believe the garbage you wrote?
I am a software engineer, and using this software in my personal and professional work has lead me to a very different conclusion, but everyone is entitled to their opinion.
Also, Google and Facebook are still spending like drunken sailors. Nobody has stubbed their toe on hard limitations yet. So yes of course people will figure out how to optimize the cost of AI in their products. Just probably not this year.
There will be people who want to host things on-device. At some point, you could probably do most day-to-day tasks with a Siri-like agent, so you don't necessarily need it to be on a datacenter rack somewhere.
More complex tasks being run quickly opens up a choice: insanely beefy individual devices, on-prem hosting, or cloud hosting, whether that be some data center running FOSS models, or ones from people like Anthropic or OpenAI.
Beefy hardware for individual users? Not cost-effective. Could have people share that hardware by putting it in a data center. Do you want to operate that data center? For proven business cases, sure, why not? If you're still working out what your scale will be, maybe you ask the Googles, Amazons, or Microsofts of the world to rent you the hardware so you don't have wasted or too little capacity.
The real question is, how much value is there in a few companies that talk about how their eventual goal is to create AGI as opposed to just giving you enough intelligence to augment your current workers?
The answer is "probably not enough to justify more than one company having a valuation of over a trillion dollars, and that's generous".
Let's also not forget that even the open models are not getting smaller, they are getting larger. Of course, you can distill them down into something that will fit on smaller compute, but at the end of the day, the data centers of compute, still play a huge role.
This just doesn't pass the smell test.