Great post! I agree with the obvious need here, but two points:
1. If tacit knowledge and niche data for *economically valuable tasks* becomes more and more valuable due to its scarcity, then companies possessing this data increasingly have the advantage in negotiations with labs. I don't think realistically most large enterprises with this kind of data will agree to anything other than ZDR, and if they have even a whiff that labs are trying to hack their way around this they will switch vendors in many cases. Fundamentally they shouldn't do so either! It's against their interests in a lot of scenarios. This is exactly why the publishers who made deals with OAI to train on their data in 2023/2024 have now in many cases regretted the deal, because they now value that data much more highly (rationally).
2. There is an obvious solution to these issues: just make models which are capable of learning from deployments once "released from the nest", training on the job to learn from private data *within* the environment of the data owner, without any accrual to the lab originating the model. Better generalisation would also reduce the need to collect data from many sources on the same topic.
Overall I think it's better for the world if the incentives shift in this way, and labs are forced to make breakthroughs in generalisation/sample efficiency/continual learning to continue to grow model performance and thus revenue—rather than trying to convince companies to arguably act against their interests by commoditizing their edge.
Thank you for this thoughtful piece. Granting your premise that the token/param ratio stalls without more data, I'd just widen the escape routes a bit — there seem to be three, not one.
We either learn to need less data (biology hints it's possible — a fruit fly avoids an odor after a single shock pairing, so our data hunger may be an architecture artifact, not a law), unlock the siloed stuff that already exists (your collection thesis — e.g. healthcare and other private lakes), or generate genuinely new grounded data through simulation of physical/biological systems, where the simulation is the new measurement rather than synthetic text a model can't exceed itself with (one could safely say that even a model with "high temperature" output — btw, that was a great tweet — can only stray so far from where its training data sits).
On that third route, a caution aimed not at you but at the broader chatter: people keep claiming models are close to "cracking science." Solving an unsolved math problem like the recent unit-distance progress is real, but it leans on an abundant, digitized literature. Predicting what actually happens inside a cell is a different game precisely because that data is sparse — you can't simulate your way out of a domain you don't yet understand well enough to model. Which is really just your coverage point, pushed one level down.
Hard to say which route dominates, but at this point I'd bet all three matter — and they will regardless of whether we end up calling the result AGI or ASI.
"I really want to drill this in: The speed at which we automate the economy is going to be directly rate-limited by our ability to collect data about it."
This exact issue comes up whenever I talk to companies trying to implement Claude. They can hardly describe their workflows. So much tacit knowledge gets in the way of strong outputs. I wonder how much teaching we'd need to do to get people to be able to describe their workflows before we could collect it.
Interesting. The narrative so far (Dec 2022 - June 2026) seems to be that the model companies are converging toward commoditization and Nvidia is the ultra-valuable monopoly. But this feels like a reversal: the models are heading toward specialization in different directions and as the hardware players catch up that side is becoming the commodity?
I think ‘specialization’ is complicated. For example, I think Anthropic’s RL team has been much more focused much earlier than OpenAI’s RL team on coding quality and SWE-Bench scores, and this leads to serious differences in Claude vs. ChatGPT’s capabilities. On the flip side, It’s clear OpenAI cares a lot about math, directs more effort towards it, and you see much better scores there.
I don’t expect OpenAI or Anthropic to diverge too far into specializing to different domains: Both companies care too much about both coding and math to just leave one hanging, but it’s more about resource allocation, team quality, and how these things are developed, which is deeply tied to data.
And as the labs prioritize certain domains & capabilities, there is a real opportunity to headstart a persistent lead, as Anthropic seems to have with coding for a while, that can be difficult to overcome.
No comment on the hardware side, don’t know nearly enough to compare on that front.
Great post! I agree with the obvious need here, but two points:
1. If tacit knowledge and niche data for *economically valuable tasks* becomes more and more valuable due to its scarcity, then companies possessing this data increasingly have the advantage in negotiations with labs. I don't think realistically most large enterprises with this kind of data will agree to anything other than ZDR, and if they have even a whiff that labs are trying to hack their way around this they will switch vendors in many cases. Fundamentally they shouldn't do so either! It's against their interests in a lot of scenarios. This is exactly why the publishers who made deals with OAI to train on their data in 2023/2024 have now in many cases regretted the deal, because they now value that data much more highly (rationally).
2. There is an obvious solution to these issues: just make models which are capable of learning from deployments once "released from the nest", training on the job to learn from private data *within* the environment of the data owner, without any accrual to the lab originating the model. Better generalisation would also reduce the need to collect data from many sources on the same topic.
Overall I think it's better for the world if the incentives shift in this way, and labs are forced to make breakthroughs in generalisation/sample efficiency/continual learning to continue to grow model performance and thus revenue—rather than trying to convince companies to arguably act against their interests by commoditizing their edge.
Thank you for this thoughtful piece. Granting your premise that the token/param ratio stalls without more data, I'd just widen the escape routes a bit — there seem to be three, not one.
We either learn to need less data (biology hints it's possible — a fruit fly avoids an odor after a single shock pairing, so our data hunger may be an architecture artifact, not a law), unlock the siloed stuff that already exists (your collection thesis — e.g. healthcare and other private lakes), or generate genuinely new grounded data through simulation of physical/biological systems, where the simulation is the new measurement rather than synthetic text a model can't exceed itself with (one could safely say that even a model with "high temperature" output — btw, that was a great tweet — can only stray so far from where its training data sits).
On that third route, a caution aimed not at you but at the broader chatter: people keep claiming models are close to "cracking science." Solving an unsolved math problem like the recent unit-distance progress is real, but it leans on an abundant, digitized literature. Predicting what actually happens inside a cell is a different game precisely because that data is sparse — you can't simulate your way out of a domain you don't yet understand well enough to model. Which is really just your coverage point, pushed one level down.
Hard to say which route dominates, but at this point I'd bet all three matter — and they will regardless of whether we end up calling the result AGI or ASI.
This is awesome.
"I really want to drill this in: The speed at which we automate the economy is going to be directly rate-limited by our ability to collect data about it."
This exact issue comes up whenever I talk to companies trying to implement Claude. They can hardly describe their workflows. So much tacit knowledge gets in the way of strong outputs. I wonder how much teaching we'd need to do to get people to be able to describe their workflows before we could collect it.
Interesting. The narrative so far (Dec 2022 - June 2026) seems to be that the model companies are converging toward commoditization and Nvidia is the ultra-valuable monopoly. But this feels like a reversal: the models are heading toward specialization in different directions and as the hardware players catch up that side is becoming the commodity?
Data as a moat seems like it should last a while.
I think ‘specialization’ is complicated. For example, I think Anthropic’s RL team has been much more focused much earlier than OpenAI’s RL team on coding quality and SWE-Bench scores, and this leads to serious differences in Claude vs. ChatGPT’s capabilities. On the flip side, It’s clear OpenAI cares a lot about math, directs more effort towards it, and you see much better scores there.
I don’t expect OpenAI or Anthropic to diverge too far into specializing to different domains: Both companies care too much about both coding and math to just leave one hanging, but it’s more about resource allocation, team quality, and how these things are developed, which is deeply tied to data.
And as the labs prioritize certain domains & capabilities, there is a real opportunity to headstart a persistent lead, as Anthropic seems to have with coding for a while, that can be difficult to overcome.
No comment on the hardware side, don’t know nearly enough to compare on that front.