The highly anticipated "Sora 2," or rather "Sora Turbo," has finally made its debut. This is likely the most anticipated part of OpenAI's long and tedious 12-day carnival. However, in the ten months since the initial release of Sora, the world of AI models has undergone massive changes, and user expectations have steadily eroded over time.
Just as the vast majority of people cannot perceive the difference between the o1 model and GPT-4o without a frame-by-frame comparison, most people cannot see the progress Sora Turbo has made relative to Sora.
OpenAI repeatedly claims that the "Scaling Law" is still valid. But is this actually the case? Did models truly "hit a wall" in 2024?
In short, they did, but this wall is not a "model wall" or a "Transformer wall." Instead, it is a "compute wall," a "data wall," and a "cost wall."
Objectively speaking, even though ten months have passed since the stunning debut of the first version of Sora, OpenAI may still be unable to prepare enough data and compute power to realize the "Scaling Law" that continues to work in theory. NVIDIA, Microsoft, and a host of suppliers are still working overtime to overcome difficulties in deploying the next generation of superclusters under the Blackwell architecture. OpenAI's collaborative teams are still doing their best to prepare even more massive amounts of high-quality training data.
Regardless of the magnitude of compute and data, an increase of at least an order of magnitude in combination is required to reach a definitive conclusion on whether the "Scaling Law" continues to function.
However, the cost to prove or disprove this is too high, both in terms of capital and time: we need stable, high-load power (small nuclear power?), massive Blackwell clusters (100,000 GPUs?), and significantly more data (whether through systematic labeling of human data or synthetic data)...
This also seems like a prisoner's dilemma, one that belongs to the few players chasing AGI. In this predicament, following the existing path still seems to be the one with the highest certainty. So, if compute, energy, and the market have all responded sufficiently, what have we underestimated?
Data.
Data is still far, far, far from enough.