The pace of AI R&D continues to heat up as costs inevitably rise. What does this mean for the future of frontier AI research?
This year’s Stanford University’s AI Index Report 2024 confirmed what we already knew about AI. It’s hot. We saw significant increases in academic publications, patents, and open-source contributions dedicated to artificial intelligence. And corporations are all over it, too. Corporate interest in AI has also surged, with the term “AI” being mentioned on nearly half of the S&P 500’s most recent Q1 earnings calls, according to an analysis from Factset. While much of the AI Index report aligns with expectations, some findings from the R&D side were particularly noteworthy. One interesting trend was the growing divide between industry and academia in frontier AI research.
Industry vs. Academia: A Growing Divide in Frontier AI Research
Although academia contributed 81.1% of the AI publications in 2022, industry dominated frontier AI research by a clear margin. According to the report, industry-led frontier AI research in 2023 will produce 51 new machine learning models compared to the contribution of 15 new models in academia. The industry’s dominant position is largely due to the escalating computational and financial resources required for cutting-edge AI development.
The Skyrocketing Costs of AI Model Training
Training costs for state-of-the-art AI models also reportedly reached unprecedented levels. Stanford’s AI Index team partnered with Epoch AI, an AI research institute, to develop more robust estimates of AI model training expenses. These estimates were calculated using cloud computing rental rates as a foundation for cost assessment. The analysis also incorporated key factors such as the model’s training duration, the type and quantity of hardware used, the hardware’s utilization rate, and the market value of the training hardware at the time of the model’s development. And the numbers are staggering. OpenAI’s GPT-4 is estimated to have cost $78 million to train, while Google’s Gemini Ultra, one of the latest industry-leading models, is estimated at $191 million. To put this in perspective, it cost Google just $930 to train its Transformer model back in 2017.
Petaflops and Power
When we consider these figures in terms of computational power and calculating speed, the expense makes a bit more sense. A petaflop (PFLOPS) represents one quadrillion floating-point operations per second. According to Stanford’s AI report, the original 2017 Transformer required “around 7,400 petaFLOPs” for training. Compare this with the much more powerful Google Gemini Ultra, which “required 50 billion petaFLOPs.” This exponential increase in computational requirements helps explain the dramatic rise in training costs for cutting-edge AI models—and why AI frontier research is primarily led by industry juggernauts with seemingly unlimited financial resources.
The Financial Reality of Cutting-Edge AI
It’s clear that cutting edge AI research and development requires serious financial resources. Corporations like Google and Microsoft certainly have the deep pockets and investment incentives necessary to push AI to its limits (not to mention their ability to actually create and define new limits) and solidify their positions in our future world of work and play. But what does this concentration of power mean for the democratization of AI?
The success and rise of “Big AI” giants undoubtedly mean success for us all—but it also means greater barriers to entry for small upstarts and potential positive disruptors as the financial chasm between AI’s established players and newcomers grows ever wider. Certainly, we’ll see a monopolization of advanced AI capabilities by a few tech giants—and possibly a narrower focus on commercially viable applications rather than research and exploration for exploration’s sake.
Tightening the regulatory environment to deploy AI more safely and competitively might only exacerbate the issue. Increased regulation, while necessary, could further tilt the already lopsided playing field in favor of Big AI. These giants would likely be the only ones able to afford the costs of lobbying, building, and designing around Washington’s new maze of AI regulations and policies.
Who Will Control AI Development?
Although the rapid advancement of AI capabilities is incredibly exciting, it brings with it significant challenges. The growing divide between industry and academia, along with the skyrocketing costs of frontier research, are no doubt ushering in one of the most interesting and complex challenges of our time. So, as academia hands off the torch to industry, we should all consider the very real questions this scenario raises about who or what owns and controls the future landscape of AI development. As with most things, the power of the purse may ultimately dictate the direction of progress.
