Meta has hit a roadblock with its largest AI project yet. The company is delaying the release of its “Behemoth” AI model, which was supposed to come out this summer. This is the second delay for Behemoth, which was first pushed back from April. Meta says the model is not improving fast enough to meet its high standards.
Behemoth is part of Meta’s Llama family of AI models. These models are open source, meaning anyone can use them for free. They are meant to compete with closed AI systems like those from OpenAI and Google. But building a model as big as Behemoth is proving harder than expected.
Why Behemoth Is Delayed
Meta’s engineers are struggling to make Behemoth significantly better than its smaller cousins. The model has 2 trillion parameters, which are like settings that help it learn. For comparison, GPT-4, OpenAI’s latest model, has about 1.7 trillion parameters. More parameters usually mean a smarter AI, but they also make the model slower and harder to train.
Insiders say Behemoth’s performance in tests is not much better than Meta’s existing models. Mark Zuckerberg and other senior managers have become impatient with the slow progress. The team behind Behemoth faces high expectations, while others are concerned about whether it’ll meet its promise.
Bigger Is Not Always Better
AI developers are discovering that scaling up models isn’t always a recipe for success. Large tech companies are taking longer than expected to release their leading-edge models. Collaborative efforts to build better models are being slowed down by two major challenges.
Collections of top-notch data that can train AI models are quickly becoming unavailable. Dominant language models, such as Behemoth, require vast amounts of data to develop, yet the most valuable data is privately owned. Using it could lead to lawsuits. Second, adding more parameters is giving smaller improvements than before. It is like studying harder but only getting slightly better grades.
What This Means for Businesses
Meta’s open-source models are popular with small companies and researchers because they are free. But most big businesses still pay for closed models from Google or Amazon. These models come with support and are easier to use.
Even though Behemoth is delayed, Meta’s smaller models are still useful. For example, the Llama Scout model can process 10 million tokens, equal to 7.5 million words. This lets it handle very long documents, like entire books. But without Behemoth, Meta may struggle to keep up with rivals promising smarter AI.
Internal Problems at Meta
The holdup is creating internal pressure at Meta. Eleven important members of the Llama model development team have departed over the past year. They’re discussing removing some managers to help progress move more quickly. At the same time, Meta has allocated up to $72 billion for AI development this year.
Zuckerberg has said AI is critical to Meta’s future. The company uses Llama models to power features like smarter ads and content recommendations. But if Behemoth keeps getting delayed, Meta risks falling behind in the AI race.
The Future of AI Development
Making Behemoth reveals how difficult it is to go beyond the current limits of AI. As companies increasingly focus on applications, they are concentrating on improving the utility of AI technology for everyday use. Google’s Gemini model works smoothly with Gmail and Docs, and OpenAI is advancing its efforts to help AI carry out everyday activities. Meta may also have to change its approach in order to succeed. It might be better to develop tools that allow businesses to put AI into action. This could include providing improved training for developers or teaming up with cloud providers.
