On April 5, local time, Meta released the first large model versions of its latest open source artificial intelligence software Llama 4, Llama 4 Scout and Llama 4 Maverick. These are also the company’s two most powerful AI large language models (LLMs) to date.
However, Meta said that a more powerful large model named Llama 4 Behemoth is still being trained, which will act as a “teacher for new models” in Meta’s mixed expert model (MoE) architecture.
This is also the first MoE model architecture adopted by Meta based on Llama software. In the MoE model, a single token only activates a portion of the total parameters. Meta said that the MoE architecture is more computationally efficient during training and inference, and provides higher quality than dense models under a fixed training FLOPs budget.
Take the Llama 4 Maverick model as an example, which has 17 billion activation parameters and 400 billion total parameters. Meta uses alternating dense layers and mixture of experts (MoE) layers to improve inference efficiency. In this way, although all parameters are stored in memory, only a portion of the total parameters are activated when serving these models.
The release of Meta’s latest large model also means that the investment competition among technology giants in the wave of generative artificial intelligence has further escalated. It was previously reported that Meta postponed the release of the latest version of the large model because during the development process, Llama 4 did not meet Meta’s expectations in terms of technical benchmarks, especially in reasoning and mathematical tasks.
“Our goal is to build the world’s leading AI, open source it, and make it universally available so that everyone in the world can benefit from it,” said Meta founder and CEO Mark Zuckerberg in a video on Instagram. “I think open source AI software will build leading models, and with Llama 4, that’s starting to happen.”
Google CEO Sundar Pichai also congratulated Llama on the release of its latest model on social media. He said: “The world of artificial intelligence will never be boring! Congratulations to the Llama team, keep going!”
Additionally, Meta will host its first LlamaCon AI conference on April 29. The company also expects to launch a standalone application for the Meta AI chatbot in the second quarter of this year.
Chris Cox, Meta’s chief product officer, said last month that Llama 4 will advance the development of AI agents with higher levels of reasoning and action. These AI agents will be able to go online and perform a wide range of tasks useful to consumers and businesses.
meanwhile Meta is investing heavily in AI infrastructure, with plans to spend $65 billion this year to expand its AI infrastructure, which could include a nearly $1 billion data center project in central Wisconsin.
However, just before Meta released its new model, Joelle Pineau, the company’s head of artificial intelligence research, announced his departure last week. Pineau is one of Meta’s top artificial intelligence researchers and has led the company’s Basic Artificial Intelligence Research (FAIR) department since 2023, responsible for the company’s cutting-edge computer science-related research, including Meta’s open source Llama series of AI models and other technologies.