Lucy Jane 
Meta aims to foster open AI development with Llama 3, contrasting competitors’ closed methodologies.
On Tuesday, Meta executives unveiled the initial official affirmation and additional information regarding the forthcoming release of Llama 3, an eagerly awaited iteration of the organisation’s open-source large language model, at a prominent AI event in London.
“We expect to begin rolling out our new suite of next-generation foundation models, Llama 3, within the next month, or ideally in a very short period,” Nick Clegg, president of global affairs at Meta, said at Meta AI Day London, as reported by TechCrunch.
The release of Llama 3, according to Clegg, will commence this year and will comprise “a variety of models with distinct functionalities and adaptability.”
Llama 3 is anticipated to be the most sophisticated open-source model available upon its release, as Meta has made substantial investments in its development.
Meta states that 140 billion parameters were used to train the model, which is twice the capacity of Llama 2. In January, Mark Zuckerburg, CEO of Meta, provided hints regarding certain technical aspects.
Zuckerberg stated, “We are constructing a vast compute infrastructure to support our future roadmap, including 350,000 H100s by the end of this year and nearly 600,000 H100 equivalents of compute if other GPUs are included.”
This quantity of computing power is considerably more than OpenAI employed to train GPT-4, an endeavour that reportedly demanded 25,000 GPUs throughout 90 to 100 days.
Additionally, Zuckerberg disclosed that its AI assistant, Meta AI, will be enabled by Llama 3.
Chief Product Officer Chris Cox stated Llama 3 will be incorporated throughout Meta.
“We intend for Llama 3 to fuel various products and experiences throughout our app family,” he explained.
The ramifications of the Llama 3 launch transcend Meta, as the organisation’s guiding principle is to construct it as an open-source prototype, which starkly opposes the closed and proprietary methodology adopted by competitors such as OpenAI with ChatGPT.
Meta’s Commitment to Open AI Development
The platform’s objective with open sourcing language models is to foster an environment conducive to open AI development and establish the Llama family as the cornerstone for an extensive array of applications and tools developed by external researchers and developers.
“It is critical to recognize that innovations are invariably built upon previous contributions from others, which may even be quite similar in nature,” tweeted Yann LeCun, chief of AI research at Meta, last month.
“Open research is crucial for this reason: it accelerates the progress of the discipline for everyone.”
Already, this inclusive philosophy has generated a dynamic community uniting in support of Llama.
Presently, Mistral, Falcon, and Beluga are among the most sophisticated open-source language models; they were developed through the refinement of the Llama 2 foundation model.
On specific benchmarks, a number of these community models have achieved performance comparable to or surpassing that of GPT-3.5.
As an additional open-source foundational model, the publication of Llama-3 likely paves the way for a new generation of LLMs that will further elevate the standard for AI efficiency and quality.