According to reports, Meta intends to soon release a new language model called Llama 3 that can compete with the powerful GPT-4 by OpenAI, but which would remain freely accessible under the license lama.
This was revealed by Jason Wei, an OpenAI engineer who previously worked at Google Brain, during a social event organized by Meta for the Generative AI group. Wei claims to have overheard a conversation about it Meta would now have enough processing power to train Llama 3 and 4. Llama 3 aims to achieve the performance level of GPT-4 while remaining freely available.
Overheard on a Meta GenAI social:
“We calculated that llamas 3 and 4 should be trained. The plan is for Llama-3 to be as good as GPT-4.”
“Wow, if Llama-3 is as good as GPT-4, are you still going to open source it?”
“Yes we will. Sorry folks with alignment.”
– Jason (@agikoala) August 25, 2023
However, this is an unconfirmed rumor and may be incorrect or subject to change. Meta has not issued an official statement on the eventual launch of Llama 3. About five months elapsed between the release of Llama 1 in late February 2023 and the release of Llama 2 in late July 2023 at Meta. GPT-4 has a more sophisticated architecture than standard Llama
GPT-4 probably owes its high performance to the use of a more complex expert mix architecture with 16 expert networks, each with around 111 billion parameters. Therefore, upgrading from Llama 2 to Llama 3 could be more difficult than just increasing the amount of training and could take longer than upgrading from Llama 1 to Llama 2.
llama 2 reaches the level of GPT-3.5 in some applications and is also supported by the open source community fine tune and adding features. For example, the Code Llama model based on Llama 2 achieves the same results as GPT-3.5 and GPT-4 (depending on the measurement type) in the HumanEval benchmark for programming.
However, Meta acknowledges in the Llama 2 article that there is still a big gap in performance with closed models like the GPT-4 and GPT-4 PaLM-2 from Google.
The Financial Times reported in mid-July The main goal of Meta’s Llama models was to break the dominance of OpenAI in the language model market. Meta would try to push llama models as the base technology in this market, similar to that Google broke up with Android in the mobile marketto then suggest more offers. Meta also benefits from the rapid development Templates from the open source community.
OpenAI CEO Sam Altman said in early June 2023 that GPT-5 was still a long way from adoption. Google plans to launch Twinsthe next generation of multimodal models, late 2023 or early 2024.