According to executives and researchers leading Meta's AI efforts, who were obsessed with beating OpenAI's GPT-4 model while developing Llama 3. Unsealed court filings Tuesday The company's current AI patent cases include Kadrey v. Meta.
“Honestly… our goal needs to be GPT-4,” Ahmad Al-Dahle, Meta's Vice President of Generative AI, said in an October 2023 message to Meta researcher Hugo Touvron. “We're going to have 64k GPUs. We need to learn how to build boundaries and win this race.”
Although Meta generates open AI models; The company's AI leaders, like Anthropic and OpenAI, have focused more on beating competitors that don't typically release the weights of their models, rather than walling them behind APIs. Meta's executives and researchers hold Anthropic's Claude and OpenAI's GPT-4 as the gold standard to work toward.
One of Meta's biggest rivals, French AI startup Mistral, has been mentioned several times in internal messages, but the tone has been mixed.
“Mistral is peanuts to us,” Al-Dahle said in the message. “We should have done better,” he said later.
Although tech companies are competing against each other with sophisticated AI models these days. These court filings reveal just how competitive Meta's AI leaders really are — and seem. still there. At several points in the message exchange; Meta's AI leads said they were “very aggressive” in getting the right data to train Llama. At one time, In a message to colleagues, the executive even said, “Llama 3 is literally all I care about.”
Meta's executives to ship AI models; Prosecutors in this case allege that they often cut corners in their frenzied race to train copyrighted books in the process.
Touvron noted that the mix of datasets used for Llama 2 was “bad”, and talked about how Meta could use better data sources to improve Llama 3. Touvron and Al-Dahle discussed clearing the way. Use the LibGen dataset that contains copyrighted works. Cengage Learning; Macmillan Learning; From McGraw Hill and Pearson Education.
“Are there the right datasets out there?” Al-Dahle said. “You want to use it but can't for some stupid reason?”
Meta CEO Mark Zuckerberg, Llama's AI models and OpenAI; Google and others have previously said they are trying to close the performance gap between closed models. Internal messages reveal intense pressure within the company.
“This year, Llama 3 is competing with the most advanced models and leading in some areas,” Zuckerberg said. Letter Starting in July 2024 “Starting next year, we expect future Llama models to be the most advanced in the industry.”
Finally when the Meta In April 2024, Llama 3 was released.Open AI model Google; It competes with the top closed models from OpenAI and Anthropic, and outperforms the open options from Mistral. However, The data Meta used to train its models — data Zuckerberg reportedly gave the green light to use despite its copyright status — is facing scrutiny in several ongoing lawsuits.