Open Source Models:

As we have explored before, there are ongoing questions regarding Open vs. closed source AI models.

🚨 This week, Meta released its newest open-source Llama Model, a competitor to closed AI systems like OpenAI’s ChatGPT.

Lama 3:

Community and Open Source Focus: Meta continues its commitment to the open-source community by making Llama 3 widely accessible and encouraging community feedback and innovation.

Model Specifications: The initial release includes models with 8 billion and 70 billion parameters, showcasing advanced reasoning and coding capabilities, and setting new industry benchmarks.

Coming Capabilities: “More advanced reasoning, like the ability to craft longer multi-step plans, will follow in subsequent versions…Versions planned for release in the coming months will also be capable of "multimodality," meaning they can generate both text and images” per Inc

Llama 3 Performance:

• The MMLU benchmark: is a set of 15,000 multiple-choice questions across 57 topics, to test the general knowledge of Models

  • Llama 3 scored 66.6

• AGIEval English 3-5 Shot Test: giving an AI model a sequence of 3 to 5 examples (or "shots") in Middle English, and then asking it to continue or respond in the same style

  • Llama 3 scored 45.9

• BIG-bench Hard Test 3-shot CoT: Providing an AI model with three example tasks that demonstrate step-by-step reasoning (Chain of Thought) for solving complex problems, and then assessing the model's ability to generalize this reasoning approach to a new, similar task.

  • Llama 3 scored 61.1

• With the 70B Parameter model scoring even higher on these benchmarks.

Open Source and Safety:

There are ongoing questions over the safety of open source models as closed source models allow for a controlled environment for them to be deployed.

• System-Level Safety Approach: Llama 3 incorporates a system-level approach that empowers developers by making them central to the design process, allowing them to tailor AI behavior to their specific needs, enhancing both utility and safety.

• Red Teaming and Fine-Tuning: rigorous red teaming, where human experts and automated systems generate adversarial prompts to test and improve the model's responses “For instance, we apply comprehensive testing to assess risks of misuse related to Chemical, Biological, Cyber Security, and other risk areas” -Meta

Why this Matters:

Llama 3 is a big step in advancing open source models.

Such high-performance models being open source democratizes access to AI. Meta is empowering developers, researchers, and even high school students to create innovative applications that can benefit society.