Predictions 2024: Open Source LLMs Show Potential to Overtake Closed LLMs

LLMs

In the first of our 2024 AI Predictions Mini-Series, we speculate how open-source LLMs, rapidly innovating throughout 2023, could outperform closed LLMs in capability and the rate of their adoption by developers and organizations. 

What we’re expecting in 2024:

– Open-source Large Language Models (LLMs) like Llama 2 from Meta will surpass the capabilities of their closed counterparts.

– Open-source LLMs will gain popularity due to their accessibility, transparency, and the collaborative efforts of the global AI community.

– Developers and organizations will increasingly turn to open-source LLMs to harness the power of natural language processing and generation.

Open Source LLMs vs. Closed LLMs: What’s the Difference?

Two clear approaches to developing AI LLMs exist: open-source LLMs and closed LLMs. Open-source LLMs are publicly available models that can be used, modified, and improved by anyone. Closed LLMs are proprietary models; generally, the code, training methodology, and software are kept secret.

OpenAI’s ChatGPT-4, Google Bard, and Anthropic’s Claude have closed LLMs. Meta’s Llama 2 is open source. There’s also the United Arab Emirates Technology Innovation Institute’s (TII) Falcom 180B, Abacus AI’s Giraffe, and Mosaic’s MPT-7B, all open-source, to name a few of the most discussed. It’s worth noting that “open-source” can be a grey area. Some LLMs are more open-source than others. 

Open-source LLMs foster collaboration, innovation, and transparency. Developers learn from each other, share code, build on existing work, and solve problems together. Open-source projects allow developers to inspect and audit AI, which can help solve trust issues, increase accountability, and clarify ethical standards. 

Closed LLMs provide robust security and privacy for developers. They are less at risk from malicious actors, and potentially, training data is better protected. The level of accuracy, functionality, and quality expected from corporate developers can reduce bugs and inconsistencies and increase the reliability of closed models that follow strict guidelines. 

Open Source LLMs will Surpass the Capabilities of Closed LLM Competitors

As the first open-source LLMs launched, often as an endeavor to democratize this keystone technology, they often performed poorly and saw extensive criticism. As AI’s breakout year has progressed, so have LLMs, so much so that open-source models could surpass proprietary competitors. AI experts believe Llama 2, created by Meta and Microsoft, is a serious threat to closed LLMs like GPT-4, and it consistently outperforms its open-source competitors. 

One study shared by Cobus Greyling found Llama-2 beats GPT-3.5 Turbo in certain benchmarks and that open-source LLMs used for LLM-based agents are able to surpass Chat-GPT-3.5 Turbo after extensive and task-specific pre-training and fine-tuning. In addition, open-source ToolLlama is better at tool usage, and Gorilla is better than GPT-4 at writing API calls. 

Another report from the Prompt Engineering Institute says Llama 2’s 40% more training data has led to performance gains, and Meta’s ability to leverage more public data makes Llama 2 more capable.  Llama 2-Chat beat ChatGPT-3 and other models for helpfulness, and the largest Llama 2 model, Llama-2-70b, matched GPT-4 at 85% for factual accuracy. The report summarizes that Llama 2 “stacks up impressively against GPT-4.”

Accessibility, Transparency, and Collaboration will Increase the Popularity of Open-Source LLMs

The cost of developing and training LLMs makes creating proprietary models prohibitive for most companies. Open-source LLMs, which have already been substantially trained, costing millions, are more accessible for companies, startups, and even individual developers. 

It’s estimated that ChatGPT-3 cost OpenAI $4.6 million to train and that a scalable enterprise in-house AI development could cost upwards of $95,000 per year. Pricing for SMB projects is also often less than transparent from LLM providers.

Llama-2’s 85% accuracy reportedly comes 30x cheaper than ChatGPT-4 and with a smaller model size and less complexity than GPT. Prompt Engineering Institute found Llama-2 was substantially less costly per paragraph summary and per 100,000 words than GPT.  Llama-2 also allows AI workflows to be contained internally with zero data exposure and can be self-hosted and modified for more control over data and privacy.

Llama-2, and indeed its open-source competitors, have and will continue to innovate and improve rapidly with the community development and knowledge sharing inherent to open-source technologies. 

Developers and Organizations will Increasingly Turn to Open-Source LLMs

Github’s Octoverse report on the state of open-source and AI says open-source powers “nearly every piece of modern software” and that generative AI projects are now in the top 10 open-source projects on the platform. Github COO Kyle Daigle writes:

“As more developers experiment with these new technologies, we expect them to drive AI innovation in software development and continue to bring the technology’s fast-evolving capabilities into the mainstream.”

Developers are using LLMs to develop APIs, bots, assistants, mobile applications, and plugins, propelling adoption and increasing the AI talent pool. 

The Octoverse report, citing statistics from the Linux Foundation, also reveals that 30% of Fortune 100 companies have Open Source Program Offices (OSPOs). This illustrates how organizations are increasingly embracing open-source contributions to power their operations, and it’s a trend likely to be profound in AI and LLMs, given the cost of development and the benefit of contributions from multiple organizations, experts, and communities to improve AI and overcome its flaws.  

A recent MIT Sloan article penned by Aron Culotta and Nicholas Mattei suggests open-source LLMs are a solution to building generative AI solutions locally rather than risking sensitive data with a third party. 

Open-source LLMs could also become increasingly popular given the apparent divides and less obvious reasons behind these conflicts within companies like OpenAI. Closed LLMs are also iterated rapidly, leaving little time for enterprises to understand or adapt to changes as they already face a significant learning curve deploying AI with customers and employees. 

Lastly, open-source LLMs offer significant freedom to developers and organizations to innovate them in the direction best suited for the aspired purpose and for the available budget and infrastructure. Open-source LLMs can be customized for lower-cost operation, for deploying multiple AI applications, and for user benefits like extended context windows. 

To Conclude 

The drawbacks of open-source LLMs include greater risks of misuse, the challenge of intellectual property rights, and a potential lack of quality control, which can create inconsistencies or even security risks. We’ve not delved into them in depth for this prediction, but these risks and others are notable, and when it comes to AI, addressing and mitigating risk is vital. 

What is clear is that, despite the risks, open-source LLMs have the potential to overtake the performance and popularity of closed LLMs. 

Open-source collaboration is driving rapid innovation, and when coupled with transparency right back to the bare code of these LLMs, as well as potentially lower development and deployment costs, it makes them an attractive proposition.

Build a Custom GPT for your business, in minutes.

Deliver exceptional customer experiences and maximize employee efficiency with custom AI agents.

Trusted by thousands of organizations worldwide

Related posts

Leave a reply

Your email address will not be published. Required fields are marked *

*

3x productivity.
Cut costs in half.

Launch a custom AI agent in minutes.

Instantly access all your data.
Automate customer service.
Streamline employee training.
Accelerate research.
Gain customer insights.

Try 100% free. Cancel anytime.