The AI renaissance and why Open Source matters

Sharing knowledge and sharing code has always been a key driver for innovation in Artificial Intelligence. Researchers have gathered together since AI was established as a field to develop and advance novel techniques, from Natural Language Processing to Artificial Neural Networks, from Machine Learning to Deep Learning.

The Open Source community has played a key role in advancing AI and bringing it to solve real-world problems. Libraries and frameworks like TensorFlow, PyTorch, Keras, and Scikit-learn, for example, have allowed researchers and data scientists to study and make use of AI.

In recent years, we have seen the emergence of LLMs (Large Language Models). While most of the early models such as BERT and GPT-2 were commercial friendly, proprietary models like OpenAI’s GPT-4 have gained wide popularity. Models like LLaMA, Alpaca, Vicuna, and Koala have not yet adopted a license that allows for commercial use.

Fortunately in these past few years, we have seen the research community build several important Open Source models, such as EleutherAI’s GPT-Neo, GPT-J, GPT-NeoX, and Pythia, as well as models from BLOOM (BigScience Large Open-science Open-access Multilingual Language Model) and LAION (Large-scale Artificial Intelligence Open Network).

Most recently, we have seen Open Source models like DataBricks Dolly 2.0, Stability Diffusion’s StableLM, Cerebras GPT, h2oGPT, RedPajama, UC Berkeley’s OpenLLaMA, MosaicML’s MPT, NVIDIA’s NeMo, Hugging Face’s StarCoder, and TII’s Falcon emerge and gain traction.

Additionally, it’s important to note that smaller and more specific models are outperforming larger models, allowing small organizations and even individuals to innovate. Fine-tuning these models using techniques like LoRA (Low-Rank Adaptation of Large Language Models) and  QLoRA have allowed for faster iterations.

Recently, users have been able to run and even fine tune these smaller models from their desktop and mobile (e.g. GPT4All and PrivateGPT) and their browser (e.g. WebGPT). This will unleash a new wave of applications, where users’ privacy will be respected while allowing them to take advantage of AI. Businesses will also gain full control over these models and be able to run them on prem or on the public cloud using privacy-enhancing technologies. Additionally, these models will also be able to run from IoT devices and on the edge.

The world of AI is at an important crossroads. There are two paths forward: one where highly regulated proprietary code, models, and datasets are going to prevail, or one where Open Source dominates. One path will lead to a stronghold of AI by a few large corporations where end-users will have limited privacy and control, while the other will democratize AI, allowing anyone to study, adapt, contribute back, innovate, as well as build businesses on top of these foundations with full control and respect for privacy.

If you are interested in learning more about Open Source and AI, please join our “Deep Dive: Defining Open Source AI” series.