NTIA engages civil society on questions of open foundation models for AI, hears benefits of openness in the public interest

The recent US Executive Order on AI directs action for numerous federal agencies. This includes directing the National Telecommunications and Information Agency (NTIA*) to discuss benefits, risks and policy choices associated with dual-use foundation models, which are powerful models that can be fine-tuned and used for multiple purposes, with widely available model weights. 

The NTIA process is centered on a Request for Comment soliciting public feedback about how making model weights and other model components widely available creates benefits or risks to the broader economy, communities, individuals, and to national security.

NTIA also initiated a series of listening sessions last December. Owing to OSI’s critical effort in the Defining Open Source AI project, we are grateful to have been included in their most recent listening session organized by the Center for Democracy & Technology (CDT) for Civil Society organizations. We joined other non-profits working in the public interest to share comments, concerns and encouragement in a generous two hour session with NTIA staff. 

The core of the discussions was centered around open versus closed models. Several organizations brought historical perspectives going back to battles over Open Source in the 90s. A short list of key takeaways from organizations weighing in during the session:

• Open models represent marginal risk. More research is needed to understand where unacceptable risks lie beyond generating negative scenarios – for both open and closed models.
• Encouragement to not regulate the emerging technology itself, rather focus on addressing bad actors and bad behavior.
• Understand the benefits to research in open models, and in particular to provide transparency and accountability to privacy, security and bias concerns.
• Consider equitable access to economic benefits by keeping models open as well as an established factor in innovation.
• Completion of the OSI’s Defining Open Source AI and clarifying terms would greatly assist policy discussions.

NTIA staff expressed an interest in understanding what lessons we might draw from the Open Source software community’s experience with the federal government over the years. (OSI expects to speak to this in their formal response to NTIA’s Request for Comment).

OSI ED Stefano Maffulli provided OSI’s perspective in his comments at the meeting:

The Open Source Initiative is a 501(c)(3) nonprofit organization that is driving a global, multistakeholder discussion to find an unequivocal definition of Open Source AI. We’ve been maintaining the Definition of Open Source software for over 25 years, providing a stable north star for all participants in the Open Source ecosystem, including US federal agencies. 

The Department of Defense, Department of Commerce, Office of Management and Budget, Center for Medicaid/Medicare Services and others are examples of agencies which have relied on the standard Open Source Definition maintained by OSI in crafting their IT policies. 

The Open Source Definition has demonstrated that massive social benefits accrue when you remove the barriers to learning, using, sharing and improving software systems. There is ample evidence that giving users agency, control and self-sovereignty of their technical choices produces an ecosystem based on permissionless innovation. Recent research estimates that if Open Source software didn’t exist, firms would have to spend the equivalent of 8.8 trillion dollars to replace it. This is all based on the clear definition of Open Source software and the list of approved licenses that the Open Source Initiative maintains.

The same kind of unambiguous definition of terms is also needed and deserved in the domain of AI. We’re aware of various uses of the term ‘Open Source’ referring to AI systems and machine learning models whose terms of service have a wide range of obligations and restrictions. 

We found AI systems available publicly with full implementation details, code and data distributed without any obligations as well as other systems only available with limited implementation details, no data, very limited amount of description of the data used to train the model… all generally referred to as “Open Source.”

It’s worth noting that Open Source licenses are a way to flip the intellectual property system: the approved licenses grant rights to users instead of removing them. When thinking about the terms of distribution for model weights, which are basically facts, we should aim to remove the intellectual property regime to begin with.

We’re very concerned about the “economic upside capture” licensing terms we’ve seen in popular models like Llama2, for example. These terms of use are designed to create a network that favors only one economic actor (like the original distributor).

Uncertainties break the innovation cycles. This lack of clarity of terms doesn’t help consumers, scientists, developers or regulators. We’re on target to deliver a usable definition of Open Source AI by the end of October 2024. The definition work is focusing on identifying the preferred form to make modifications to an AI system: the equivalent of “source code” for software programs. This preferred form will be the basis to grant users the same level of self-sovereignty over the AI technologies.

* The NTIA, located within the US Department of Commerce, is the Executive Branch agency that is principally responsible by law for advising the President on telecommunications and information policy issues.

Coming up next: What might we draw from Open Source software’s experience with the federal government?