Sign up for our daily and weekly newsletters to stay up to date with the latest updates and exclusive content on industry-leading AI coverage. More information
As research and adoption of artificial intelligence continues at an accelerating pace, so too does the risks associated with the use of AITo help organizations navigate this complex landscape, researchers from MIT and other institutions have developed the AI Risk Repositorya comprehensive database of hundreds of documented risks of AI systems. The repository is intended to help decision makers in government, research, and industry assess the evolving risks of AI.
Bringing order to AI risk classification
While many organizations and researchers recognize the importance of addressing AI risks, efforts to document and classify these risks have been largely uncoordinated. This has led to a fragmented landscape of conflicting classification systems.
“We started our project with the goal of understanding how organizations are responding to the risks of AI,” Peter Slattery, a prospective postdoc at MIT FutureTech and project lead, told VentureBeat. “We wanted a fully comprehensive view of AI risks to use as a checklist, but when we looked at the literature, we found that existing risk classifications were like pieces in a jigsaw puzzle: interesting and useful in isolation, but incomplete.”
The AI Risk Repository addresses this challenge by consolidating information from 43 existing taxonomies, including peer-reviewed articles, preprints, conference papers, and reports. This rigorous curation process has resulted in a database of over 700 unique risks.
The repository uses a two-dimensional classification system. First, risks are categorized based on their causes, taking into account the responsible entity (human or AI), the intent (intentional or unintentional), and the timing of the risk (pre-deployment or post-deployment). This causal taxonomy helps to understand the circumstances and mechanisms through which AI risks can arise.
Second, the risks are classified into seven different domains, including discrimination and toxicity, privacy and security, disinformation and malicious actors and abuse.
The AI Risk Repository is designed as a living database. It is publicly accessible and organizations can download it for their own use. The research team plans to update the database regularly with new risks, research findings, and emerging trends.
Evaluating AI Risks to the Enterprise
The AI Risk Repository is designed as a practical resource for organizations across industries. For organizations developing or implementing AI systems, the repository serves as a valuable checklist for risk assessment and mitigation.
“Organizations using AI may benefit from using the AI Risk Database and taxonomies as a useful foundation for comprehensively assessing their risk exposure and management,” the researchers write. “The taxonomies may also be useful for identifying specific behaviors that should be performed to mitigate specific risks.”
An organization that, for example, AI-driven recruitment system can use the repository to identify potential risks related to discrimination and bias. A company that AI for content moderation can leverage the domain of 'misinformation' to understand the potential risks associated with AI-generated content and develop appropriate safeguards.
The research team acknowledges that while the repository provides a comprehensive foundation, organizations need to tailor their risk assessment and mitigation strategies to their specific contexts. However, a centralized and well-structured repository such as this reduces the likelihood of critical risks being overlooked.
“We expect the repository to become increasingly useful to enterprises over time,” Neil Thompson, head of the MIT FutureTech Lab, told VentureBeat. “In future phases of this project, we plan to add new risks and documents and ask experts to review our risks and identify omissions. After the next phase of research, we should be able to provide more useful information about which risks experts are most concerned about (and why), and which risks are most relevant to specific actors (e.g., AI developers vs. high-volume AI users).”
Shaping future AI risk research
In addition to its practical implications for organizations, the AI Risk Repository is also a valuable resource for AI risk researchers. The database and taxonomies provide a structured framework for synthesizing information, identifying research gaps, and guiding future research.
“This database can provide a foundation to build on when doing more specific work,” Slattery said. “Before, people like us had two choices. They could invest a lot of time going through the scattered literature to develop a comprehensive review, or they could use a limited number of existing frameworks, which might miss relevant risks. Now they have a more comprehensive database, so our repository will hopefully save time and increase oversight. We expect it to become increasingly useful as we add new risks and documents.”
The research team plans to use the AI Risk Repository as a basis for the next phase of their own research.
“We will use this repository to identify potential gaps or imbalances in how organizations are addressing risk,” Thompson said. “For example, to examine whether there is a disproportionate focus on certain risk categories while others of equal significance remain underexposed.”
In the meantime, the research team will continue to update the AI Risk Repository as the AI risk landscape evolves, and ensure it remains a useful resource for researchers, policymakers, and industry professionals working on AI risk and mitigation.