As proposed AI regulations gain momentum, regulators are recognizing that one size does not fit all. In the EU and the US, regulators now emphasize a proportional approach, where the compliance burden scales in line with an organization’s size and resources. But regulations still lack explicit guidelines for small and medium businesses, who often do not have dedicated AI personnel, to implement an effective AI Governance program. The question of ‘What is Good AI Governance for my Business’ is still left for interpretation.
There are 5 broad levels of AI Governance Maturity that an organization may adopt. Each comes with key tradeoffs between speed of development, and risk mitigation, as well as costs to implement. The appropriate maturity level for an organization takes into account the organization’s size, existing AI expertise, and what internal resources are already available. This breakdown of maturity levels and considerations can help organizations identify their ideal ‘target’ maturity level, and build towards it.
Level 1: No AI Governance
Many organizations start without any AI Governance at all: no structured guidance on governance practices, no clear oversight from anyone, and no technical testing standards or documentation requirements. There may be elements of best practices and good governance techniques, but they are typically driven from the bottom up and are likely tied to employees seeking to implement it themselves. The weakness of grassroots governance is sustainability – when internal champions leave, the established processes leave with them.
At this level, companies experimenting with AI are focused on innovating quickly to find the most valuable applications – not on reigning in experimentation with governance guidelines. But ignorance is not bliss. While this maturity level does allow for the most innovation without internal bureaucracy, the lack of any guidance, best practices, or stakeholder engagement actually decreases the likelihood that any successful AI experiment would get implemented. Without a governance structure, individual AI advocates struggle through unclear implementation roadmaps, disorganized and inefficient resource allocation, and risk backlash in the testing and release stage that hinders any future initiatives. In addition, they stand to lose trust with customers increasingly concerned about how companies are using AI.
Level 2: Self-Managed with Best Practices
The next step up is self-managed AI governance, with individual teams instituting pockets of best practice processes. For example, there may be clear policies to require code reviews for AI/ML code and a technical leader may need to sign off on every new model and AI use case. Other characteristics at this level include some developer-focused tools to support AI quality control, standards for documenting details about the AI System, or a structured AI product discovery process stipulating target user interviews. Regardless of the specific policy, oversight and enforcement at this level are done by people within the same department. This is analogous to the ‘first line’ of defense in the financial model risk management world.
Many software engineering teams enforce standards for cybersecurity and technical robustness in the same way, and industry standards such as SOC-2 help organizations adopt this kind of model. This maturity level should be the absolute ‘minimum’ to implement AI for any organization of any size, including startups. Internal standards and policies should be defined and enforced by the R&D leadership. Systems should be well documented at a bare minimum at the technical level, testing frameworks should be in place for known technical risks, and the go/no-go gates should be commonly understood. This level still promotes innovation as most decision making is still done within the same business function and new experiments can be quickly spun up and down. However, it still provides the first level of defense by ensuring AI cannot be deployed without some oversight and standards being met.
Level 3: Dedicated Internal Overseer
Many organizations need someone who isn’t in the ‘AI Hype’ bubble to also be directly involved in the AI development process. Individuals on ‘privacy’ teams, ‘trust and safety’ teams, or even within a traditional compliance or legal team will have a go/no-go decision-making role in order to launch any AI system. Bringing in the risk, legal, or public policy perspective, provides a second level of defense and additional accountability to ensure AI systems are being developed and deployed responsibly.
Because this maturity level is now ‘cross-functional’, the standards for documentation are much higher as details must be defined both in technical, and non-technical terms. In addition, the processes are heavier as they involve setting up cross-functional meetings and discussions. Capturing those discussions will be a key part of AI compliance in upcoming AI regulations. At this level, companies have chosen to sacrifice some agility and speed in exchange for better risk management, and better guarantees around legal compliance and accountability. This maturity level is ideal for any public company or large enterprise that is heavily using AI across multiple business units.
Level 4: AI Ethics Committee
Even a dedicated internal team may not have all the relevant skills to assess specific AI use cases, especially novel or high-risk ones. A more comprehensive AI ethics committee – with experts across AI technical development, ethics, and legal – provides a stronger oversight function than simply having one team review things. This structure is analogous to the Institutional Review Boards (IRBs), who provide ethical oversight for scientific research projects.
This maturity level builds off Level 3’s Dedicated Internal Overseers in second-party review and sign-off procedures by adding on extra protections to ensure that multiple people with a broader set of perspectives participate in the discussions. The same AI Ethics committee should be tasked with setting organization-wide standards and processes for responsible AI and actually enforcing these standards. This maturity level further sacrifices speed for more assurance that AI risks have been identified and mitigated to ensure trust in the system. Large public companies in heavily regulated industries or complex cross-functional operations should aspire for this maturity level.
Level 5: External Oversight
The highest maturity level an organization can adopt is leveraging an external body for AI oversight. This could be a government regulatory body, a third-party auditor, or an external AI ethics committee similar to the pharmaceutical industry’s oversight body – where the Food & Drug Administration (FDA) and similar international regulators review and approve new pharmaceutical products. While there are concerns that a hired auditor may just rubber stamp systems to give the appearance of compliance – called ‘Ethics Washing’ in other industries – an independent 3rd party could still provide a strong level of assurance about AI systems.
This maturity level is the ‘heaviest’ in terms of mandatory documentation, internal processes and standards. The external oversight criteria involves not just information about an AI product itself, but also proof that specific policies are being tightly enforced within an organization to ensure AI safety requirements are met. Global ‘big tech’ companies, highly regulated products with global scale, organizations creating physical AI agents (e.g. robots or autonomous vehicles), and many public sector organizations will be subject to this maturity level for at least some of their products to help build public trust in their systems.
Other factors that impact your target AI Maturity Level
Use Case Risk Level
Regulations also take into account the sector an organization operates in and the specific use cases. Even a small startup creating a ‘high risk’ AI system (as defined by the EU AI Act) will need to clear higher regulatory scrutiny. Examples of this include AI tools for recruiting or software applications for children. Even though the EU is working on regulatory sandboxes to help facilitate growth of these startups, startups with high risk AI uses will still be required to submit documentation to authorities. The regulatory reporting burdens, and all the underlying infrastructure, will still need to exist even if the financial penalties will not apply.
Industry
Similarly, governance standards will be higher for organizations operating in an already heavily regulated industry such as healthcare, financial services, or law enforcement. Organizations operating in highly regulated sectors should aspire to one level higher than they otherwise would target on our maturity scale. For example, a small startup that might normally just implement best practices should start out with a dedicated oversight role if they are operating in a highly regulated industry. Along similar lines, a large enterprise operating in this space should consider regularly leveraging an external oversight committee or regular third-party audits of their systems.
Scale
‘Low’ risk AI use cases, such as a music recommendation algorithm, can become a higher risk system if they achieve a certain level of scale and market share. For example, while the ‘harms’ of a bad song recommendation may be small on the listener, bad recommendations at scale can be harmful to the artists whose livelihood is dependent on their songs being streamed. This scale may also apply to open source systems that many other systems rely on. Open source providers may not have any ethical responsibility when at a small scale, but once they become a dominant player, their decisions about system updates do have significant impacts on downstream users. Much like the sector and high-risk consideration, organizations that achieve a large scale (10+ million users) should also consider stepping up one level on the maturity scale.
Final Thoughts
Striking the right AI Governance Maturity balance for your organization is not a one-step process. It requires constantly reassessing your organization’s AI priorities, the relevant regulatory landscape, and technological limitations.
Different teams and business units within a single organization may also adopt different levels of maturity, or have some hybrid between the levels. For example, many organizations have already implemented their own ‘risk-based escalation’ framework where low-risk use cases follow best practices, slightly higher-risk ones involve the compliance team, and the highest-risk use cases are overseen by an AI ethics committee, or receive a regular outside audit.
Organizations need to not only conduct an initial assessment of their desired AI maturity level but also constantly readjust based on the tradeoff between speed and innovation and their own risk tolerance. Only then can they be truly AI ready.
Read the full article here
