Gartner on AI Governance: Importance, Issues, Way Forward

AI governance assures the responsible and transparent use of AI within organizations. Gartner’s survey of 200 IT and Data and Analytics leaders (from early 2023) reveals that a lack of AI governance can lead to costly failures.

In this article, we’ll summarize Gartner’s take on AI governance and explore its importance in enterprise AI use cases. We’ll also look at challenges in implementing AI governance and recommend ways to overcome them.

---

---

Table of contents #

1. What is AI governance, according to Gartner?
2. Why does AI governance matter?
3. Challenges in implementing AI governance: Gartner’s perspective
4. Gartner’s recommendations for future-proofing AI projects
5. Governance is never done, it only changes and evolves
6. AI Governance & Gartner: Related Reads

---

What is AI governance, according to Gartner? #

Gartner defines AI governance as the process of assigning and assuring organizational accountability, decision rights, risks, policies and investment decisions for applying AI.

AI governance is asking the right questions and giving the answers - Source: Svetlana Sicular, VP Analyst, Gartner.

AI governance helps guarantee that AI systems are transparent, fair, and accountable, i.e., responsible AI. Here’s how.

If not properly managed, AI algorithms can inadvertently perpetuate biases, violate privacy laws, generate nonsensical responses, or lead to unintended consequences. This adversely affects the outcomes generated by AI systems.

With AI governance, you can establish guidelines for data quality, algorithm design, and human oversight, as well as promote responsible AI. Gartner calls responsible AI an umbrella term for making appropriate business and ethical choices when adopting AI. This includes:

• Business and societal value
• Risk
• Trust
• Transparency
• Fairness
• Bias mitigation
• Explainability
• Sustainability
• Accountability
• Safety, privacy, and regulatory compliance

Responsible AI is just three years from reaching early majority adoption - Source: Anushree Verma, Director Analyst at Gartner.

---

Why does AI governance matter? #

Organizations (without AI governance) are exponentially more inclined to experience adverse outcomes - Source: Avivah Litan, Gartner Distinguished VP Analyst.

As Avivah Litan points out, AI systems can produce adverse outcomes and need good governance to churn out value.

However, organizations tend to deploy AI models or applications without considering the risks involved, which crop up when there is no AI governance in place. According to Gartner, these risks include:

• A lack of explainability — what AI systems are, how they function, what are the expected outcomes and potential biases
• A risk to data confidentiality when integrating AI models and tools from third-party providers
• A lack of monitoring — controls must be applied continuously, throughout model and application development, testing and deployment, and ongoing operations
• A need for new methods to test, validate, and improve AI-powered workflows
• An evolving regulatory landscape — the EU AI Act, regulatory frameworks in North America, China and India establish regulations to manage the AI risks

AI governance holds the key to identifying and eliminating these risks, paving the way for a future where AI is harnessed responsibly and transparently.

While several organizations understand the importance of AI governance, only some have succeeded in implementing it.

Gartner’s survey of 200 IT and Data and Analytics leaders in early 2023 found that only 12% have a dedicated AI governance framework. The majority (55%) of respondents mentioned that their organizations hadn’t implemented an AI governance framework yet.

Gartner survey of IT and Data and Analytics leaders on AI strategy - Source: Gartner.

Why isn’t AI governance in place yet? In the next section, we will understand the reasons that impede the implementation of AI governance.

---

Challenges in implementing AI governance: Gartner’s perspective #

The biggest challenges in implementing AI governance are:

• Skill gaps: Organizations often need specialized expertise to design, implement, and oversee AI governance frameworks.
• Lack of clarity about AI’s business impacts: Many organizations struggle to quantify the tangible benefits and risks associated with AI initiatives. This makes it harder to prioritize and allocate the necessary resources.
• Production-first mentality: The rush to deploy AI systems often overshadows concerns about trust, risk management, and ethical considerations, which become an afterthought.
• Lack of data governance: Without proper data governance practices, issues like data quality, bias, and privacy violations can affect the effectiveness and trustworthiness of AI systems.
• Organizational fragmentation: Data, tools, and technology ownership is often spread across domains. Adding AI risk management to the mix makes the situation more challenging, as the onus lies on multiple functions — legal, compliance, AI development, and security.
• Poor collaboration between business functions: AI projects often involve multiple stakeholders from different departments. A lack of collaboration and communication between these functions can affect the development and implementation of cohesive AI governance strategies.

Challenges in implementing AI governance according to a Gartner survey of IT and Data and Analytics leaders - Source: Gartner.

A common thread across all the above challenges is the failure to embed AI governance into every workflow.

When governance is seen as a separate domain or an afterthought, it’s ineffective. The consequences of not having AI governance in place are evident — increased costs, failed AI initiatives, decreased revenue, and more.

The consequences of not having AI governance in place, according to a Gartner survey of IT and Data and Analytics leaders - Source: Gartner.

So, how can enterprises successfully implement AI governance? Gartner has a framework in mind.

AI TRiSM framework for AI governance: Gartner’s recommendation to tackling challenges in enterprise AI #

Gartner proposes a comprehensive AI trust, risk, security management (TRiSM) approach for AI governance, ensuring AI systems are “compliant, fair, reliable and protect data privacy.”

The AI TRiSM framework ensures the reliability, trustworthiness, security and privacy of AI models. This drives better outcomes related to AI adoption, achieving business goals, and ensuring user acceptance.

The four pillars of this framework are:

• Explainability/model monitoring: Make AI systems transparent and understandable so that you can explain how AI models arrive at their decisions. Explainable AI “offers product leaders a differentiation to the black-box nature of AI-based solutions that can make them seem non-transparent.”

  Meanwhile, tracking the performance and behavior of AI models can help detect anomalies, drift, or unintended consequences, so that you can flag and fix them immediately.

• ModelOps: ModelOps oversees the governance and life cycle management of AI and decision models. This is crucial for the efficient and reliable operations of AI systems.

• AI application security: Address security challenges targeting AI systems and develop specialized practices, policies, threat detection and prevention tools to safeguard them. A core element is adversarial attack resistance, which involves teaching AI models to either ignore or respond differently to malicious inputs.

• Privacy: Examine the data privacy challenges that AI systems pose by reviewing the model architecture, training data, and your retention policy for training data. Gartner advises asking questions, such as:

  • Does this AI model process (re)identifiable data? Or is the data anonymized, but still useful for running the AI model?
  • Is the training data credible and legitimate? Can it be used to train AI models, or does it risk exposing sensitive information?
  • Do we need to retain the training data, and for how long?
  • Did we unintentionally use sensitive or PII data to train our AI models? If so, then is there a risk of the model leveraging this data for future uses?

AI TRiSM framework for AI governance - Source: Gartner.

It’s vital to note that the AI TRiSM framework will only work if it’s inherent to every workflow, incorporating governance by design rather than as a separate function.

Governance needs to be embedded into the way data teams work - Source: Atlan’s active data governance manifesto.

Another aspect of governance to note is that it’s not a “one-and-done” effort. It continuously changes and evolves alongside innovations.

So, how do you lay a future-proof foundation for your AI projects? Let’s look at what Gartner has to say.

---

Gartner’s recommendations for future-proofing AI projects #

Gartner recommends that organizations implement an AI governance program to catalog and categorize AI use cases.

Gartner’s recommendations for future-proofing AI projects - Source: Anurag Raj, Sr Principal Analyst at Gartner.

Raj’s opinion above refers to GenAI capabilities, such as cataloging, classification, and others that solve specific business challenges — enriching customer data, better targeting, etc.

Besides leveraging GenAI capabilities to implement AI governance, you could take a few more actions. Anushree Verma, Director Analyst at Gartner, advises:

• Monitor and incorporate the evolving compliance requirements of responsible AI by developing a framework that maps the organization’s GenAI portfolio of products and services to AI-specific regulatory requirements
• Operationalize AI trust, risk and security management by integrating responsible AI
• Ensure service provider accountability for responsible AI governance by enforcing contractual obligations and mitigating the impact of risks arising out of unethical and noncompliant outcomes

In addition to the above, Gartner shares some insight on ensuring compliance with AI-specific regulations like the EU AI Act. As AI models evolve and become more mainstream, the regulations dictating their secure and ethical use will continue growing.

So, let’s see what Gartner has to say on the subject to help enterprises avoid hefty fines — up to €35 million or 7% of global turnover.

Regulatory compliance and AI: Gartner’s take #

Gartner suggests discovering and cataloging “AI-enabled capabilities with enough detail for the subsequent risk assessment.” Nader Henein, VP Analyst at Gartner, suggests dividing your AI systems into the following categories:

• AI In-the-wild: Public domain AI tools (ChatGPT, Gemini, etc.) that your team uses for work-related purposes
• Embedded AI: Tools, systems, platforms, and solutions (spam or malware detection) used within your organization that have built-in AI capabilities
• AI In-house: AI capabilities trained, tested, and developed internally within your organization
• Hybrid AI: Enterprise AI capabilities that are built in-house using one or more off-the-shelf foundational models

You can compile a list of AI tools/capabilities by surveying your employees, internal product and engineering teams, and third-party solution providers.

---

Summing up: Governance is never done, it only changes and evolves #

GenAI has led to organizations leveraging unstructured data, vector databases, LLMs, and AI applications to extract business value and drive innovation. These developments require you to revisit your approach to data governance and optimize it for future-forward use cases.

That’s why data governance fundamentally needs to be open and extensible to change, adaptive, embedded within each workflow, and understandable by all data practitioners.

Data governance is the connective tissue for data practitioners, and AI governance is no different. Gartner’s perspective on AI governance emphasizes that it’s central to the success and scaling of AI projects. AI governance impacts your business outcomes, productivity, efficiency, and revenue — the only way forward is to adopt governance by design, rather than an afterthought.

---

AI Governance & Gartner: Related Reads #

• A Guide to Gartner Data Governance Research — Market Guides, Hype Cycles, and Peer Reviews
• Gartner Data Catalog Research Guide
• Gartner Active Metadata Management
• Gartner on Data Mesh
• Gartner on Data Fabric
• Gartner on Data Lineage
• Gartner on DataOps
• Gartner Magic Quadrant for Metadata Management
• Gartner Magic Quadrant for Data Quality

---