According to the 2024 Gartner Hype Cycle Report for AI, generative AI (GenAI) has climbed past the peak of expectations and is skiing down the trough of disillusionment. Instances of AI going rogue have played a significant role in driving this shift in perception.

Businesses are wary of the risks that accompany AI

Discussions with dozens of technology and business executives reaffirm the opportunities and risks enterprises see in operationalizing AI. Data scientists and AI engineers greet each foundation model release with cheers. Likewise, every major AI failure raises business executives’ alarm levels. If ChatGPT cannot be trusted to correctly count the number of ‘R’s in the word strawberry, how can more critical tasks be entrusted to AI?

Semantic clarity in AI is the first step to proper perspective

Businesses are sometimes so spooked by AI’s inherent risks that they treat any solution carrying the label with suspicion. Even mature solutions face increasing scrutiny and long delays. For instance, predictive models for managing credit risk, clustering models used to build customer segments and personas, or computer vision models for scanning health records are now taking longer to be accepted. The fears often stem from semantic confusion when comprehending the AI and GenAI landscape.

AI covers a vast landscape of techniques and tools. AI solutions are built using probabilistic and statistical models with a margin of error. These models are trained on data that is rarely perfect. Even if the data is clean, not every future scenario must be represented in the historical data used to train the models. For these reasons, AI can misbehave, malfunction, or go rogue.

The nature of risk is dependent on the type of problem solved and solution techniques used. AI solutions span the following areas:

1. Reporting, business intelligence, and analytics

2. Data science and predictive modeling

3. Machine learning (including recommender systems, anomaly detection, and deep learning)

4. Natural language processing

5. Computer vision

6. Generative AI / large language model / large multimodal model

Framing AI solutions in the context above can help characterize the nature of the risks involved.

AI risks vary widely in intensity and impact

Some AI applications are more vulnerable to certain types of risk than others. These risks fall into the following categories:

1. Inaccuracies: They often lead to faulty KPI measurements and incorrect reporting.

2. Inference errors: Spurious correlations found in data and incorrect cause-effect relationships lead to wrong inferences, data misinterpretation, and misunderstanding of business drivers.

3. Prediction errors: These include issues like wide forecast error margins and excessive or costly misclassifications.

4. Bias: AI Algorithms can systematically perpetuate and amplify bias present in data, potentially causing discriminatory impact on customers.

5. Hallucinations: Large Language Models can dream up non-existent details and convincingly present them as facts. They can mislead customers and create adverse consequences.

6. Privacy & security violations: These concerns include non-permissible data usage, data leakage to third parties or external AI models, and regulatory or compliance breaches.

Concentration of AI risks

Targeted strategies can minimize AI risks

Enterprises can deploy AI with confidence by adopting targeted measures to minimize risk. Here are several to consider:

1. Data quality assurance: Data integrity is critical for reducing Inaccuracies, Inference Errors, Prediction Errors, and Hallucinations. High-fidelity data instrumentation and capture at source, high-quality data processing, and reliable data pipeline operations with end-to-end visibility can achieve this.

   For example, when dealing with health data, it’s essential to implement automated data checks to ensure the data is complete, accurate, and within acceptable range before feeding them into models for training.

2. Robust analysis and modeling framework: A thoughtful choice of analytic methodology, modeling framework, and feature design can improve the accuracy and transparency of AI models. This is critical in highly regulated industries like consumer credit or insurance.

   Risk assessments made during marketing pre-qualification or underwriting processes are better served by parsimonious AI models that use linear functions or decision trees with clear structure instead of complex deep learning models or ensemble models where decisions made are more difficult to trace and explain.

3. Optimized GenAI workflow: AI hallucinations can be reduced by consciously optimizing each step in the model workflow, including data processing, task-specific Prompt Engineering and Retrieval Augmented Generation (RAG), LLM selection, domain-specific finetuning, and evaluation. LLMs are trained on a vast corpus of data covering a wide range of subjects.

   However, their solutions can be finely tailored and optimized for the local context. For instance, an AI agent for employee onboarding can be optimized using RAG workflow aided by employee handbooks and standard operating procedures in combination with prompts specific to new employees and questions they typically ask during their onboarding phase.

4. Input and output guardrails: Controls on AI model data inputs, evaluation of model outputs, and guardrails on customer-facing actions can help minimize the impact of bias and hallucinations. Examples of input guardrails include using content filters to screen out inappropriate or biased language in user messages or prompts and automatically redacting or anonymizing names, addresses, and other unique customer identifiers.

   Output guardrails could take many forms, such as setting confidence thresholds before model responses are shared with users and running results through bias detection algorithms to flag gender, race, ethnicity, or other biases.

5. Infrastructure and security protocols: Selection of the correct infrastructure configuration, e.g., On-Premises, Cloud, or Private Cloud, is central to addressing privacy and security risks. Data encryption, self-hosted LLMs, data anonymization, and filters to screen data before it reaches cloud LLM APIs significantly protect sensitive information and ensure data privacy.

6. Data and model governance: Data and Model Governance provides the structure and discipline to manage all types of AI risks. This requires combining people, processes, and technology across multiple organizational functions. Financial services and healthcare organizations are maturing their governance practices by treating data and AI as products.

   This involves establishing data and model stewardship, monitoring data flows and AI model parameters, completing formal reviews and signoffs for releasing data and AI products, and designing human-in-the-loop AI processes for regular business operations.

A systematic approach maximizes the benefits of AI

AI presents both an exciting array of opportunities and some inherent pitfalls. A prudent approach is to weigh the benefits of AI when it functions correctly against the consequences in case it goes wrong. 

Applications that generate answers to routine customer inquiries help customers research desired products online and generate personalized messaging has a risk-benefit profile that is exceptionally conducive to the use of AI. On the other hand, applications such as offering advice on health, wellness, or nutrition and evaluating investment options carry severe consequences that require the lowest margin of error and the highest levels of diligence at every step. 

The possibility of risk need not deter enterprises from adopting AI. Proportionate investments in risk mitigation can maximize the benefits while keeping the downsides in check.

About the Author

Niranjan Krishnan is a seasoned data and AI leader with two decades of experience in delivering on the promise of data. He has led large cross-functional teams and deployed dozens of AI/ML solutions across industries. Krishnan is the Head of AI Solutions at FPT Software. A trusted ally to executives and business leaders, he is passionate about Responsible AI solutions that create measurable value for businesses and customers.