Introduction

As AI tools become standard fixtures in enterprise workflows, the processes organizations use to evaluate and approve them have not kept pace with the specific risks those tools introduce. Standard IT procurement frameworks were designed for conventional software, where failure modes are explicit and bounded. AI systems fail differently: they produce outputs that appear authoritative while being factually incorrect, they inherit the biases and legal risks embedded in their training data, and their performance changes over time in ways that routine monitoring does not always catch.

The cost of applying an inadequate evaluation framework to an AI tool is measurable. According to the IBM 2025 Cost of a Data Breach Report, breaches involving unsanctioned AI tools averaged USD 4.63 million per incident, USD 670,000 above the broader enterprise average. The Verizon 2025 Data Breach Investigations Report documented that third-party involvement in breaches doubled from 15% to 30% in a single year, the largest single-year shift in the report's history. This insight examines what a rigorous AI vendor due diligence process encompasses, where the gaps in standard evaluation frameworks tend to appear, and how organizations can build procurement practices that are both appropriately thorough and operationally sustainable.

Definitions

Model Drift

The gradual degradation of an AI model's accuracy over time as real-world inputs diverge from the distribution of data on which the model was trained. Unlike a software defect, model drift is not resolved by a patch; it requires retraining or recalibration. Its onset is typically gradual and often invisible without active performance monitoring, making contractual SLA provisions covering baselines, measurement intervals, and remedies an important component of AI vendor agreements.

Training Data Provenance

The documented origin, composition, and licensing status of the data used to train an AI model. Provenance determines whether a model carries discriminatory patterns inherited from historical datasets, copyright exposure from unlicensed training material, or privacy liabilities from data collected without adequate consent. Opacity about training data composition is common among AI vendors and represents a material risk dimension that standard procurement questionnaires do not typically address.

Shadow AI

The use of AI tools by employees without organizational authorization, typically through personal accounts or consumer-facing platforms that expose company data to third-party systems outside the organization's control. According to IBM's 2025 Cost of a Data Breach Report, shadow AI accounted for 20% of enterprise breaches and added an average of USD 670,000 per incident above the standard breach cost.

Why Standard Procurement Frameworks Fall Short

Conventional enterprise software evaluation centers on a familiar set of criteria: functional capability, security certification, integration compatibility, and vendor financial stability. These criteria remain relevant for AI tools, but they do not address the failure modes that are specific to AI systems, and those failure modes are the ones most likely to produce organizational harm.

SOC 2 certification, a standard requirement in most enterprise SaaS procurement, assures that a vendor's security controls meet a defined operational standard. It provides no assurance about how the underlying model behaves when it produces an error, what the training data contains, or whether the system's outputs carry copyright or discrimination liability. As the GLACIS 2026 vendor diligence guide notes, over 66% of B2B buyers require SOC 2 certification before signing with SaaS vendors, yet even certified AI vendors frequently lack transparency into model behavior, training data provenance, and bias mitigation, which are the dimensions where AI-specific risk concentrates.

The gap between what standard procurement confirms and what AI deployment requires means that organizations can complete a conventional due diligence process in good faith and still accept risks they have not assessed. Addressing this gap requires extending the evaluation framework with criteria that are specific to how AI systems are built, how they fail, and how the contracts governing them allocate responsibility for those failures.

The Risk Landscape of AI Vendor Selection

The failure modes of AI tools fall into categories that have no direct equivalent in conventional software risk management, and each requires a distinct evaluation approach.

Hallucination is the most consequential AI-specific failure mode for knowledge-intensive deployments. Large language models generate outputs by predicting statistically plausible continuations of their inputs; they do not have a mechanism for distinguishing between accurate and inaccurate claims. Analysts estimate that global enterprise losses attributable to AI hallucinations reached USD 67.4 billion in 2024. In legal services specifically, 68% of professionals cite hallucinations as their primary AI concern, with over 40% of law firms reporting that AI-generated documents require full manual revision before use. The practical implication for vendor evaluation is that hallucination rate measurement, vendor reporting cadences, and contractual liability allocation for erroneous outputs are all dimensions that diligence should address explicitly.

Supply chain risk has emerged as an equally significant concern. Third-party involvement in breaches doubled from 15% to 30% in a single year per the Verizon 2025 DBIR, and AI pipelines amplify this risk because each external dependency, whether an API, plugin, or fine-tuning provider, represents an additional surface for model poisoning, data interception, or behavioral manipulation. The ENISA Threat Landscape 2025 documents model poisoning, Rules File Backdoor attacks on AI coding assistants, and slopsquatting as categories of AI-specific supply chain attacks that traditional security tooling is not configured to detect.

Bias and discrimination liability represent a third category of AI-specific risk with direct legal consequence. A model trained on historical data that reflects discriminatory practices will replicate those patterns in its outputs, and the organization deploying the model can face liability alongside the vendor for the resulting decisions. The Mobley v. Workday case illustrates how AI-driven employment decisions can generate discrimination claims against both the vendor and the deploying organization under federal and state law. Because most vendors cannot or will not disclose full training data composition, due diligence must seek contractual representations about bias testing methodology and remediation obligations as a substitute for direct transparency.

What a Rigorous Evaluation Covers

An AI vendor evaluation that addresses the risk categories above extends the standard procurement checklist across four substantive dimensions: data handling, model behavior, regulatory compliance, and contractual terms. Each dimension raises questions that have no equivalent in conventional SaaS procurement.

Data Handling and Privacy

The foundational data handling questions for any AI vendor are: whether the vendor uses customer inputs to train or improve their models; where data is stored and whether its location creates cross-border compliance exposure; what the data retention period is; and whether deletion upon contract termination extends to backup and archival systems or only to production environments. Nearly 47% of generative AI users access tools through personal accounts, bypassing enterprise controls entirely, which makes the approved vendor's data handling practices the primary line of defense for organizational data rather than a secondary consideration. The EU Data Act, effective September 2025, now mandates specific data portability and switching rights, and vendor contracts should reflect these obligations explicitly.

Model Behavior and Performance

Model behavior evaluation goes beyond capability demonstration to assess how the system performs under conditions that reflect the organization's actual deployment context. This includes requesting sandbox access for adversarial testing before contract signature, establishing documented hallucination rates for the specific use case category under evaluation, and understanding the vendor's process for disclosing model updates that materially change output behavior. The Forbes Technology Council's 2026 AI vendor checklist identifies pre-contract red-teaming as a minimum standard for responsible AI procurement, noting that vendor demonstrations are designed to show favorable outcomes and should not substitute for independent evaluation.

Model drift requires contractual provisions rather than technical controls alone. Performance baselines should be established against the organization's actual use cases rather than generic benchmarks, with SLA provisions specifying measurement intervals, degradation thresholds, and graduated remedies covering notification, service credits, and termination rights for persistent performance failures.

Regulatory Compliance

The regulatory environment for AI procurement has become materially more complex over the past eighteen months. The EU AI Act, fully applicable to high-risk systems from August 2026, requires conformity assessments, risk management documentation, and EU database registration. General-purpose AI model providers were required to meet training data transparency obligations from August 2025. In the United States, Colorado's AI Act, effective June 2026, classifies employment-related AI systems as high-risk; New York City Local Law 144 requires bias audits and public disclosure for automated employment decision tools; and Illinois mandates disclosure when AI influences employment decisions.

A critical aspect of compliance evaluation is establishing how responsibility is allocated between the vendor and the deploying organization. These regulatory obligations apply to the deploying organization regardless of whether the AI is built internally or sourced from a vendor, and due diligence should establish which requirements apply to the specific deployment context, whether the vendor's system meets them, and what documentation the vendor provides to support the deploying organization's own compliance obligations.

Contractual Terms and Liability Allocation

Standard SaaS agreements typically disclaim all responsibility for output accuracy and exclude liability for model behavior, placing the full consequence of AI-specific failure modes on the deploying organization by default. For AI deployments in consequential workflows, these terms represent a significant and often unexamined risk transfer. The minimum contractual provisions for a responsible AI vendor agreement include explicit representations about training data provenance and bias testing results; hallucination liability allocation that does not rely entirely on blanket disclaimers; model drift SLAs with measurable thresholds and defined remedies; data deletion obligations extending to backup and archival systems; audit rights; and termination rights for material model changes or sustained performance degradation.

As Colin S. Levy's 2026 guide to AI vendor contracting notes, AI vendor lock-in carries distinct risks compared to conventional software lock-in. Losing access to a conventional SaaS tool is operationally disruptive. Losing access to custom model embeddings built over years of document analysis, or to workflows designed around specific model behaviors that a vendor can change unilaterally, can represent a material loss of organizational capability.

Key Evaluation Questions by Domain

The following evaluation questions represent the areas most likely to reveal AI-specific risks that standard procurement assessments do not surface. They are organized by the four dimensions described above and intended as a starting framework that organizations should adapt to their specific regulatory environment, deployment context, and risk tolerance.

Data Handling

Does the vendor use customer inputs to train or fine-tune models, now or in the future? Where is data stored, and under which jurisdiction's laws? What is the deletion timeline upon contract termination, and does it extend to backup systems? Which third-party subprocessors have access to customer data, and what security certifications do they hold?

Model Behavior

What is the vendor's documented hallucination rate for the relevant use case category? Is sandbox access available for adversarial testing before contract signature? How does the vendor disclose model updates that materially change output behavior? What performance SLAs exist beyond standard uptime commitments, and what remedies apply in the event of model drift?

Regulatory Compliance

Which AI regulations apply to this deployment context and geography? Has the vendor completed any applicable conformity assessments under the EU AI Act? What documentation does the vendor provide to support the deploying organization's own compliance obligations? How are compliance responsibilities allocated between vendor and deployer under the contract?

Contractual Protections

Does the contract include explicit representations on training data provenance and bias testing methodology? How is hallucination liability allocated, and does the vendor accept any responsibility for material output errors? Are there termination rights for material model changes, regulatory non-compliance, or sustained performance degradation? What data portability and switching rights apply upon contract expiry or termination?

Building a Sustainable Diligence Process

A point-in-time evaluation at initial procurement addresses only part of the risk exposure associated with AI vendor relationships. AI models are updated continuously, vendors are acquired, and the regulatory environment is evolving; a vendor that passed evaluation at one point in time may present materially different risk at a later one. A Forrester 2024 AI Governance analysis found that organizations implementing structured initial screening reduce unnecessary evaluations by roughly 40%, enabling governance teams to allocate capacity toward more complex requests. Achieving that efficiency while maintaining thoroughness requires a process that is systematic and consistently applied rather than improvised on a case-by-case basis.

Ongoing monitoring should be triggered at contract renewal, following material vendor updates, and after any ownership changes. The NIST AI Risk Management Framework provides a broadly adopted foundation for continuous AI vendor risk management, organized around Govern, Map, Measure, and Manage functions that translate into a practical ongoing monitoring cadence. Organizations that require NIST AI RMF alignment as a vendor evaluation criterion create an auditable standard that procurement decisions can be assessed against over time, rather than relying on point-in-time certifications that may not reflect a vendor's current posture.

Underpinning any diligence process is an organizational AI acceptable use policy that establishes what is required from AI tools before they can be deployed, and what the approval pathway looks like. Without this foundation, procurement reviews lack consistent criteria and employees have limited guidance for navigating tool selection on their own. The 223 monthly AI policy violations Netskope documents in the average enterprise reflect not primarily deliberate circumvention but the absence of sufficiently clear and accessible guidance. A diligence process that is perceived as slow or opaque will be routed around; one that is structured, time-bounded, and returns clear outcomes is more likely to be used as intended.

Conclusion

Closing the gap between how AI tools are procured and the risks they actually carry does not require organizations to slow AI adoption. It requires extending existing procurement frameworks with evaluation criteria specific to how AI systems are built, how they fail, and how the agreements governing them distribute responsibility for those failures. Organizations that build this capability as a repeatable institutional practice, applied consistently across every AI tool request, create the conditions under which AI capability can be deployed confidently, scaled responsibly, and audited clearly when questions arise.