Key Institutional Considerations for Designing Effective Independent Verification Organizations

November 10, 2026

The short version

There is growing public and political interest in creating independent verification organizations (IVOs). But IVOs do not exist in a vacuum: they require specific parameters and skilled talent to conduct independent evaluations, and rely on an ecosystem of intermediaries and governance bodies for their process and oversight design.

Our new paper, Key Institutional Considerations for Designing Effective Independent Verification Organizations, looks at what IVOs need to assess AI tools in the contexts where they are deployed and used, so evaluation evidence is rigorous, robust, and trustworthy and serves companies, organizations, communities, and the public.

An IVO is envisioned as a licensed, independent, expert-led body that impartially assesses whether an AI system meets specific safety, risk, and trustworthiness criteria. The idea has broad, bipartisan support and is moving into law. But momentum is outpacing design.

We draw from existing institutions and parallel problem spaces to provide recommendations for robust institutional design. We make five main arguments, which address our primary questions:

IVOs should serve more than frontier companies. Evaluation has to cover AI tools where they are deployed and used, assessing everyday risks as well as catastrophic ones. Evaluation findings should only apply to a scoped “evaluation boundary” and should be updated when changes are made to the AI tool or its accompanying technology stack.
IVOs should have adequate and appropriate access to information to carry out evaluation. IVOs need information from both developers and deployers, set out in a pre-evaluation plan. If information is withheld or missing from the company side, “insufficient evidence” should be deemed a valid finding.
Evaluator professionalization and validation methods should be built out in parallel to IVO design. IVOs depend on certified evaluators, validated methods, and a designation process — all of which should sit outside the IVO itself. These fields, practices, and institutions are not fully developed; new requirements should phase in as evaluation science matures.
Funding an evaluation should not buy control over it. Companies should be required to pay into a pooled fund managed by a separately governed fund administrator that selects, assigns, and pays IVOs.
IVOs evaluate; their findings enable actions by other ecosystem players. IVO findings are evidence, not verdicts. Separate bodies set the rules, oversee IVOs, and act on findings. Many of those bodies do not exist yet.

Read the full paper: Key Institutional Considerations for Designing Effective Independent Verification Organizations

Why this matters now

The policy window for IVO design is now open. California, Connecticut, Illinois, and Virginia have enacted IVO or IVO-related laws, and two federal proposals are on the table. Much of this activity is concentrated on frontier companies, model-level testing, and catastrophic risk.

We argue that this focus is myopic. AI tools are in everyday use, already causing known harms in existing domains and sectors. In a survey cited in Stanford's 2026 AI Index, 91 percent of North American respondents said their organizations use AI in at least one business function. These organizations need evidence relevant to their domain and local context on their AI tool adoption and use. This is why evaluation needs to go beyond assessing the capabilities of underlying models.

Further, the current state of evaluation legislation puts frontier companies in the driver’s seat. Frontier companies are pushing to “embed” outside evaluators from organizations they select to conduct evaluations, raising concerns over the independence of evaluation questions, access arrangements, methods, and criteria for credible evidence.

The choices made now will decide which risks IVOs and their evaluations address, which companies they cover, and who their evaluations ultimately serve.

Five questions, five answers

The paper takes on five questions that keep coming up in IVO design. We provide a brief overview of where we landed on each, citing sections in the full paper where further details are available.

Figure 1. Five key questions in IVO design.

1. Whom do IVO evaluations serve? (Section 3)

More than frontier companies. IVOs should also serve enterprise companies & AI-adopting organizations that deploy and use AI tools, and the communities and public who depend on them. Not every AI tool needs an evaluation conducted by an IVO. For example, the same AI model may be used to screen job applicants and draft emails, but only the former may warrant an IVO evaluation. This is why evaluation in context is critical.

An evaluation should have a specific “evaluation boundary” that outlines the specific AI tool tested and under what parameters. Findings cannot be automatically carried over to a different setting, workflow, or group of users. When changes are made to an AI tool or its accompanying technology stack, the evaluation must be updated as well.

2. How can IVO mandates be designed to ensure adequate and appropriate access that enables oversight while protecting proprietary information? (Section 4)

Through scoped, protected access to whole-of-system information. Before testing starts, the IVO should prepare a “pre-evaluation plan” that clearly defines the question, system version, intended use, affected populations, proposed methods, and evidence needed to reach a conclusion. IVOs should be able to access information from developers and from the organizations deploying and using the tool.

Requests for information access should be adequate, appropriate, and proportional to the question at hand. If more information is needed, the IVO can file a request to the company outlining why. The IVO designation & oversight body can check that the access requested is justified, and an arbitration authority can settle access disputes if needed. If information is missing or withheld by the company side, the findings should detail “insufficient evidence” as a valid finding. This way, withholding information cannot be used to game an assessment.

A tiered reporting and publication structure across ecosystem players allows for proportional sharing of findings. We suggest that the company engaging in assessment and the IVO designation & oversight body should receive the full report and record. If needed, a regulator or court should also be provided the full findings. Where an evaluation draws on both upstream information from the developer and downstream information from the deployer or adopter, each party should receive findings relevant to its own system and responsibilities, but not the other’s confidential information. Dispute resolution bodies should receive findings relevant to their case; certification bodies, validation institutions, standards-setting bodies, and policymakers should receive anonymous, aggregated insights to inform curricula and governance improvements. Each evaluation should produce a structured statement on the evaluation boundary, overall evaluation finding, and limitations that is made available to the public.

3. How can a successful IVO regime be developed while appreciating the nascency of the evaluator workforce and science of evaluation? (Section 5)

Build the professionalization and intermediary institutions in parallel, and phase in requirements as evaluation science matures. IVOs depend on three things that must sit outside the IVO itself: (1) qualified evaluators, (2) a way to show that evaluation methods are valid, and (3) a formal process for granting, reviewing, and revoking IVO designation. None of these functions are fully developed today.

The current pool of qualified evaluators is small, and it overlaps heavily with frontier company staff and the evaluators they contract. That risks a "monoculture" that weakens political and ideological scrutiny. Moreover, evaluators need both domain knowledge as well as technical skill to conduct rigorous evaluation in context. We call for training and certification pathways, periodic recertification, and professional codes of conduct to professionalize the field of evaluation and improve the overall caliber of evaluators.

Evaluation methods need validating too. Showing that an evaluator can apply a method properly does not necessarily mean that the method itself is able to support the intended claims. Investing in “scientific foundation reviews” or audits helps test and improve evaluation methods.

Effective independent verification will require an IVO designation & oversight body to determine who qualifies as an IVO and to oversee their activities. Designation should be provided for defined categories of work, and periodically reassessed.

Given that neither standards nor institutions for establishing evaluator competence (e.g., evaluation curricula, proficiency testing, credentialing, or professional codes of conduct) and methodological validity (e.g., foundation reviews, or validation criteria and processes) exist yet, designation and oversight should be incrementally staged, improving as evaluation science and infrastructure mature.

4. How can IVOs be financially independent of the entities they are evaluating, without being a taxpayer burden? (Section 6)

Funding an evaluation should not buy control over it. Today, frontier companies choose and pay for their own evaluations. This introduces financial conflicts of interest that weaken the credibility of evaluation and can incentivize lenient evaluation approaches.

This "issuer-pays" model has failed before. Before the 2008 financial crisis, banks paid credit rating agencies to rate their securities. A bank that did not get a AAA rating from one agency simply went to another that would give it. Ratings stopped reflecting real risk, and securities backed by risky mortgages were labeled top-rated. When the housing market collapsed, so did their value.

We propose a pooled fund model that frontier companies and, eventually, enterprise companies & AI-adopting organizations are required to pay into. A separately governed fund administrator manages these monies and selects, assigns, and pays IVOs. Companies can challenge an assignment over a documented conflict or qualification gap, but they cannot choose their own evaluator. The fund administrator pays at agreed milestones. Acceptance turns on the condition that the work was conducted by relevantly skilled evaluators and that the work followed the approved pre-evaluation plan, with any changes documented and justified.

5. Which functions belong to IVOs, and which belong to other bodies in the assurance ecosystem? (Section 7)

IVOs evaluate; their findings enable actions by other ecosystem players. An IVO's job is to run bounded evaluations in specific contexts of use. Four principles divide the rest of the work:

IVO evaluation findings are evidence, not verdicts. Regulators, courts, insurers, and procurers should decide what the findings mean.
Those who set rules do not apply them. Lawmakers and standards-setting bodies should define what gets evaluated and how. Neither runs evaluations to preserve the independence of the evaluation.
Those who apply rules do not police themselves. An IVO designation & oversight body should qualify IVOs and review their work. Accredited certification bodies should certify evaluators. An arbitration authority and skilled IVO panels should handle disputes across the evaluation cycle when necessary.
Those who pay for evaluations do not control them. Companies should fund evaluations through the pooled fund. They should not be in charge of selecting evaluators or distributing funds so as to shelter findings from coercion.

At present, many of these bodies do not exist, or do not have an IVO-specific mandate. This is an urgent area for policy action.

Figure 2. The IVO ecosystem: who plays what role.

How an evaluation works

Figure 3 outlines our proposed IVO process across its ecosystem. Funding comes first: companies pay into the pooled fund before any evaluation begins.

Figure 3. The life of an IVO evaluation.

Decide (Section 3.1). Does this AI tool, in this specific use, require evaluation? And is an IVO required to conduct it?
Bid with a plan (Sections 4.2 and 6.3.2). IVOs bid by submitting pre-evaluation plans that explain their relevant domain and technical competency for the outlined question(s).
Select (Section 6.3.2). The fund administrator engages an allocation committee to select the IVO. Companies can challenge a conflict or qualification gap, but cannot choose their own evaluator.
Access (Section 4.2). Information is shared under security and confidentiality protections. The IVO designation & oversight body checks that the access requested is proportionate. An arbitration authority settles access disputes if needed.
Evaluate (Section 4.2). The IVO produces its findings: what passed, what failed, and what could not be assessed.
Respond and retest (Section 4.3). The company can take corrective action or raise reasoned disputes. The IVO retests corrections, and a skilled IVO panel reviews technical disputes. Unresolved issues remain visible in the assessment record.
Pay on milestones (Section 6.3.3). The fund administrator pays at agreed milestones on the condition that the work was carried out by relevantly skilled evaluators and followed the approved pre-evaluation plan, with any changes documented and justified.
Report and publish in tiers (Section 4.3). Each party gets the findings relevant to it, and the public gets an assurance statement.
Act (Section 7.2). Other ecosystem players make decisions based on evaluation findings. Regulators determine legal or corrective obligations, courts adjudicate liability, insurers underwrite coverage, and procurers decide on use. Aggregate insights help standards-setting bodies, validation institutions, and certification bodies improve standards, methods, and evaluator training. Communities and the public gain better insight into how the AI tool’s use affects them.

What this means

For regulators and legislators (Section 9.1)

Mandate IVO evaluation by risk and context of use, at the levels where AI is built and used, so that evaluation serves companies, sectors, communities, and the public. → Sections 3.1, 3.3, and 3.4
Require companies to share information that IVOs need for holistic evaluation, and allow insufficient evidence to be reported as a valid finding, so that withholding information cannot be used to secure a passing result. → Sections 4.1 and 4.2
Build the institutions IVOs depend on; phase in requirements as evaluation science matures. → Sections 5.1, 5.3, 5.4, and 7.2
Require companies to fund IVO evaluations without choosing their own evaluator or otherwise influencing assessment outcomes. → Sections 6.1, 6.2, and 6.3

For oversight and intermediary bodies (Section 9.2)

IVO designation & oversight body and accredited certification bodies: Grant IVO designation and evaluator certification for specific work and maintain it through quality performance. → Sections 4.2, 5.2, 5.4, and 8.2
Standards-setting bodies and validation institutions: Set minimum standards that are clear about what current evaluation methods can and cannot support. → Sections 5.3 and 8.2
Standards-setting bodies and validation institutions: Prioritize validating methods for evaluating AI tools in context of use. → Sections 5.3 and 8.1
Arbitration authority: Resolve access disputes by specifying what is necessary and proportionate to share for the evaluation, rather than by company preference. → Sections 4.2 and 7.2
Fund administrator: Assign and pay IVOs on achievement of milestones, not on evaluation conclusions. → Sections 6.1, 6.3, and 7.2

For organizations considering becoming IVOs (Section 9.3)

Build credibility through specialization. → Sections 5.4 and 6.3
Require evaluators to obtain professional certification and adhere to professional codes of conduct. → Sections 5.2 and 6.3
Treat the pre-evaluation plan as the basis of the engagement that sets the terms for evaluation access, bidding, and payment. → Sections 4.2 and 6.3
Scope each finding precisely to its evaluation boundary, so that it is not generalized from one deployment or use case to another. → Sections 3.1 and 4.3
Protect the independence of services provided and findings reported through impartiality and neutrality. → Sections 4.3, 5.2, 6.3, and 8.4

For frontier companies and enterprise companies and AI-adopting organizations (Section 9.4)

Frontier companies: Define the protections under which upstream information will be shared with IVOs for developer-level evaluations and evaluations of AI tools deployed by other parties. → Sections 4.1, 4.2, 4.3, and 8.3
Enterprise companies & AI-adopting organizations: Make IVO evaluation, scoped to an AI tool’s specific use, part of procurement, due diligence, and tool assessment processes. → Sections 3.1, 3.4, 4.2, and 4.3

Conclusion

The current arc of IVO evaluation is bending toward frontier companies and catastrophic risks. We set out an ecosystem that assesses everyday and catastrophic risks; covers frontier companies and enterprise companies & AI-adopting organizations; and, ultimately, benefits communities and people as a whole.