An idea, an honest list of what we do not know yet, and a pilot for the first three to five teams who would like to test it with us.

Written with Anton Griev, who is designing OVL’s architecture.

The Short Version

• Companies now run several AI assistants that answer questions about the same data. They are fast, confident, and on the same question sometimes different. Nothing in the stack checks them against one another, and most companies have no written-down definitions to check them against.
• OVL is our plan for fixing both halves. One product builds the missing definitions from the documents you already have. The other validates AI answers against them. Each feeds the other.
• OVL is pre-product. We are looking for three to five data teams for a six-to-nine-week pilot, from $15,000, credited in full toward your first year if you decide to continue. If the pilot is not for you, the appendix at the end is a ten-minute audit of your own stack that is useful on its own.

This is for you if your company runs an open lakehouse stack (Apache Iceberg tables read by at least two query engines, for example Trino, ClickHouse or Spark), several AI assistants already generate analytics, and you are the person who gets the call when two numbers disagree.

This is not for you if your analytics lives in a single warehouse with a single assistant, you do not use AI in analytics yet, or you are looking for a finished product today. We do not have one.

Ready to talk now? Book a 15-minute call with Kirill or email us.

How to read this: in 3 minutes, read this section and “The Pilot.” In 10 minutes, add “Three Problems, Stacked” and “The Idea.” The full piece takes about 25.

A Tuesday Before the Board Meeting

It is the Tuesday before the board meeting. The CFO asks the assistant built into the BI tool:

What was net revenue by market last quarter?

It answers in seconds, with a clean chart. 42.7 million.

In the finance team’s workspace, a different assistant, attached to a different query engine, answers the same question from the same Iceberg tables. 41.9 million. Two markets swap places in the ranking.

Neither query failed. Neither dashboard shows an error. Both answers are plausible, and both came from systems that are very sure of themselves. One goes into the board deck, because it arrived first and has the nicer chart. The other goes into the month-end close.

Six weeks later someone notices that two documents disagree and starts a search that takes a week. It ends in a function that treats a rounding rule differently in two engines.

This is an illustration, not a case study. The numbers are invented. The structure of the failure is not.

Here is a small, real example of that structure. In October 2026 a ClickHouse developer opened a pull request removing three function aliases from ClickHouse’s experimental Trino-compatible SQL dialect, because they “silently returned results different from the Trino functions of these names.” The names matched. The results did not.

That is the whole problem in one sentence: the same name does not guarantee the same answer. What makes it uncomfortable is not that an AI got something wrong. People get things wrong all the time. It is that nothing in the stack was designed to notice.

Illustration — Same table, two answers. The numbers are invented.

Answers Got Cheaper. Trust Did Not.

A few years ago an analyst wrote the query, and the person who wrote it could explain it. Now every tool in the stack has an assistant: the BI tool, the query engine, the notebook. They generate SQL, run it and draw the chart. The cost of an answer fell to seconds. The cost of knowing whether the answer is right did not move.

Surveys and regulators have noticed the general shape of this, though not this exact failure. EY’s 2025 survey of 975 C-suite leaders in 21 countries, at companies with more than $1 billion in revenue, found that 99% reported financial losses from AI-related risks, and 64% lost more than $1 million. That survey covers AI risk broadly (regulatory non-compliance, biased output and more), not analytics errors specifically, so we use it as context, not as proof.

FINRA’s 2026 oversight report, written for broker-dealers, is closer. It names hallucinations as a risk of generative AI, meaning output that is inaccurate or misleading yet presented as fact. For AI agents it adds acting without human validation, and multi-step actions that are hard to reconstruct afterwards. Most of our readers are not brokerages. The logic still travels: an answer that looks authoritative and cannot be reconstructed is a governance problem, whoever you are.

The failure is rarely dramatic. It is quiet. A number is a little off, it travels to a slide, and the decision gets made faster than anyone can verify it. We wrote about the broader version of this shift in The Data Layer Is Becoming A Trust Layer.

Follow one answer from question to decision and you can count the places where an error can enter.

Figure 1 — Where an error can enter an AI answer.

Three Problems, Stacked

There is not one problem here. There are three, stacked on top of each other. Fixing one leaves the other two.

1. The assistants disagree, and nobody checks across the stack

Every engine and BI vendor now ships its own assistant, and each one checks, at best, its own work. Snowflake’s Cortex AI Guardrails, generally available since April 2026, protect against prompt injection and jailbreaks. Databricks’ June 2026 launches, Genie One and Genie Ontology, are likewise platform features. These are sensible products for their platforms. As far as we can tell, neither is designed to compare what Trino says with what ClickHouse says about the same table.

And the gap is real even when storage is shared. Two engines reading the same Iceberg snapshot can still differ in NULL handling, time-zone and date-function defaults, JOIN semantics and implicit casts, window-frame defaults, decimal and timestamp representation, and ordering. Anton wrote about this in detail in Your SQL Ran. Your KPI Is Still Wrong.

2. There is nothing to check against

To validate an answer you need the definition. What is “net revenue”? Who owns it? Which version is current? In most companies the answer lives in a Confluence page from two years ago, a Slack thread, half-filled dbt YAML, a spreadsheet, and the head of one long-serving analyst.

A data catalog is the standard answer, and a good one. But a catalog is only as good as what is in it, and filling one by hand is a multi-month project that many teams start and few finish. So the validator, if you had one, would have nothing to validate against.

3. The right metric for the wrong decision

Even with a correct AI and a perfect catalog, a technically correct answer can be wrong for the decision at hand. Gross merchandise value goes into a board report where recognized revenue is the canonical measure. Data refreshed weekly informs a “should we hire next week” call. A comparison runs across a silent change in methodology.

This happens without AI. AI makes it faster and more confident. We call it decision-data fit, and we want to be straight about its status: it is a hypothesis we are testing, not a finding. More on that below.

Figure 2 — Three problems, stacked. Fixing one leaves the other two.

That is why a single-purpose tool does not close the gap. A validator with no catalog has nothing to validate against. A catalog with no validator never learns where it is thin. And neither helps with the third problem.

Why the Obvious Fixes Do Not Fix It

If you are thinking “my platform vendor will solve this,” that is a fair first reaction. Here is our reading of the landscape, based on public information as of October 2026. If we have misread your product, tell us and we will correct it.

Approach	What it typically checks	Across engines?	Uses your definitions?	Builds the missing definitions?
Platform-native guardrails and semantic layers (Snowflake, Databricks)	Their own assistant’s work, inside their platform	No	Yes, if you model them there	Partly, inside the platform
General AI evaluation and observability (Braintrust, Fiddler)	Model behavior: quality, safety, policy	Not their focus	Typically not	No
Data observability (Monte Carlo, Metaplane)	Whether data is fresh, complete and unbroken	Not the point	No	No
Catalogs (OpenMetadata and similar)	Store the definitions	Not the point	Yes	Largely what people enter or curate
Semantic-layer automation (Solid)	By its own description, builds a semantic layer for enterprise data and validates how AI systems use it	Not stated	Yes	Yes
OVL (planned)	AI answers against your definitions	Yes	Yes	Yes, from your documents

Two things stand out.

General AI evaluation is crowded and being absorbed. In the past year, Check Point bought Lakera (closed October 2025, reportedly around $300 million) and Cisco completed its acquisition of Galileo in May 2026 and is folding it into Splunk Observability. Braintrust raised $80 million at an $800 million valuation, and Fiddler $30 million for what it calls the control plane for AI. We do not think the world needs another general evaluator, and we are not building one.

The gap we see is narrower and deeper: validating analytics answers against your definitions, across your engines. As far as we can tell, nobody is doing cross-engine result consistency for AI-generated analytics. We looked, and we may have missed someone. If so, please tell us.

The closest neighbor is Solid, which raised a $20 million seed round in February 2026 and describes a “highly accurate semantic layer” for enterprise data, plus validation of how AI systems interact with it. That is close to our idea in spirit. We start from open lakehouse stacks, where definitions often live in prose documents, and we compare results across engines. We do not know how far their product goes on either point, and we would not assume it stops short. We take their funding as a good sign for the problem, not a threat.

The Idea: Two Products and One Loop

OVL is two products joined by a loop.

OVL Catalog Synthesis reads the scattered places where your definitions already live: Confluence, Notion, Slack, Git, dbt YAML, BI-tool descriptions, spreadsheets. It extracts small atomic facts (“net revenue excludes refunds,” “owner: finance”), reconciles them across sources and flags where sources conflict. Then a person decides: accept, edit or reject. The output is structured and goes where your tools already look: entries in an OpenMetadata glossary, dbt MetricFlow definitions as pull requests, governance tags. Every entry keeps its provenance, a pointer back to the document it came from. Nothing is published without a human.

OVL Validate sits between the AI and the reader. It takes the question and the generated query and checks them against the catalog. There are eleven checks, eight of them deterministic (we list them in the appendix). It returns evidence, not a score, and three possible verdicts: equivalent within the supported scope, different, with the specific evidence, or inconclusive when a safe comparison is not possible. Abstaining is a feature. A validator that turns uncertainty into a green checkmark is worse than none.

The loop is what makes the two more than their sum. When Validate fails on something, the failure points at a specific hole: this metric is ambiguous, undefined or has no owner. That hole goes back to Synthesis, which goes looking in your documents, proposes an entry, a person curates it, and the next validation is sharper. Usually documentation is written and someone hopes it is right. Here, a failing check tells you exactly which definition is missing. Validation becomes the ground truth that documentation has to justify.

Figure 3 — Two products and one loop (planned).

Figure 4 — What happens to one answer in OVL Validate (planned).

We plan to build it in the open: an open-source core under Apache 2.0, on open standards that have only just converged. The first version of the Open Semantic Interchange specification appeared on 27 January 2026. OpenMetadata now exposes its metadata to AI agents through an MCP server. Apache Polaris became a top-level Apache project in February, and the Fivetran and dbt Labs merger closed on 1 June. A tool built natively on these standards is possible now in a way it was not a year ago.

It helps to say what OVL is not. It is not another LLM evaluator. It is not a data-quality tool, which checks that the data is right, while we check that the AI’s reading of the data is right. It is not another data catalog, because it feeds the one you already have. And it is not guardrails against prompt attacks.

A Walkthrough on a Stack That Does Not Exist Yet

Everything in this section is a design walkthrough on an invented stack. No number or finding here is a result. The point is to show what a pilot would try to do.

Imagine a mid-size company. Eighty dbt models, a dozen with descriptions. Two hundred Confluence pages, some with metric definitions. A Slack channel with months of “wait, which revenue?” Trino and ClickHouse over Iceberg tables. Two AI assistants and a notebook copilot.

Phase one: Synthesis. It crawls the sources, extracts candidate facts and reconciles them. What comes out is not a finished glossary. It is a review queue: candidate terms with a confidence level, sources and conflicts. For example: Finance’s Confluence page says net revenue excludes refunds, the dbt model includes them. The head of data spends a couple of hours accepting, editing and rejecting. Accepted entries go out as glossary entries and pull requests. The first useful artifact is the list of conflicts. It may be the first time anyone has seen them side by side.

Phase two: Validate. An analyst asks an assistant: What was net revenue by market last quarter? Validate returns a report card:

1. Cross-engine. Trino and ClickHouse results disagree: two markets swap places in the top ten. Evidence attached, with the first rows that differ.
2. Snapshot intent. “Last quarter” was resolved against a table snapshot taken several days into the new quarter.
3. Lineage. An upstream table failed its last two scheduled runs.
4. Definition. The query used net_revenue, which is deprecated. The current definition is net_revenue_v2.
5. Governance. The result includes rows for EU customers with no region check.
6. Inconclusive. One comparison could not be made safely, because a function’s semantics fall outside the supported subset. Validate says so and stops, instead of guessing.

Illustration — A design walkthrough of a validation report on an invented stack.

Phase three: the loop. Flag four shows that the current definition has no recorded owner. Synthesis re-reads the sources, proposes an owner and a definition text, a person curates it, and the next check is sharper.

That is the idea in miniature. A pilot is how we find out how much of it survives contact with a real stack.

What We Know, What We Assume, and How to Prove Us Wrong

We would rather show you our assumptions than hide them. Here are the five that matter most. None is validated yet. The pilot is how we plan to find out.

#	What we assume	How we test it	If we are wrong
1	Teams on open lakehouse stacks run three or more AI assistants that generate analytics.	Stack reviews of about twenty target companies, plus conversations.	We are early; revisit in a year, or narrow to a single BI surface.
2	Disagreements between engines or assistants reach decision-makers and cost real time.	About ten conversations asking: “Have you seen Trino and ClickHouse give different answers?”	The pain is too rare for a top-down sale. We would have to narrow to a within-platform product, which is hard.
3	Platform-native checks do not cover open lakehouse stacks.	Stack analysis of target companies.	The market has more competition than we think; we sharpen the vendor-neutral position.
4	OpenMetadata, OpenLineage and the semantic standards are common enough in this segment to build on.	Community survey and pilot conversations.	More integration work per customer, and slower pilots.
5	The mismatch between a decision and the data used for it is a pain buyers recognize, even without AI.	Count real cases during the pilot.	We stay with an AI-specific message.

How to prove us wrong: write to us with the stack you run, and the last time two numbers disagreed. Or did not. Both answers are useful.

The Pilot

We are looking for three to five teams to run a pilot with us. It is a real working engagement, not a demo. If you already know it fits, book a fit call or write through the contact form.

Who it is for

• 200 to 3,000 employees.
• An open lakehouse stack: Iceberg and at least two engines (Trino, ClickHouse, Spark), plus dbt.
• Several AI assistants already generating analytics.
• A catalog in any state: empty and sparse is fine. It is the best case for the first phase.
• Plenty of scattered documentation: Confluence, Notion, Slack, wiki, dbt YAML.
• The head of data, or someone empowered, on the call.
• Likely fits by industry: iGaming, fintech, B2B software, ad tech, healthcare and crypto. EU-based teams are a priority.

What happens

Six to nine weeks from kickoff, in three phases. Before kickoff there is a short scoping call and paperwork (NDA and a data processing agreement), usually two to three weeks.

1. Catalog Synthesis. We crawl your sources and build the review queue with your data lead. Accepted entries land in your OpenMetadata glossary and as dbt pull requests.
2. Validate. We run the checks on real AI-generated queries from your analysts, against the glossary built in phase one, including a comparison across your engines.
3. The loop and the review. Gaps found by Validate go back to Synthesis. We hold weekly check-ins and finish with a review session and a written report.

Figure 5 — The pilot at a glance.

What we need from you

• A kickoff call of one to two hours, a thirty-minute weekly check-in, and a one-hour final review.
• Read-only access to your metadata catalog (if you have one), your dbt manifest, one or two BI tools, and query history from Trino and ClickHouse.
• Five to ten representative AI-generated queries from your analysts.
• Roughly ten to fifteen hours of your team’s time across the pilot, mostly your data lead’s.

How your data is handled

This is the first question we expect, and we would rather answer it properly than with a slogan in a blog post. Before any access is given, we agree with you in writing how your data is handled: where processing runs, which model providers, if any, are involved, and what leaves your environment. An NDA and a data processing agreement come first. If our answer does not satisfy your security team, we stop before kickoff.

What you get

• A glossary of curated terms in OpenMetadata, plus dbt YAML pull requests, each with provenance back to its source document.
• Findings from running Validate on your real queries, including a cross-engine drift report: where your engines give different answers to the same question.
• An honest accuracy benchmark of your AI assistants across the validation categories.
• A prioritized list of fixes for the highest-risk gaps.
• A final review session.

What we are aiming for: at least thirty reviewed glossary terms, at least three real failure modes that would have shipped wrong (one of them a cross-engine inconsistency), and at least one executive-facing workflow validated end to end. These are targets, not promises. If your stack turns out to be clean, we will say so, and that is a finding too.

What it costs

From $15,000, credited in full toward your first year if you decide to continue. The exact price depends on how many sources and tools we connect, and we agree it before the pilot starts. We ask for no logo, no quote and no case study. If we learn something useful from your stack, we use it only in anonymized form, as failure modes. We would be glad to write a case study together if you want one, and nothing is published without your approval.

Why not free? Because a free pilot tells us less. Free work tends to slip down the priority list, and we would rather work with fewer teams who care.

What if it does not work

Then we part cleanly. You have paid for a clear answer, you keep the glossary, the findings and the reports, and the credit simply does not apply because you did not continue. No obligation, and the door stays open. We would rather end a pilot honestly than push a deal that does not fit.

Who it is not for

If you run a single warehouse with a single assistant, if you are not using AI in analytics yet, or if you want a finished product to buy today, this is not the right moment. Tell us anyway if you would like us to write when that changes.

Who We Are, and What We Cannot Do Yet

Anton Griev has spent more than fifteen years as a technical leader and CTO across iGaming, fintech and telecom, and builds OVL’s architecture. His public repositories include a Helm chart for running Cube.js on Kubernetes, a DWH Helm scaffold, a JSON-API-to-Kafka loader and a high-availability fork of Cube.js. It is infrastructure work around the stack OVL is built for.

Kirill Artsymenia runs Proof Engine, where he works on demand validation and go-to-market for new products. In OVL he owns that side: customer conversations, pilots and partnerships.

What exists today: OVL itself is pre-product, and there is no public code yet.

What we cannot do yet:

• We have no production deployments.
• Our first scope is narrow: Trino and ClickHouse over Iceberg, with dbt and OpenMetadata.
• We will not replace your governance function. OVL is a tool for it.
• We will not claim a check is deterministic when it is not.

That is why this is a pilot and not a sales pitch.

If You Back Data Infrastructure

We will keep this short. The window we see: open semantic and metadata standards have just converged, general AI evaluation is being absorbed by large vendors, and platform vendors are building within their own walls. The bet is that a vendor-neutral layer, open at the core, that builds the missing definitions and validates AI answers against them is useful on lakehouse stacks, where no single vendor sees the whole picture. We are testing that bet with the pilot.

If you invest in data infrastructure or AI governance and would like to see the deck, write to us.

How to Reach Us

If you run data for a team that fits the pilot: send us three lines about your stack: which engines, which assistants, and where your definitions live. Then, if there is a fit, a fifteen-minute call.

• Book a 15-minute fit call with Kirill
• Write through the contact form
• Email kirill@proofengine.studio (subject: “OVL pilot”)

If you work with these stacks and think we are wrong: please say so, on the same addresses. The most useful reply you can send is the last time two numbers disagreed in your company, and what it cost.

We usually reply within 24 hours.

---

Appendix A: A Ten-Minute Audit of Your Own Stack

Eleven questions, one for each check in OVL Validate. Answer yes only if you could show the evidence today, not if you believe it is probably fine. Each “no” is a place where a wrong AI answer can reach a decision without anyone noticing.

#	Can you tell, today, whether…	What it catches
1	an AI-generated answer used the current, approved definition of each metric?	Deprecated or look-alike metrics
2	an answer silently dropped a filter or dimension that the question implied (a market, a date range)?	Missing or wrong filters
3	the aggregation is sound: no summed ratios, no double counting after a join, no averages of averages?	Aggregation errors
4	a “growth” or “trend” claim ignores seasonality, a partial period or a methodology change?	Misread trends
5	one number mixes currencies or units?	Currency and unit errors
6	the sources behind an answer were up to date when it ran?	Stale data, broken upstream jobs
7	an answer exposed rows or columns the asker should not see (personal data, row-level rules)?	Governance breaches
8	the question itself was coherent before an answer was generated (ambiguous period, ambiguous metric)?	Bad questions
9	two of your engines, asked the same question, would give the same result, and anything compares them?	Cross-engine drift
10	when someone asks for “last quarter” or “as of month end,” which table snapshot the answer actually used?	Wrong snapshot
11	the metric, its freshness and its lineage are suitable for the decision being made, not just correct?	Decision-data mismatch (our hypothesis)

Score yourself. 0 to 3 yes: most AI answers in your stack go out unchecked. 4 to 7: partial coverage, with the gaps probably concentrated where engines or tools meet. 8 to 11: unusually well covered, and you have probably built some of this yourself. We would like to hear how.

Appendix B: A Short Glossary

• Lakehouse: analytical data stored in open table formats on object storage, readable by several query engines.
• Apache Iceberg: an open table format that lets several engines read the same tables and their history.
• Query engine: software that runs SQL over the data (Trino, ClickHouse, Spark).
• Semantic layer / metric definition: the agreed, written definition of a business measure such as “net revenue,” with its filters and owner.
• OpenMetadata: an open-source metadata catalog; here, the place where approved definitions and glossary terms live.
• dbt and MetricFlow: tools for transforming data and for defining metrics as code.
• Lineage: the record of where a table or number comes from and what it feeds.
• Provenance: a pointer from a catalog entry back to the document it was extracted from.
• Cross-engine consistency: whether two engines give the same result for the same business question.

FAQ

Is OVL a product I can buy today?

No. OVL is pre-product and has no public code yet. We are looking for three to five teams for a pilot, from $15,000, credited in full toward the first year if you continue.

How is this different from the guardrails or evaluation tools I already have?

Platform guardrails and semantic layers check a vendor’s own assistant inside that vendor’s platform, and Snowflake’s Cortex AI Guardrails protect against prompt injection and jailbreaks. General AI evaluation tools check model behavior. OVL is meant to validate AI-generated analytics against your own definitions, across your engines, and to build those definitions from your documents.

What does the pilot cost, and what if it does not work?

From $15,000, credited in full toward your first year if you continue, with the price agreed before the pilot starts. If it does not work, you keep the glossary, findings and reports and owe nothing further. We ask for no logo, quote or case study.

What access do you need?

Read-only access to your metadata catalog if you have one, your dbt manifest, one or two BI tools, and query history from Trino and ClickHouse, plus five to ten representative AI-generated queries. Before any access is given, we sign an NDA and a data processing agreement and agree in writing how your data is handled: where processing runs, which model providers are involved and what leaves your environment.

Sources

• EY, survey on responsible AI governance and financial losses from AI-related risks (October 2025).
• FINRA, 2026 Annual Regulatory Oversight Report; summary by Debevoise, FINRA’s 2026 Regulatory Oversight Report: Continued Focus on Generative AI and Emerging Agent-Based Risks.
• ClickHouse, pull request #126011, “Follow-up to the Trino compatible syntax in ClickHouse SQL” (15 October 2026).
• Snowflake, Cortex AI Guardrails (General availability), release note of 20 April 2026.
• Flexera, Databricks Data + AI Summit 2026 recap.
• SecurityWeek, Check Point to acquire AI security firm Lakera (the price is a press estimate), and Check Point, Q3 2025 financial results, which report the deal closed on 22 October 2025.
• Splunk, Splunk Observability + Galileo: Bridging the AI Trust Gap (28 May 2026).
• SiliconANGLE, Braintrust lands $80M Series B to become the observability layer for AI (17 February 2026).
• Fiddler, Series C announcement (27 January 2026).
• Calcalist, Solid raises $20 million seed round (18 February 2026).
• Snowflake, Open Semantic Interchange: specs finalized (27 January 2026).
• OpenMetadata, release notes (the MCP server).
• Apache Polaris, Apache Polaris graduates to top-level project (19 February 2026).
• Fivetran, Fivetran and dbt Labs complete merger (1 June 2026).
• Proof Engine blog, Your SQL Ran. Your KPI Is Still Wrong. and The Data Layer Is Becoming A Trust Layer.

OVL is pre-product. Everything described here as planned is a design hypothesis, not a shipped capability.