The people who ask do not know the answer, so nobody sees the mistakes. We measure how often your AI assistant is right, find the step that breaks, and check again every month.
For AI assistants in pilot or in production. Fixed price, agreed before we start.
Your team asksHow long was the new drug tested on patients?
Your AI assistant18 days. Source: assessment report, page 58.
Each phase can break on its own. So we test each one on its own: with vector search, a graph or an agent.
Pages are missing, or old versions sit next to new ones.
No public number yet. We measure it on your data.
Tables fall apart: “Day 182” becomes “Day 18”.
Page headers become pieces of their own, or a fact is cut off from its heading.
The right passage is in the index, but not in the top 10.
The right passage was found, but the answer leaves part out or cites the wrong source.
No public number yet. We measure it on your data.
In the example above, phase 2 broke. A better AI model would not help there.
How we measure search1 Found at all?2 Near the top?3 Really useful?How it works →
AI assistant for employees
About 70,000 pages from 8 sources: process handbooks, laws and industry rules. Answers were stuck at 51%. We measured every step on its own and found the cause in the search. Two fixes lifted the answers.
Documents change. New models come out. We set up a test bench for your system, and it checks every month that the answers stay good.
Do your documents reach the AI in one piece?
Always the first step. Can be booked on its own. Fixed price. We only need read access.
How good is your AI, step by step?
Fixed price. The price of step 1 is fully credited.
We build the fixes from the plan into your system.
Fixed price after the audit. You keep running the system.
Does your AI stay good when documents and models change?
3 months, renewed 3 months at a time. You run the system, we measure and advise.
Runs in your infrastructure, in the EU. No lock-in.
✓ Fits: system in pilot or in production · pharma, medical devices, energy, finance, insurance
✗ Does not fit: a first demo · you want us to run the system
A public benchmark tests somebody else's documents. New models are often not in it yet. Only a test on yours shows whether a model works on yours.
The result is not one score but, per stage: how often the right passage is found, what it costs, and how fast it is.
A benchmark winner does not help if your data may not leave the EU, or if only Azure is approved. We start with your rules, then measure what is left, on your documents.
turbopuffer, Qdrant, pgvector or Azure AI Search. Chosen by size, cost and where it may run.
Keyword, vector or both, and how to merge the results.
Which one, and if it pays for its cost and delay.
Hosted or on your servers, and in which region.
Each step has a fixed price. It depends on the scope: documents, sources and questions. We agree it before we start. The price of step 1 is credited against step 2.
No. The test bench runs in your infrastructure, in the EU. We only use services you approve. For step 1 we only need read access to the documents.
Mostly your subject experts: they check test questions and answers for the gold set. We do the rest. We name the effort for your team in the proposal, before we start.
Those tools mostly measure the end result. We measure each step on its own and compare alternatives on your documents. So you see which step breaks and what fixes it. We are happy to reuse the test questions you already have.
Yes. We work in your cloud (Azure, AWS, Google, STACKIT or IONOS) or on-prem, on your own servers. For example with turbopuffer, Qdrant, pgvector or Azure AI Search. We start with your rules and only test what you are allowed to use.
Yes. The test bench, the gold set and the reports are yours, and you can keep using them yourself. The ongoing check makes sure someone runs them every month, tests new models, and tells you when quality drops.
The ongoing check renews 3 months at a time. You can cancel at the end of any term. The gold set, the test bench and the reports stay with you.
It gives you evidence: test questions with proven answers, measurements per step, and an audit report for QA and notified bodies. You need that for documentation and proof. It is not legal advice.
Windtunnel, Berlin. I build and evaluate AI assistants and agents on company knowledge, also under MDR requirements. With hard documents: tables, scans and regulatory filings.
About 8 years in data and ML, 5 of them in enterprise consulting, for clients such as Airbus, Lufthansa, Deutsche Bahn and Hapag-Lloyd.
The name comes from my aerospace degree at KTH Stockholm: there, you test in a wind tunnel before you fly.
No slides. No demo. Five questions on your system, and an honest answer on fit:
Or email [email protected]