ZynoviqExpert ground truth: what a competent professional considers correct
AI needs more than answers. It needs reliable professional ground truth: the expected outcome, the action that should follow, and the reason a practitioner would give for it. Zynoviq produces that ground truth with qualified professionals, set independently of the vendor of the AI being tested.
Why answers alone are not enough
A benchmark that only stores the right answer can tell you that an AI was wrong. It cannot tell you why, how serious the error is, or what the AI should have done instead. Professional work is full of conditions, exceptions and judgment calls, and a test that ignores them rewards fluent guessing.
Ground truth closes that gap. Each case records what a competent professional would conclude, the rule or practice that decides it, and the mistakes that commonly get it wrong. Where a written rule exists (the law, a standard, or your organisation's written policy) the answer is anchored to it and cited.
What professionals contribute
| Contribution | What it captures |
|---|---|
| Scenarios | Realistic, synthetic situations drawn from how the work is actually done |
| Expected answers and outcomes | What a competent professional would conclude |
| Expected actions | What the AI should do next — post, hold, ask, escalate or refuse |
| Reasoning and rationale | Why that answer is correct, step by step |
| Business rules and decision criteria | The rule, threshold or condition that decides the case |
| Exceptions and edge cases | Where the usual answer stops being right |
| Common mistakes | The wrong answers practitioners see most often |
| Evaluation rubrics | How to judge a partly correct answer, and how serious each error is |
From individual judgment to canonical ground truth
One professional's opinion is not ground truth. Every case passes through a chain designed to catch individual error and surface genuine disagreement.
- 01ProfessionalSolves the case and writes the rationale.
- 02Independent professionalSolves it separately, without seeing the first answer.
- 03Domain reviewerCompares both against the governing rule.
- 04AdjudicationA senior professional settles any disagreement and records why.
- 05Canonical ground truthVersioned, dated and traceable to its authority.
Read more about the quality chain, and how answers are tied to the law, dated and versioned in evidence and currency.
Every mission starts with a written specification
Before any professional writes a case, we agree a mission specification with you. It fixes what "correct" means, how it will be marked and what you will receive, so there are no surprises at delivery.
| The specification covers | What it settles |
|---|---|
| Customer and AI system | Who the mission is for, and the model, agent or assistant being tested or trained |
| Domain and task type | The professional area and the kind of task — answer, decide, act, reconcile |
| What counts as correct | The ground truth: the law, standard or written policy that decides each case |
| How correctness is marked | Grading: rule-based fields, professional marking of reasoning, or both |
| Expert level and review depth | The capability required of authors, and how many independent solvers and reviewers each case gets |
| Difficulty mix | The balance of routine, hard and edge cases |
| Evidence requirements | What authority each answer must cite |
| Prohibited content | What must never appear in a case — real personal data, for example |
| Review and acceptance criteria | How cases are reviewed, and what you check before accepting the delivery |
| Deliverable | Format and volume of what you receive |
| IP, confidentiality and reuse | Who owns the output, and whether any of it may be reused |
| Timeline | Agreed delivery dates and service levels |
| Commercial model | Fixed, recurring, licence or usage — quoted per mission |
See the eight-step workflow, or tell us what your AI needs to do.
The Professional Intelligence Unit
The output of a mission is a set of certified records. We call each one a Professional Intelligence Unit: a single case with everything needed to use it, trust it and trace it.
| Each record carries | What it tells you |
|---|---|
| Task, domain, sub-domain | What the case tests, and exactly where it sits — for example GST → input tax credit → eligibility |
| Difficulty | Routine, hard or edge case |
| Scenario | The synthetic facts and documents |
| Expected outcome and reference answer | What a competent professional would conclude or do, in full |
| Expert rationale | Why that answer is correct, with the rule behind it |
| Author and reviewer qualification | The capability each contributor is qualified in — by anonymous ID, never by name |
| Reviewer agreement and adjudication status | How far the independent solvers agreed, and whether adjudication settled the answer |
| Provenance and metadata | Where the case came from and how it was produced |
| Version and timestamp | Which version this is, and when it was certified ("valid as of") |
| Permitted use | Evaluation, training, or both — as licensed |
| Evaluation tags | Labels that let you slice results by capability, difficulty or failure type |
See a complete example, one synthetic case from draft to certification, in evidence and currency.
Where the rules are written — and where they are configured
For domains governed by law, such as GST, the expected answer rests on the statute, rules, notifications and binding rulings, with a valid-as-of date.
For enterprise processes inside systems like SAP, Oracle or Salesforce, there is often no single public rule. There, expected outcomes are set against your documented configuration and written policy, combined with established professional practice. We ask for those documents during scoping and state in each case which source decided it.
What ground truth can power
Training and test cases come from the same production system but never from the same items. Training and test separation.
Expert case subscription
Ground truth goes stale as rules change. With an expert case subscription, you receive a recurring stream of newly certified cases, a fixed number per month or quarter agreed in the contract, so your benchmarks stay current. Subscriptions are quoted per mission. Benchmarks and licensed packs · Formats and licences.
Expert ground truth: questions people ask
Who writes the ground truth?+
Professionals qualified in the specific capability the case tests, not just holders of a relevant credential. At least two solve each case independently before a reviewer sees it. How we qualify professionals.
What happens when professionals disagree?+
Disagreement goes to adjudication by a senior professional, who settles the answer against the governing rule and records why. Genuine, unresolved ambiguity is labelled as such rather than hidden. Adjudication.
Is any of it written by AI?+
No. Answers and rationale are written, reviewed and certified by people. No answer is written or certified by AI. Marking today is rule-based and human; AI-assisted marking of free text is planned, opt-in and always checked by people. The AI boundary.
What if the law or our policy changes?+
Every answer carries a valid-as-of date and the authority it rests on. When the rule changes, affected cases are revised through the full quality chain and released as new versions. Evidence and currency.
Can we use the ground truth in our own evaluation tools?+
Yes. Ground truth is delivered in structured formats designed to run in your existing evaluation or training pipeline. Formats and licences.