Our method in ten minutes.
Everything an auditor, buyer or researcher needs to understand how a Zynoviq case is made and how a Zynoviq result is produced, on one page. Each section links to the full detail.
1. How cases are written
Every engagement starts with a written specification you approve: instructions to authors, accepted answers and alternatives, scoring rules, critical errors, difficulty mix, evidence requirements, prohibited content and acceptance criteria. Cases are then written from a fixed template (facts, documents, question, expected answer fields) by professionals qualified in the specific capability. Every case is synthetic: the patterns are real, every name, number and document is invented. Fixed, rule-based checks reject incomplete fields, invalid formats, signs of real data and duplicates. The eight steps · Synthetic source.
2. Who solves them, and how they are qualified
Solvers, reviewers and adjudicators are qualified professionals (Chartered Accountants, GST practitioners, Company Secretaries, payroll and SAP finance specialists in the live and on-request domains). Each passes identity and credential checks against the profession's register, then a timed calibration test of 25 cases with certified answers in the specific capability, then 30 pieces of probationary work fully reviewed. After that, every contribution keeps being measured against other solvers, adjudicators and hidden control cases. Internal scores route work and are never published or sold. At least two professionals solve each case independently; a higher-tier professional reviews. Qualification · Calibration.
3. Adjudication
Same answer for the same legal reason: on to review and the legal desk. Different answers, or the same answer for different reasons: a third solver. Still split: a senior adjudicator with 12 or more years of practice decides and writes the reasoning, never on a case they solved. Genuinely contested questions are marked ambiguous, kept out of pass/fail scoring and reported separately with consensus and dissent recorded. The tempting wrong answer stays in the record. Every certified record carries a 0 to 100 confidence score built from agreement, who agreed, review and adjudication, legal evidence and currency; items below 60 are not certified. Adjudication · Confidence.
4. Authority and dating
Agreement is not truth. The legal desk ties every final answer to the governing section, rule, notification, circular or ruling, checks the citation says what we claim and is still current, and records a valid-as-of date. No citation, no certification. For configured business processes with no public rule, the authority is the customer's documented configuration and written policy, named in the record. Evidence.
5. Marking
| What is marked | Rule | Mode |
|---|---|---|
| Yes/no answers and codes | Exact match against the certified key | Rule-based |
| Amounts | Match within ₹1 of the certified amount | Rule-based |
| Legal basis | The key section, rule, notification or circular must be cited; accepted alternatives listed in the key | Rule-based |
| Free-text reasoning | Marked by qualified professionals against the rubric in the specification | Human |
| Free-text reasoning, AI-assisted | Opt-in per customer; every disagreement and a sample of agreements reviewed by people; agreement rate reported | Planned |
Per case we capture the response, tool calls and steps where the integration exposes them, the final outcome, latency and errors. Critical errors are those the specification names as such. AI never sets, changes or certifies ground truth; AI-assisted marking only ever compares your AI's output with an answer people have already certified. The AI boundary · Sample report.
6. Training and test separation
Training data and benchmarks come from the same production system but never from the same items. Cases are assigned to a training pool or a held-out pool at creation and never move. Separate access rights keep training contributors away from held-out keys. Duplicate and near-duplicate detection runs across both pools. Answer keys sit in an isolated store readable only by the marking service and are never published for paid or confidential benchmarks. Every result records the benchmark version. We never sell improvement data for a system whose benchmark score we publish. Training and test separation.
7. Versioning and refresh
Every record carries a version, a valid-from date and, once superseded, a valid-until date and change reason. The legal desk tracks notifications, circulars and rulings as they are issued; affected records go back through the full quality chain and are released as new versions, with the old version kept in history. Test pools are refreshed quarterly and each pack's legal basis is fully reviewed at least once a year. Our target for re-certifying affected answers is 10 working days; it is a target stated in contracts, not a guarantee. Refresh.
8. Known limits
- Two reviews do not make an item 100% correct. High-risk items get an authoritative-source check, specialist adjudication or an independent calculation, and we track errors found after certification.
- Where qualified professionals reasonably disagree there is no correct answer to mark against; such cases are reported as ambiguous, not scored.
- A good score on a benchmark is evidence about that benchmark version, on that date, in your test environment. It does not mean the system is safe everywhere.
- Our reports are findings, not a certification, audit, approval or assurance of your AI. We also sell data that improves AI, and Zynoviq's separate division builds AI products, so we are not a disinterested party. How we stay separate.
- Coverage is limited to domains where qualified professionals are calibrated: GST live; payroll and company law on request; others with their stated status. Domains.
- Marking is rule-based and human today. AI-assisted marking of free text is planned, opt-in and always checked by people.
- Headline quality figures (first-pass agreement, adjudication rates, post-certification errors) are planned for quarterly publication from mid-2027; they are not published yet.
Method note: questions people ask
Can we cite this method in our own documentation?+
Yes. The method is published here so that a report can be read without us. Cite the page and the benchmark version named in your report. A PDF version is planned.
Does the method change between customers?+
No. The specification changes (topics, difficulty mix, scoring rules), but the quality chain, marking rules, separation controls and versioning are the same for every engagement.
Is a good score on a Zynoviq benchmark a certification?+
No. It is evidence that a named system, on a named benchmark version, on a named date, produced these results. It is not a certification, approval or assurance of the system. What we do not do.
Where is the full detail?+
The six detail pages under How it works: expert ground truth, quality chain, evidence and currency, data rights and the AI boundary.
Zynoviq