ZynoviqThis is what you receive.
Below is a shortened evaluation report for an imaginary GST assistant on the India GST AI Benchmark — input tax credit. The figures are invented to show the structure. Your report follows the same structure, for your own system.
Testing record
Every report opens with the record an auditor, risk committee or buyer needs: what was tested, against what, when, with what result, on what basis, and how it compares with earlier runs.
| Field | Entry |
|---|---|
| System | An imaginary GST assistant: product name, version and configuration as declared by the customer (named in a real report) |
| Benchmark | India GST AI Benchmark — input tax credit · Illustrative version 1.2 · law valid as of 1 October 2026 |
| Run | Illustrative Run date, customer test environment, option B (customer ran the questions, Zynoviq marked), 500 cases run |
| Result | Illustrative 72% correct overall · scores by topic below · critical failures listed first |
| Basis | Certified answer, legal authority and professional review for every case; structured fields marked by rule, free text by qualified professionals |
| History | Illustrative First run on this benchmark version; no earlier runs to compare — later reports show regressions and recoveries against each previous run |
Report summary
| Item | Result |
|---|---|
| System tested | An imaginary GST assistant (a real report names the product and version) |
| Domain | GST — input tax credit |
| Cases | 500 synthetic cases, law valid as of 1 October 2026 |
| Overall correct | Illustrative 72% |
| Confidently wrong | Illustrative 17% |
| High-severity errors | Illustrative 31 |
| Ambiguous cases (reported separately) | Illustrative 12 |
Score by topic
| Topic | Correct | Pattern |
|---|---|---|
| Tax rate lookup | Illustrative 91% | Strong |
| Invoice matching | Illustrative 86% | Strong |
| Blocked credits and exceptions | Illustrative 58% | Misses provisos |
| Reverse charge | Illustrative 61% | Treats notified services as normal charge |
| Place of supply (services) | Illustrative 55% | Confuses billing and delivery locations |
| Time limits for credit | Illustrative 64% | Uses outdated deadlines |
Three failed cases, explained
Illustrative The cases are synthetic; the legal points are real as of 1 October 2026.
Case 1 — Blocked credit exception (high severity)
- Facts: a factory with 400 workers runs a canteen required by the Occupational Safety, Health and Working Conditions Code, 2020; the catering bill carries ₹10,000 GST.
- AI answer: credit not allowed; food is a blocked credit.
- Correct answer: credit allowed; the law permits it where the employer is legally obliged to provide the service.
- Basis: CGST Act, 2017, section 17(5)(b), proviso; CBIC Circular 172/04/2022-GST.
- Pattern: rule applied without its exception.
Case 2 — Reverse charge on legal services (high severity)
- Facts: a registered company receives legal services from an individual advocate.
- AI answer: the advocate charges GST on the invoice; the company claims credit normally.
- Correct answer: the company pays the GST itself under reverse charge.
- Basis: Notification 13/2017-Central Tax (Rate), as amended.
- Pattern: notified reverse-charge service not recognised.
Case 3 — Deadline for claiming credit (medium severity)
- Facts: invoice dated February 2025; credit first claimed in December 2025.
- AI answer: allowed, within the September return deadline of the following year.
- Correct answer: not allowed; the deadline is 30 November following the end of the financial year, or the annual return date if earlier.
- Basis: CGST Act, 2017, section 16(4), as amended by the Finance Act 2022.
- Pattern: outdated rule.
What the report recommends
Illustrative
- Add reference material on blocked-credit exceptions and notified reverse-charge services.
- Update the time-limit rule; check any other deadline logic.
- Route place-of-supply decisions for services to the expert review queue until re-tested.
- Re-test after changes, and again after the next rule change.
Yours to keep
| Deliverable | What it is |
|---|---|
| Report | Overall score, score by topic and difficulty, high-severity errors, confidently-wrong rate, and the testing record above |
| Failed cases with correct answers | Every case your AI got wrong: the facts, your AI's answer, the correct answer, the reason and the law |
| Failure patterns | Failures grouped into recurring causes: missed exceptions, outdated rules, wrong facts, invented authority |
| Executive summary | Two pages for leadership or a risk committee: what the AI can be trusted with today, and what it cannot |
| Recommendations | What to fix first, what to route to human review, when to re-test |
| Regression set | The failed cases as structured records, ready to re-run after every change |
| Walkthrough call | With the domain lead who oversaw the marking |
| Audit record | Every case sent, every answer received, every judgment and reviewer, available on request |
| Custom cases built only for you | If you commissioned them: owned by you, priced as exclusive, quoted |
Licensed, not handed over
| Item | Why it stays with us |
|---|---|
| The shared case pool | It is built once and used to test many systems fairly; handing it over would let it leak into AI training and make future tests meaningless |
| The full answer key | Same reason: an exam is only useful if the answers stay private |
| Cases your AI got right | Kept in the pool for future re-tests; you see your scores on them |
This is what keeps your next re-test honest. Training and test separation · Data rights.
What you will not receive
- A certificate, approval or "passed" badge for your AI. We certify our answers, not your system.
- A claim that we independently evaluated, audited or assured your AI. "Independent" means only that the expected answers are set by qualified professionals independently of the vendor of the AI being tested. What we do not do.
- Anyone else's results, or yours shared with anyone else.
Formats
Report as PDF and web view; failed cases as a spreadsheet or JSON file; regression set as structured records; audit record as an export on request. No self-service dashboard today.
Sample evaluation report — FAQ
Is this a real report?+
No. The system, the scores, the counts and the run details on this page are invented to show the format. Nothing here describes a real AI system or customer. The three GST points are real and stated as of 1 October 2026.
What does "high severity" mean?+
An error with money or penalty impact, such as credit lost or tax underpaid. Medium means a wrong entry with the right tax; low means a citation or format error.
Can we share the report with our board, auditors or a buyer?+
Yes, within your organisation and with your advisers, described accurately as findings against certified ground truth, not a certification. The testing record is written so they can read it without us. Evidence.
In what format are results delivered?+
The report as a PDF and web view, failed cases as a spreadsheet or JSON file, the regression set as structured records, and the audit record as an export on request. There is no self-service portal.
Can we own the test cases outright?+
Only custom cases built just for you, priced as exclusive work and quoted. The shared pool and its answer key stay licensed, so the test stays fair next quarter.