A real AI acceptance test, from start to finish

98 questions to an assistant on the Spanish low-voltage regulation, the criteria agreed in advance, the six serious errors and the signed report.


A real test, from start to finish. An assistant on the Spanish low-voltage regulation, tested on 2026-11-14. I built it for this demonstration with an open model running locally, so it is not a client’s system. Result: unfavourable. Four of the six criteria are not met, including serious errors, where the agreed threshold was zero.

Criteria:

The errors in serious questions:

Before changing the model, fix how the document's tables are read and add a premise check. Then repeat the test with the same cases.