nullius: AI acceptance testing
Check, before launch, whether your AI assistant gets wrong what matters. Independent acceptance tests against criteria signed in advance, with a dated and signed report.
nullius: independent acceptance testing for AI assistants and systems that answer from your documents, by Yago Mendoza, in Barcelona.
Get your AI assistant tested before it goes live. For companies about to put into use an AI assistant or search tool that answers from their documents. I check with real questions from your work whether it gets the important things wrong, and hand you a signed report with every error and its proof.
The test fits the platform you already use. The test treats your assistant the way a user sees it: it asks questions and reads the answers. So it does not matter which model drives it or what tools it is built with, and nothing in it has to change. To connect it, a test access is enough: an API, or the same screen your team uses. I have already run the same demonstration test on the three marked models.
- Claude (already tested)
- Mistral (already tested)
- Qwen (already tested)
- Azure OpenAI
- AWS Bedrock
- Google Vertex AI
- LangChain
- LlamaIndex
- Ollama and local models
Names cited only to indicate compatibility. The trademarks belong to their owners, with no commercial relationship.
Fixed-price engagements:
- Technical diagnosis, €650 plus VAT, one week. To decide with judgement before spending more: for example, whether to accept an integrator's delivery or whether a full test is needed.
- Comparison, €3,900 plus VAT, five weeks. To choose between two configurations or two vendors with the same questions, instead of trusting each one's demo.
- Acceptance test, €5,500 plus VAT, seven weeks from the moment the test questions and access are ready. To check, before accepting it, whether the assistant gets wrong what matters, with a result you can show your board or your committee.
Sectors:
- Healthcare: Doses, allergies, protocols and alerts, before an answer reaches a clinician or a patient.
- Pharma and life sciences: Product information, standard operating procedures and regulatory documents, quoted to the letter.
- Law and advisory firms: Clauses, deadlines and case law, without a single invented reference.
- Banking and insurance: Terms, cover, limits and fees, explained exactly as the contract states them.
- Industry: Manuals, shift reports and maintenance procedures, including the ones from the worst day.
- Energy and utilities: Technical regulations, tariffs and incidents, with the current version of each document.
- Engineering and construction: Standards, tender documents and specifications, never mistaking one edition for the last.
- Logistics and transport: Orders, routes, customs and deadlines, from end to end.
- Retail and customer service: Returns, warranties and promotions, under the policy in force today.
- Public sector: Procedures and rules that can be explained to any citizen, with their source.
- Education: Academic regulations, deadlines and course guides, without inventing requirements.
- Consultancies and integrators: An independent test that goes with your delivery to the end client.
A real demonstration test, of 2026-11-14: 98 questions to an assistant on the Spanish low-voltage regulation. Result: unfavourable. It fails six serious questions, where the agreed threshold was none.
It is not a certification nor a conformity assessment under Regulation (EU) 2024/1689.