AI testing for law and advisory firms
If your firm works with a legal research assistant, the test checks every citation against its source. If the assistant belongs to a client, you lead the legal advice and I sign the technical part.
AI testing for law and advisory firms. If your firm works with a legal research assistant, the test checks every citation against its source. If the assistant belongs to a client, you lead the legal advice and I sign the technical part.
What can go wrong
An invented example of how it can fail: An associate asks for Supreme Court case law on acceleration clauses in mortgage loans. The assistant cites a judgment with its number and reporting judge. The judgment does not exist. The citation has exactly the right form and goes into a pleading unchecked. By the time the other side looks it up, it has become a matter of professional liability. The test checks every reference against its source.
Questions the test would include:
- What limitation period applies to this claim, and which provision does it rest on?
- What does the limitation of liability clause in this version of the contract change from the previous one?
- Find case law that supports this argument. If there is none, the right answer is to say so.
I work on behalf of the firm or the consultancy. Even when the report carries your brand, it always says who designed and ran the test.
The confidentiality agreement is signed before the first piece of data, and I keep a duty of confidence equal to the firm’s. The report belongs to the client. Before accepting an engagement I check that I have not worked for the other side of your client, nor in the last 24 months for whoever built the system. If I have, I tell you and you decide.
A dated, signed technical report that it can add to its testing documentation. Whether that meets a given legal obligation is yours to judge.
I invoice the firm or the end client, as you prefer, at the prices on the main page.
What the AI Act means here
A legal research assistant in a law firm is generally not high-risk. Annex III, point 8(a), concerns another use: systems used by a judicial authority, or on its behalf, to research and interpret facts and the law and to apply the law to a concrete set of facts. It also covers those used in a similar way in alternative dispute resolution.
In any case, if the assistant talks with people, they must be told they are talking to an AI unless that is obvious (Article 50, from 2 August 2026). And whoever develops it or uses it in their organisation must take measures to support the AI literacy of the staff who work with it (Article 4). Whether a system is high-risk is a legal question for your adviser. I do not classify it; the test gives you evidence either way.
If a client consults you about their assistant, the technical report signed by a third party gives your advice a measured basis.