Five questions to ask about the AI your team is already using

If your organisation is like most, AI arrived without a decision. Someone turned on a copilot in the editors. Someone else started using an assistant for tenders. A vendor added an "AI feature" to a system you already pay for. Eighteen months later it is everywhere and nobody owns the picture.

That is not a failure. It is how every useful technology arrives. But before you put more weight on it, it is worth being able to answer five questions. Each one can be answered with evidence rather than a feeling, and I have noted where the evidence lives.

1. What AI is actually in use?

Not what was approved. What is in use. Your identity provider lists every application your people have consented to; the finance system lists every subscription; your code repositories carry the fingerprints of AI-authored changes. Put those three together and you have an inventory, with an owner and a data exposure for each tool. Most organisations are surprised by the length of the list and by how much of it nobody approved.

2. Is the AI-produced work real?

This is the one almost nobody asks. AI-produced code and content looks finished. The question is whether it holds up when measured.

For code, take a sample of recent AI-authored changes and count, the same way every time: tests that actually assert something versus tests that pass by construction; files and modules over a sensible size; duplicate paths to the same outcome; fallbacks, special cases and "fix later" notes per thousand lines; status claims with nothing proving them. Then trace two real user journeys end to end and count the hops. You want numbers with denominators, not a demo.

For documents, the same idea: sample the outputs, check every factual claim back to a source, and count the ones that cannot be traced.

3. Can you show what it did?

For any AI system that touches a customer or a decision: what did it access, what did it produce, who approved it, and can you show that from a log rather than a policy document? If the answer to the last part is no, you have a system whose behaviour you cannot explain to an auditor, a regulator or a customer, however well it performs on the day.

4. What is leaving the building?

Who is putting what into public AI tools? Client material, personal information, code, strategy documents? Usually the honest answer is "we don't know", which is itself the answer. The fix is rarely a ban; it is knowing, and then giving people a sanctioned tool that keeps the data where it belongs.

5. What is it costing, against what it returns?

Seats, tokens, subscriptions and the vendor's AI surcharge, added up, against anything you can actually measure it having changed. Not to cancel it, but because the number is usually larger than anyone thinks and the return is usually narrower than the pitch, and both facts belong in the next budget conversation.

What to do with the answers

If you can answer all five with evidence, you are ahead of nearly everyone, and the next step is scaling what works.

If you can answer two or three, you know exactly where to look.

If you can answer none of them from a log or a measurement, you are not alone, and it is worth spending two weeks finding out before the next twelve months build on top of it.