Ten Questions Smart AI Buyers Ask Before They Sign

Don't Retire. Rewire. — AI for People Who Refuse to Nod, Part 4

Here's a secret about AI vendor meetings: the smartest person in the room isn't the one who knows the most jargon. It's the one who asks the best questions. I learned this the fun way — by nodding along for years and then doing the homework. These ten questions will make you look like the most prepared buyer at the table, and they'll make your AI projects more likely to succeed, not less.

1. Where does my data go — exactly?

"The cloud" isn't an answer. Whose cloud, in which country, and is my data used to train their models? Great vendors answer this crisply. It's the foundation question — get it in writing.

2. When the AI gets it wrong, what's the plan?

Every AI makes mistakes sometimes. Mature vendors have thought about this: human review steps, confidence thresholds, rollback procedures. You're not looking for perfection — you're looking for a grown-up error plan.

3. Can I see how you tested it?

Ask for the evals: what data it was tested on, what the error rates were. Strong vendors are proud of this work and share it readily. Transparency here is a green flag, not a gotcha.

4. How do I turn it off?

Supervisors are asking banks this exact question, so you'll look prescient bringing it up first. A confident vendor shows you the off switch without flinching. It's just good change management.

5. What is the AI actually allowed to do?

Read-only? Draft emails? Move money? Every permission should be intentional. The best implementations start narrow and earn wider access over time — that's not caution, that's craftsmanship.

6. Can I see what it did?

"Show me the log of everything the agent did last Tuesday, and why." Audit trails are how you prove to your board — and your regulator — that the system works. Good vendors build this in from day one.

7. What happens when the model gets updated?

Models improve constantly, which is wonderful — unless your version changes overnight without warning. Ask how updates roll out and whether you get a heads-up. Predictability is a feature.

8. How does this fit my risk framework?

Bring your risk and compliance colleagues in early — they'll thank you, and the vendor conversation gets ten times smoother. Ask the vendor to walk through model risk, third-party risk, and data governance with you. The good ones have done this dance before and lead gracefully.

9. What's the real total cost?

License, implementation, the usage-based compute fees, the support tier you'll actually need. Get the year-two number, not the pilot number. Pilots are priced like first dates — plan the marriage.

10. If it doesn't work out, how do I leave?

Data export, format, timeline. Negotiating a graceful exit up front is just good business — and vendors who make leaving easy are usually the ones worth staying with.

This week's one assignment: take question #4 to your next vendor meeting. Just that one. You'll look like the most forward-thinking person in the room — because you will be.