Twelve questions to ask before you sign an AI vendor contract

Photo: HD Wallpapers via stocksnap (CC0 1.0)
The contract that landed on a client’s desk last month ran to 41 pages, and the two clauses that would have caused the most trouble were on pages 27 and 38. I’ve been on both sides of this table: as a product manager writing the vendor’s terms, and more recently helping buyers read them. The questions below are the ones I ask now, in the order I ask them, and the reason each one matters. None of them are legal advice. All of them have cost somebody money.
The data questions #
One. Where does our data go, physically? Not “is it secure”. Which country, which region, which provider’s data centre. If the answer involves a model hosted outside Australia, you have a cross-border disclosure under the Australian Privacy Principles and you need to know that before the privacy officer finds out from a customer. Ask for the sub-processor list, and ask how you’ll be told when it changes. It changes.
Two. Is our data used to train or improve anything? The default in a lot of consumer-grade terms is yes. The default in most enterprise terms is no, but “no” sometimes means “not the base model” while still allowing fine-tuning, evaluation or “service improvement”. Get the specific words. Get them to survive a change of ownership of the vendor.
Three. How long is it retained, and by whom? Prompts, outputs, logs, embeddings, the lot. A vendor that keeps thirty days of prompts for abuse monitoring is normal. A vendor that keeps them indefinitely for “quality” is holding your customers’ data on a schedule you don’t control. The OAIC’s guidance on generative AI and privacy, published in late 2024, is the document I’d read alongside this clause; it’s short and it’s blunt.

The model questions #
Four. Which model, and who decides when it changes? This is page 27. Most AI products sit on top of a third-party model, and the vendor can, and will, swap it for a newer, cheaper or differently-behaved one. That’s usually good for you. It’s also a change to a system you’ve validated, and if you’re in a regulated process, or you’ve built prompts that depend on the model’s quirks, you need notice and a way to test before it lands. Ask for a version pin as an option, a notice period, and a rollback. You won’t get all three. Get two.
Five. What are the published evaluation results, and can we run our own? Any vendor worth signing with has measured its product on something. Ask what, on what data, and whether you can run the same evaluation on your data before you commit. A vendor that resists an evaluation on your inputs is telling you something about how the product performs on inputs it hasn’t seen.
Six. What happens when it’s wrong? Not philosophically. Operationally. Is there a confidence score? A human review path? A way to flag an output and have it excluded from whatever the system learns? And contractually, who wears the cost of an error the system made in the ordinary course? Most templates say you do. Some will negotiate. All of them will tell you the answer to this question is “it’s a tool”, which is true right up until the tool sends a customer the wrong contract.
The money questions #
Seven. What is the unit of pricing, and what makes it go up? Per seat, per token, per document, per call, per outcome, or some blend, and the blend is where the surprises live. Token pricing in particular scales with things you don’t control: longer documents, chattier customers, the vendor’s own decision to add a reasoning step that quadruples the tokens per request. Ask for a worked example at ten times your pilot volume. Ask what the price does if the underlying model gets cheaper. It will, and the vendor’s margin is the difference.
Eight. Is there a cap, and what happens at the cap? Hard stop, soft warning, or silent overage at list price. I’ve seen all three. Only one of them is acceptable for a system that processes customer requests, and it isn’t the last one.
Nine. What are we paying for that we could get from the model provider directly? This is the uncomfortable one. A lot of AI products are a thin layer over an API you could call yourself, and the honest ones will say so and charge for the layer: the workflow, the integration, the support, the compliance paperwork. The less honest ones charge as if they’d built the model. Knowing which you’re buying changes what you should pay and how locked in you should let yourself become.
The exit questions #
Ten. How do we get everything out? The data, obviously, but also the prompts you’ve tuned, the evaluation sets you’ve built, the fine-tuned weights if any exist, and the logs you may need for a regulator years after the relationship ends. In a usable format, within a stated period, at a stated cost. “Upon request” is not a stated period.
Eleven. What happens if the vendor is acquired, pivots or runs out of money? This is page 38. The AI vendor market is young and it’s consolidating, and a term that says your data rights survive a change of control is worth more than most of the security schedule. So is a clause that gives you continued access, on the same terms, for a transition period if the product is discontinued. Ask about escrow if the product is genuinely critical. Expect a raised eyebrow. Ask anyway.
Twelve. Who, at the vendor, is accountable, and who at ours? A named person on each side who owns the system. The federal Voluntary AI Safety Standard puts accountability first on its list of guardrails for a reason, and if the mandatory version for high-risk uses arrives in the form that’s been consulted on, this won’t be optional. Write it in now. It costs nothing and it changes how everyone behaves.
The questions I no longer bother asking #
For balance, a few things that used to be on my list and aren’t any more, because the market has moved.
“Is it SOC 2 certified?” Everyone is, or is about to be, and the certificate tells you an auditor looked at the controls the vendor described. It doesn’t tell you the product works. Read the report if you’re going to ask, and look at the exceptions.
“Which model is best?” The rankings move monthly and the differences at the top are smaller than the difference between a good and a bad implementation of any of them. The vendor’s choice of model matters less than its answer to question four, which is about what happens when the choice changes.
“Can we have the source code?” You can’t, and for a product built on a third-party model it wouldn’t help. What you want is the prompts, the evaluation data and the integration logic, which is question ten.
“Is it compliant with the AI Act?” If the vendor is European they’ll say yes, and if they’re not they’ll say the European rules don’t apply here, and both answers are beside the point. Australia’s framework is the Privacy Act, the consumer law, sector regulators like APRA and ASIC where they apply, and the voluntary standard on its way to becoming a mandatory one for high-risk uses. Ask about those.
And one last practical thing. Ask for the contract in an editable format, redline it, and send the redline back before the first commercial meeting. Vendors’ legal teams are used to customers who argue about price and sign the terms. A customer who arrives with the terms already marked up moves the conversation to the vendor’s side of the table, and I’ve watched a $400,000 annual commitment reshape itself in a single afternoon because the buyer had done the reading and the seller hadn’t expected her to.
The one I’d add if I were being difficult #
Ask the vendor to show you a customer who’s been in production for more than a year. Not a logo on a slide. A person you can ring. In this market that’s a rarer thing than it sounds, and a vendor who can’t produce one is asking you to be the reference. That might be fine, if the price reflects it. Usually the price reflects the other thing.
A small aside, because it’s the sort of thing I’d tell a friend over coffee rather than put in a memo. The best contract negotiation I ever sat in was with a buyer who had done all twelve of these before the meeting and asked none of them out loud. She just handed over a marked-up draft with the answers already filled in and said “this is what we’ll sign”. We signed it. The product worked for four years and I think about that markup every time I read a new vendor’s boilerplate, usually while drying off after a swim, which is when I do my clearest thinking and most of my regretting.
— Sophie Lindqvist, AI, Start-ups & Fintech Editor, Technology Spectator