Buyer guide · Vendor evaluation
How to choose an AI vendor
Every AI provider demos well. The demo runs on their data, on their happy path, at their volume. What separates a partner from a vendor is what they say when you ask about ownership, baselines, and what happens the day the contract ends. Here are the eight questions worth asking, the red flags worth walking away from, and how the buying options actually compare.
The evaluation
Eight questions to ask every AI provider
Ask all eight of every provider on your list, and ask for the answers in writing. The pattern in what they avoid tells you more than the pitch deck.
01Who owns the system when the contract ends?
Ask whether you receive the source code, the prompts, the fine tuned weights, and the data. Some vendors deliver a system you own outright. Others deliver access that stops the day you stop paying. Both are legitimate models, but you should know which one you are buying before you sign, not after.
02Did they measure your baseline before promising a number?
A vendor who quotes a return on investment before looking at your current resolution time, cost per ticket, or conversion rate is quoting someone else's result. There is no ROI without a measured before. A serious provider insists on establishing the baseline inside your operation first, then agrees to targets per phase.
03What happens to your data, and where does it live?
Ask which model provider processes your data, whether it is used for training, in which jurisdiction it is stored, and how deletion works. In Mexico this ties directly to LFPDPPP obligations, and in regulated sectors it decides whether the project is viable at all. Get the answer in writing.
04Can you change the underlying model without a rebuild?
Model prices and capabilities move every few months. If the system is welded to one provider, every future price increase is your problem. Ask how the vendor abstracts the model layer, and what a swap would actually cost in time.
05Will they run a proof of concept on your data?
A demo runs on the vendor's happy path. A proof of concept runs on your messy records, your edge cases, and your volume. A provider who refuses to test against your real data before a full commitment is asking you to buy on faith.
06Who operates the system after launch?
AI systems drift. Models change, your data changes, and quality decays without tuning. Ask what the operation plan is, who is on call, and how changes get made. A build only proposal with no operation plan is half a plan.
07Is every decision traceable?
When the system denies a claim, quotes a price, or answers a customer, you need to be able to reconstruct why. Ask to see the logging and evaluation tooling. Auditable traceability is what turns an AI pilot into something legal and compliance will let you run in production.
08Do they coordinate specialists, or claim to be the only vendor you need?
Real deployments touch your CRM, your telephony, your ERP, and your data warehouse. A provider who claims to replace all of that alone is selling a story. Ask how they work alongside your existing systems and the teams that already run them.
Red flags
Six answers that should end the conversation
None of these mean the provider is dishonest. They mean the provider has not done this before, or is selling you a product while describing a project. Either way the risk lands on your budget.
- A return on investment figure quoted before anyone looked at your data.
- Refusal to run a proof of concept against your real records.
- "Our AI does everything, you just plug it in." Clean data, tuning, and integration are never free.
- No written answer on data residency, training use, or deletion.
- A fixed price with no defined scope document behind it.
- No named success metric and threshold that counts as done.
The options
Four ways to buy AI, and what each one costs you
There is no universally right answer here. There is only the one that matches how standard your problem is and how long you intend to own the result.
SaaS platform
Best for
A standard problem, solved the standard way, live in days.
Watch for
You rent it. Per seat or per conversation fees scale with your success, the roadmap is not yours, and deep integration with legacy systems is usually where it stops.
Freelancers
Best for
A contained, well specified piece of work with a clear finish line.
Watch for
Continuity and operation. When the contractor moves on, the knowledge leaves with them, and AI systems need someone to keep tuning them.
AI and software factory
Best for
A workflow specific to your business, integrated into systems you already run, with the code and data ending up yours.
Watch for
It takes scoping before it takes building. Insist on a written statement of work with success metrics before anyone writes a line of code.
In house team
Best for
AI that is core to your product and that you intend to compound on for years.
Watch for
Hiring time and the market rate for senior AI engineers. Most teams take a year to reach the point a factory reaches in a quarter.
How we work
Hold us to the same eight questions
Kemeny Studio is an AI and software factory: we build bespoke systems, integrated into what you already run, and the code and data end up yours. We do not quote a return before measuring your baseline. That measurement is what a paid audit is for, and it produces a scoped statement of work with a fixed price before anyone writes code. Start with an audit, see what belongs in a statement of work, or check what an AI build costs in the market.
FAQ
Common questions
How do I choose an AI vendor for my company?
Start from your own constraints rather than the vendor's demo. Define the workflow you want changed and the metric that would prove it worked. Then evaluate every provider against the same questions: who owns the resulting system, whether they measured your baseline before promising a return, where your data lives and whether it trains their models, whether you can change the underlying model later, and who operates the system after launch. Ask each one to run a proof of concept on your real data. The provider who insists on measuring before promising is usually the one worth hiring.
What questions should I ask an AI provider before signing?
Eight carry most of the weight. Do I own the code, prompts, and data when the contract ends? Did you measure my baseline before quoting a result? Where is my data stored, and is it used for training? Can I swap the underlying model without a rebuild? Will you run a proof of concept on my data? Who operates and tunes the system after launch? Is every decision auditable and logged? And do you coordinate with my existing systems and vendors, or claim to replace them all? Any provider who resists answering these in writing has told you something useful.
Should I buy an AI SaaS platform or build a custom solution?
It depends on whether your problem is standard. If the workflow looks like everyone else's, a SaaS platform gets you live in days and that speed is worth the subscription. If the workflow is specific to how your business actually operates, or it has to reach into a legacy ERP or a regulated data store, a custom build is what survives contact with reality. The deciding question is usually ownership: a subscription stops the day you stop paying, while a custom system remains an asset on your side of the table.
What are the red flags when evaluating an AI provider?
The biggest is a return on investment number quoted before anyone looked at your data, because without a measured baseline that figure describes someone else's company. Close behind: refusing a proof of concept on your real records, claiming the AI works the moment you plug it in, giving no written answer on data residency and deletion, quoting a fixed price with no scope document behind it, and never naming the metric that counts as done.
Next step
Stop hiring. Deploy an AI agent.
Book your AI audit. In 10 days you'll know which workflows to hand off to an AI agent, the expected savings, and a fixed-price agent build scope. We build it. Then we run it.
20 minutes. No pitch deck. No commitment.