"AI-powered" is on almost every agency's homepage right now, and it means almost nothing on its own. Some of those agencies have shipped real, working systems for real clients. Some have a template, a chatbot wrapper, and a very good pitch deck. From the outside, in a first call, they can sound identical. Here's how to actually tell the difference before you commit budget and time to one.
Ask to See Something Real
Not a case study slide. Not a mockup. Something you can actually use right now. If an agency's proof of work is entirely screenshots and testimonial quotes, that's worth noticing. A voice AI agency should be able to hand you a phone number to call. An automation agency should be able to show a live workflow running, not a diagram of one. If the best they can offer is "we can't show you due to client confidentiality," that's sometimes true and sometimes a way to avoid the question, so listen for whether they offer any live alternative at all.
Ask How They'll Keep You Looped In During the Build
This matters more than it sounds like it should. A lot of build-and-disappear engagements go sideways not because the final product was wrong, but because the client had no visibility into the six weeks it took to get there, and by the time they saw it, a wrong assumption was already baked in. Ask specifically: will you see progress during the build, or only at the reveal? Is there a shared plan before work starts, and does it explain the reasoning, not just the deliverable list?
Ask What Happens After Launch
Automations and AI systems aren't static once they're live; they need monitoring, and they need to adapt when your process changes. An agency that treats launch as the finish line is setting you up to be back at square one in six months. Ask what support or retainer options exist, and what specifically they cover.
Watch the Pricing Conversation, Both Directions
- Too vague: an agency that won't discuss pricing structure at all, even in general terms, after a real scoping conversation.
- Too fast: an agency that gives you a firm number before asking real questions about your process. Accurate scoping requires understanding the exceptions in your workflow, not just the headline task, so a fast firm quote is often a sign the number was decided before the scoping happened, not after.
Custom AI and automation work varies enormously by scope. A number that means something usually only exists after a real discovery conversation, not before one.
Red Flags Checklist
- Portfolio is entirely mockups, no live product you can touch or call
- Firm price quoted before any real discovery conversation
- No clear answer for what happens after launch
- Vague on how (or whether) you'll see progress during the build
- Every case study sounds the same regardless of industry