A new hire doesn't send a price quote or issue a refund on day one without somebody looking at it first. Nobody calls that an AI rule. It's just how you run a business.
Building with AI is hard, and it's hard for the same reason.
Two weeks ago I wrote about the path from chatting with AI to building with AI. I've been building a few things myself, and the building itself went fine.
Then the hard part arrived: understanding what I'd built well enough to actually use it, and to know it was working. AI doesn't take the person out of the work. You still have to check it, and you have to know where to check.
Once I understood what I'd built, the answer was something I already knew. Keep a human in the loop, meaning a person looks at the work before it goes out to clients, customers, or your own team. I didn't have defined points where I was doing that.
Now I do. If an email is going out, it stays in draft until I've read it. If my AI logs a task from a call, I check that the details are accurate.
That's the same rule as the new hire, and it matters more with AI, not less. The better these tools get, the more finished everything looks, and the easier it is to stop checking.
So before AI touches anything in your business, decide where a person has to look.
A short list is enough:
- Anything a customer will see.
- Anything with a price on it.
- Anything to do with legal.
What's the first thing on your list?
A law firm, Kolmogorov Law, surveyed 500 employed US adults on September 1. 65% said they don't always check an AI answer before acting on it, and 42% have gone ahead with an answer they suspected was wrong. 30% say a wrong AI answer caused a problem at work. Only 22% work somewhere with a written policy on checking AI. Among people using AI for legal, financial or compliance questions, 43% have had problems, and 64% still don't always check.
Our Take: Your team uses AI whether or not you've said anything about it. Without a written rule, each person decides alone when to check. Write the rule down. One line per checkpoint puts you in the 22% with a policy.
Bluevine, a small-business bank, surveyed 942 owners of businesses with 2 to 249 employees. 74% are already using or testing AI tools, and 48% say it saves them at least four hours a week. But 78% don't fully trust AI to handle even low-level business tasks without a person supervising, and 31% distrust its accuracy.
Our Take: Owners already know AI needs a person watching it. The survey above shows their teams aren't always doing the watching. A short written list of where a person has to look keeps the check from depending on who happens to be using the tool that day.
Source: Bluevine
Researchers in Italy and France ran five experiments with 3,132 people, asking hard questions about small movie details where the AI's advice was usually wrong. With AI available, people nearly stopped saying "I don't know" (44% down to 3% in one experiment), even when the AI answer appeared without being asked for. They got about a third as many answers right (27.5% vs 9.2%) and rated their confidence 76 out of 100 instead of 30. Paying people for right answers helped some: they leaned on the AI less and got more right, but stayed well short of the group with no AI.
Our Take: A wrong AI answer reads as clean and sure as a right one, and you pick up that confidence when you read it. The one thing that helped in the study was making the answer matter. Decide ahead of time which answers carry a real cost if they're wrong, and put a second look on those.
Founder Reports surveyed 2,078 US workers in April. 45% have fixed or redone a coworker's AI-heavy work. That rises to 57% of managers, 63% of executives and VPs, and 73% at companies that require AI use. 77% say they review work more carefully when they know AI was involved.
Our Take: Your managers have made themselves the checkpoint, one fix at a time, without anyone asking them to. Name the spots where their review is required, and they can stop re-checking everything else.
Source: Founder Reports
Storable surveyed 1,000 employed US adults in July. 48% said one bad AI experience would send them to a competitor. 47% said they'd mainly blame the company that used the AI, not the AI itself. When asked what matters most from a company using AI, 54% picked "access to a human who can step in when needed."
Our Take: Customers want a person behind the AI, and when it goes wrong they blame your business. Put anything a customer sees at the top of your checkpoint list.
Source: Storable
Google began rolling out Gemini features in Workspace on September 2 that draft emails and book meetings from whatever app you're in. Before Gemini sends an email or books a meeting, it shows a preview card so you can "review, edit, and confirm before execution." The feature is in English on Business Standard and Plus, Enterprise Standard and Plus, and Google AI Pro and Ultra plans.
Our Take: We’re enjoying this, mostly for personal Gmail, and it’s been super helpful. The quality is an A-. Google built the checkpoint into the tool. Treat it like a new hire's first quote sitting on your desk, and read it before you hit confirm.
Source: Google
That’s it for today!
— Brian, Murphy, John, Beth, and Jake. Some of the humans behind AI Front Desk.