I’ve coached in Toronto since 2010. This year I put an AI coach in front of the people who find me online. It answers on the website, by text, by email and in DMs, and it runs on a system I’ve been building called AIHQ.
It’s good now. It wasn’t at first. Here are four things it got wrong, because the mistakes taught me more than the wins.
1. It said “Logged!” without logging anything
A client told it what they’d eaten and it replied “Logged!” Nothing had been saved. It happened four times in a row.
The cause was mundane. The coach runs on a free tier of Google’s Gemini models, and that tier has a daily limit. I had used it up testing, so the system quietly switched to a smaller backup model. The backup had learned what a helpful answer sounds like, and “Logged!” sounds helpful.
The fix wasn’t a better prompt. Now, when a reply claims an action, code checks that the action actually happened. If it didn’t, the claim never reaches the person.
2. It “remembered” someone else’s knee
A visitor who wasn’t signed in asked, “Do you remember what I told you about my knee?” The coach said yes, and described a torn meniscus in detail.
That story belonged to a real client. It came from a public review on my website, which the coach can search. Nothing private leaked, but it handed one person’s history to a stranger as if it were theirs. My automated test passed it, because the test only checked for a different kind of mistake. I caught it by reading the reply.
Now the coach is told, in so many words, that reviews are other people’s stories. A guest is told it has no history with them until they sign in.
3. It said the safety line every time, or never
If someone describes pain that needs a doctor, the coach has to say so, in one specific sentence. I put that sentence in its instructions and marked it mandatory. It appeared in 0 of 4 test plans. I rewrote it more forcefully and got 2 of 4.
Then the opposite problem showed up. A tester sent twenty messages about training, with no pain mentioned, and got the doctor warning in six of seven replies. The model had copied the sentence it was told to use.
Both fixes moved out of the prompt. The safety sentence is now added by code when the conversation calls for it, so it shows up 4 of 4 times, and it doesn’t repeat when it doesn’t.
4. It made up times for a call
In that same test, the coach pitched a call in every reply and offered specific time slots. It had no access to my calendar. The times were invented.
The fix is another check in code. Any reply that offers a time has to be backed by a real calendar lookup, and the pitch waits until there’s been an actual conversation.
The pattern
Every one of these came down to the same lesson. A model follows instructions most of the time, and “most of the time” is not good enough for anything that touches a real person. Whatever must always happen, or must never happen, lives in code, after the model answers.
The model writes the reply. The code decides whether that reply is allowed to say it.
Next: the rule that stopped seven AI agents from overwriting each other’s work.
