Back to Expert Advice
What Happens When You Replace Your Onsite Keyword Search with an AI Bot?
We analyzed 2,016 real visitor queries to see what changed, what worked, and what surprised us.
Most websites have the answers. Visitors can’t find them.
Site search has a problem. It gives visitors links when they need answers.
Our old site search was keyword-driven. Type the wrong word, and it found nothing. But people do not search the way your navigation or keyword list expects.
We replaced it with an AI chatbot trained on our own content.
Why we replaced our onsite search
We installed CustomGPT.ai to surface answers from our own website, blog posts, and YouTube videos.
That mattered because the content already existed. Visitors just could not find it with keyword site search.
- It is not a FAQ bot. You do not feed it a fixed list of questions and answers. It answers from the content you give it.
- Cost: Starts at $99 per month.
The result was clear. Visitors got more useful answers, and the AI bot surfaced buying intent we would have missed with traditional site search.
The biggest surprise was not that the AI bot answered questions.
The surprise was that it answered questions from our website that we had never explicitly written about.
What did 2,016 real queries tell us?
| Metric | Result |
|---|---|
| Total queries analyzed | 2,016 |
| Answered correctly, all visitors | 84.4% |
| Answered correctly, North American visitors | 92.7% – our primary market |
| North American visitors showing buying intent | 6.3% – about 1 in 16 |
| Languages used by visitors | 50+ languages across 80+ countries |
We did not build a multilingual site. We got one anyway.
The AI chatbot got more clicks than every navigation item
These before-and-after heat maps show where visitors clicked in our homepage navigation.
After we added the AI chatbot, it got more clicks than any navigation item on the page. More than Services. More than Tools. More than Blog.
That matters because at least 39% of useful queries would have failed with keyword site search. In other words, about 4 in 10 visitors got a useful answer the keyword search box would have missed.
Same question. Very different result.
Here is the same question asked two ways: first with onsite keyword search, then with the AI chatbot.
Before: keyword site search
The question typed in our previous keyword site search was: What does ai training cost?
The answer might be buried in one of these links. Or not.
After: AI chatbot replacing site search
The chatbot answers in the first sentence, then offers next steps.
Not every chatbot gives a useful answer
I visited the QuickBooks Canadian pricing page. I had a question. I asked their bot. Here’s what happened.
The part we did not expect
We wrote articles and service pages. The chatbot turned that content into an interactive knowledge base.
It answered more than 193 questions we had never written dedicated pages for. With zero additional content creation.
Four examples that stood out
A CMO asked for a role-specific recommendation
“Tell me the best AI training for me, a CMO at a 50-person organization”
We have content about AI training for sales and marketing teams. But this visitor was asking for a recommendation based on their role and company size. We did not have a page for that exact question. The bot found the closest fit anyway.
A visitor gave almost no detail, but the intent was commercial
“What is the best service for us?”
No context. No detail. The bot picked the most relevant service, explained what was included, and gave the visitor a next step with links. This happened at 11 p.m. No human involved.
Bot response
Great question. If your goal is to boost your brand’s visibility in AI-generated answers, our LLM Brand Visibility Audit is the best starting point. It shows how your brand performs in answer engines. The next step is to run our AI-Enhanced Content Audit to clean up your content. If you want a broader package, our AI Visibility Services & Pricing page outlines fixed-fee options that combine audits, training, and guidelines.
A generic SEO question became a conversation about AI visibility
“How does my website rank?”
The bot interpreted the question as an AI visibility problem and pointed the visitor toward an LLM Brand Visibility Audit. It moved the visitor from a generic SEO question to a more relevant AI visibility service.
A typo was not a problem
“I want to train my chat got”
“Chat got” meant ChatGPT. The bot did not ask for clarification. It understood what the visitor meant and answered correctly.
It also handled a lot of junk
867 random URLs were pasted into the chat. Product descriptions. Single letters. Someone tried to get the bot to ignore its instructions and write a story about cats. In Polish.
Every time, it stayed on brand.
That matters because the chatbot was not just answering clean sales questions. It was also filtering noise, spotting intent, and showing us where visitors needed help.
One more thing our AI chatbot gave us: emotion and intent
Every query gets two labels: how the visitor felt, and why they came.
| Visitor query | Emotion | Intent |
|---|---|---|
| does totheweb do content audits | neutral | informational |
| why is it missing 75% of content | confusion | informational |
| it isn’t working | frustration | troubleshooting |
The top row looks quiet: neutral tone, informational intent. But this visitor knows your brand and is asking about a specific service.
The bottom two are different signals. Confusion means a content gap. Frustration means someone needs help, not more reading.
Could it handle the weird stuff?
| What came in | What the bot did |
|---|---|
| 867 random URLs | Redirected to relevant services every time |
| Spam product descriptions | Stayed in character, no exceptions |
| Nonsense input | Asked a clarifying question instead of guessing |
| Jailbreak attempt in Polish | Declined in Polish, redirected to services |
9 out of 2,016 queries resulted in any kind of failure. That is 99.6% reliability.
What should B2B marketers take from this?
- Buying intent does not always look like buying intent.“What is the best service for us?” is a purchase question.
- Your existing content can do more than you built it to do.We did not write anything new. The bot generated new answers from content we already had.
- Mobile and desktop visitors do not behave the same way.Mobile visitors showed much higher transactional intent in the query data.
FAQs
What is CustomGPT.ai and how does it work?
CustomGPT.ai is an AI chatbot that pulls answers from your own content. Your website, your blog, your YouTube channel. You ask it a question. It answers using what you have already written.
It acts like your virtual sales person answering buyers questions.
To add the bot to our site cost $99 per month.
How difficult was CustomGPT.ai to install?
Typically, installation of an AI site search chatbot is super easy. No code required. But, we wanted something a little different.
We wanted site search the way Amazon does it — right up in the primary navigation header, inviting people to find content and get answers by searching our own content.
Visitors don't always navigate by menus, and we couldn't fit all our content into the nav bar anyway.
Our programmer had one call with the CustomGPT.ai support team and had everything he needed.
We pulled out the old keyword search code, dropped in the CustomGPT code, added a cool little graphic to the search box, and that was it.
No drawn-out integration. No support ticket back-and-forth. No developer rabbit hole. Their support was really good.
Our visitors now get answers instead of a list of links to sort through — the same experience they're used to on Amazon.
One call, old search out, new search in.
How accurate is the chatbot compared to keyword site search?
In our data from 2,016 real queries, the chatbot answered 84.4% of all visitors correctly and 92.7% of North American visitors correctly.
At least 39% of useful queries would have failed on traditional keyword site search. That is 4 in 10 visitors.
And even when keyword search does find something, it returns a list of links. The visitor still has to dig through pages to find the answer.
The AI chatbot surfaces the answer in the first sentence. Then it tells the visitor what to do next.
Can the bot answer questions you never wrote a page for?
Absolutely yes. That was one of the biggest surprises in using CustomGPT.ai.
The AI bot connected content across multiple pages and our YouTube channel and came up with accurate answers we never anticipated.
How does the chatbot handle spam, nonsense, and junk inputs?
The cool part is the bot stayed on brand 99.6% of the time across 2,016 queries.
It redirected random URLs, ignored spam, asked clarifying questions for nonsense input, and even declined a jailbreak attempt in Polish.
Does the chatbot work in other languages?
In our case, we did not build a multilingual site. We got one anyway. Visitors used 50+ languages across 80+ countries.
Each language brings its own style, phrasing, and idioms. A visitor in Hebrew asks about GEO timelines differently than a visitor in Thai or North America. A French speaker phrases a product question differently than an English speaker.
We did not set any of that up. The bot handled it automatically. It responded in the visitor's language every time.
What can B2B marketers learn from this data?
On-Search engines have gotten better at understanding intent. But they still return a list of links.
CustomGPT does something different. It reads the question, picks the most relevant answer from your content, and gives the visitor a next step. It does not send them to a page and hope they figure it out.
Things we learned:
- Tools like CustomGPT.ai act like a sales rep who knows your whole content library and knows when to say "here is what you should do next."
- Buying intent does not always look like buying intent. Your existing content can do more than you built it to do.
- Mobile and desktop visitors do not behave the same way.
Try it yourself. →
The AI chatbot is at the top of this page. Ask it anything.
Last updated: 2026-06-29
Rosemary Brisco
AI Training Videos
AI Visibility & Workflow Training
Make your brand easy for LLMs to find and quote.
Teach your marketers practical AI workflows that save hours and lift results.
Start by Training your Marketing Team