GoodBuy Gear is a Shopify store that uses Intercom for support, and they have Intercom's Fin AI agent enabled in their support chat. The AI was able to answer basic FAQ-type questions and redirect out to external pages, but didn't have direct access to resolve a number of common inquiries.
Basic Q/A
The agent did a decent job with basic Q/A, providing information and direct links to content from the site's FAQ. I was also able to ask about a discount and get information about signing up for their newsletter to get access to a discount code (although it couldn't sign me up directly).
Product recommendations
After browsing the site, I asked the AI to find a specific product I knew existed. It seems that the AI doesn't actually have access to their product catalog, so it wasn't able to help beyond giving me instructions for how to search on the website.
Direct issue resolution
I tried to have the AI track an order for me, which it couldn't do and simply directed me to an external tracking page.
When I asked about returns; another common inquiry; the agent again sent me to an external page.
It's disappointing that the AI is not set up with basic tools that would likely help it solve a good chunk of support issues.
Live agent handoff
Getting connected to an agent in the end was a somewhat confusing experience. It's nice that I'm able to stay in the UI, but it's not actually clear whether I'm getting connected to a live agent, or someone will follow up with me. This is one of the perils of live chat, and if you're not sure you can staff it, it's probably best to set clearer expectations for users. I got a message that I would get a response in 2 days, but then someone did respond fairly quickly.
Overall, the AI met the baseline expectation of providing accurate information from their help desk, but it was disappointing that it wasn't able to help with end to end resolution of even basic inquiries like order status or returns.
Want to set up an AI that can resolve these types of issues end to end? Get in touch.