If you visited our support chat on a weekend over the last few months, you may have noticed something different. There’s a AI support bot named Fin. Fin is part of Intercom, which is a product we have used to support our clients for 10+ years. But until recently, it’s only been us humans in the chat. 😃
We turned Fin on as an experiment.
AI is a polarizing topic, and no surprise there’s been a lot of feedback from our clients about Fin, both positive and negative. It’s been interesting to see how passionate some of our clients are around the use of AI, and how it can so quickly prompt a very negative reaction. This post is an explanation of why we are running this experiment.
The primary reason we are experimenting with Fin (i.e. AI support) on the weekends is simple – it’s because we care about the humans on both sides of the support ticket.
On one side we have our clients. If there’s one thing we’ve learned over the last 12 years, running a fitness studio or class based business is not at 9-5 gig. 😅 Our clients often need help on the weekends, and we get how frustrating it can be to have a question and have to wait until Monday.
On the other side we have our support team. We’ve worked hard to keep the price point of Punchpass affordable AND we still manage to provide world class service. Again and again that’s what our clients rave about. However Punchpass is a small team – too small to have enough people to really run rotating weekend support shifts. And we’re a global service with clients in over 50 countries around the world.
Work-life balance is an important part of our work culture – and that’s where Fin comes in. Fin lets us try to answer a solid percentage of the questions that clients have over the weekend, while still giving our team the break they deserve. To be crystal clear – this has nothing to do with cost, or eventually moving to an all AI support model. This is simply a tool to provide better 24/7 service while giving our team the work-life balance we believe is right.
So – how has it been going? Honestly, better than we expected!
Over the last 12 weeks (March 19 – June 12), Fin has been involved in slightly less than 10% of our support conversations. If you just look at chat (Fin does not currently interact with email support) it’s actually almost 18%. That fact alone highlights the weekend volume that needs some level of attention. The Fin ‘resolution rate’ is 69.2% – those are chats where either the customer gave positive feedback or did not ask to speak with a teammate (i.e. a human!) Frankly that seems pretty good!
What we have been pleased with:
- The big one, is that Fin is often right!! We have been surprised by the ability of Fin to answer relatively complicated answers. Is it wrong sometimes? Of course (but so are we!), but on the whole we have been impressed. Great support documentation and years of support chat history give the LLM a lot of data to help it find the correct answer.
- The backlog of questions that need to be answered come Monday morning is a lot less than it used to be – a LOT less. Many clients are getting the answer they need and moving on with their weekends.
What hasn’t worked so well or needs improvement?
- We need to make it clearer this is an experiment for off-hours. This post is a step in that direction.
- We need to provide better guidance around asking the support question – Fin definitely does better if more information is provided.
- We need to make it clearer that if you don’t want to interact with the AI bot you don’t have to – just ask for a teammate and your request gets queued just like it used to.
So far the experiment has been a success – we’re providing much better support over the weekends, without compromising the work/life balance we value.
