Support Stack Episode 09 – From 55% to 75% Fin Resolution in 2 Months with Tobi Davis
How do you lift Fin’s resolution rate by 20 percentage points in just two months?
In this episode, Tobi Davis, Head of Customer Experience at Pupil Progress, shares how his team reworked their Intercom setup to achieve a steady 75–80% Fin resolution rate — without adding headcount.
Tobi walks through the three levers that made the difference:
AI-first, human anytime: letting Fin greet everyone while making it easy to reach a teammate when needed.
Help content for humans and AI: rewriting articles into short, scannable sections with bullet points and step-by-steps that Fin can understand.
Role-based answers: using audiences and attributes so responses change depending on who’s asking — a maths teacher and a languages teacher never get the same answer.
He also explains how this content-first approach improved customer satisfaction and freed up his team to focus on more complex, high-impact work.
It’s a clear, practical look at what happens when you design your help centre for two audiences: people and AI.
👉 Subscribe to my daily email for more CX + Intercom insights
Episode Transcript
Conor Pendergrast (00:00)
Hi, I'm Conor Pendergrast and welcome to episode nine of Support Stack.
Tobi Davis, welcome back to Support Stack. I'm excited to have you here in your role as head of customer experience at Pupil Progress. How are you today?
Tobi Davis (00:13)
Hello.
Yes, all good. Excited to really dig into Fin
Conor Pendergrast (00:27)
Absolutely.
the genesis of this, this particular conversation, Tobi is we'd, we'd a little look behind the curtain. I don't actually just jump into these, you know, I actually speak and do a little bit of preparation. So Tobi and I spoke a few weeks ago and he mentioned a data point that absolutely blew my mind, which was that their fin resolution rate was stuck at about 55%. And through a lot of we'll say it's a lot of hard graft and a lot of work, but within two
Tobi Davis (00:36)
Ha!
Conor Pendergrast (00:56)
months, two months, it had a 20 percentage point lift. And so the average resolution rate went from 55 % to 75 % in two months. Tobi, Tobi, say the same.
Tobi Davis (01:10)
Yes.
Conor Pendergrast (01:10)
Hang on, what's that on your screen?
Tobi Davis (01:13)
Yes, mean, ⁓ very, very proud of this. And before we get into anything, I will give a quick shout out to Ellie and Larne on my team. Absolutely incredible work. As much as I would love to take credit for all myself, ⁓ you know, I definitely cannot. Because, you know, and it's interesting, it changes the way that you kind of think about things and changes the way that you kind of think about how you do support because we started using Fin June 2023.
June 2024, sorry, I mean, forgot what year it was for a second. And at first it was a bit of a disaster and then we kind of fixed it. And then we were sitting at, as you say, yeah, somewhere 40 to 50 % resolution rate and sort of thinking that was what it was gonna be, right? And it's like, okay.
Conor Pendergrast (01:47)
Ha ha.
That was it. Because
if you think about it, what you're saying is we were sitting at a point where Fin was handling half of the Incoming Conversation volume. We would have done, imagine thinking back five years ago, you would have bitten off so many hands just to get that.
Tobi Davis (02:12)
Yeah.
That's exactly right. And kind of you and I were just speaking about the resolution bot, is what we had come from beforehand, you know, and even,
Conor Pendergrast (02:29)
Yes.
Tobi Davis (02:32)
forget 50 sort of 15 resolution from that would have been, ⁓ I don't think we were getting that. I don't think we were. But if we did, I'd have been like, well, so, you know, moving from the resolution bot to to Fin and, you know, starting that off as, you know, a challenge, as I'm sure everybody who who adopted Fin has and, you know, working on that, getting it to the point where it's about 40, 50 percent resolution. I was like, brilliant. You know, this is this is this is
Conor Pendergrast (02:35)
15, what? 10%, yep.
Tobi Davis (03:01)
Fabulous, this is exactly what we want. And then, sort of from January onwards, you know, we really amped up the work in terms of...
in terms of two things, audiences and our content, just the way that our help center content was structured. And as a result of doing that, as you can see, sitting relatively consistent around 50 % here, and you can see into the summer, we were at a much more consistent 70 % to 80%. And that seems to be roughly from the new academic
year onwards, you can see that's now sort of relatively stable around 75 to 80.
Conor Pendergrast (03:54)
Yeah.
And this is looking on a week by week basis, just for anyone who's looking at the chart, which is why there's a little bit of fluctuations, but you, but, that's fantastic. So I think if we're gonna, so obviously the point of this whole series is to give actionable takeaways to people so that they can know,
Tobi Davis (04:10)
Yes.
Conor Pendergrast (04:11)
well, if I want to improve my resolution rate in the same way, every business is unique and every, every helps customer support experiences unique. But what if we're learning from pupil progress and how you.
and Ellie and Larne improved this experience and improved Fin's ability to manage these interactions. There were some things in particular that were that have massive returns on investment for you. Is that right?
Tobi Davis (04:38)
100%, yeah. I think one of the main things for us, and it was sort of something that we resisted at first was...
sort of going all in to AI first or going all in to, Fin is our first point of call. Because there's something kind of we had identified even before the jump here, which is a bit of like an 80-20 split, right? So we were finding that the bulk of our customers, around 80%, just kind of wanted an answer as quickly as possible. You know, they weren't really fussed about having a conversation with us or how
having
that kind of personal connection. They just couldn't do something that they needed to do. They wanted to just be able to do it and move on and, you know, get on and log off and go do something else, I would imagine.
The flip side to that is you have the 20 % who are kind of the opposite. They're prepared to wait a bit more time. They're prepared to, you know, to sort of potentially go through a bit of a longer process to have a conversation with a person, be able to ask their questions in a way that, you know, they know a person will be able to understand and do it that way. And obviously, as you say, I'm sure that will be, you know,
numbers there will be different for every business. I think as we spoke about in a previous episode, know, lot of our users are not super.
Technology literate or or you know, maybe a bit more resistant to Adopting new technologies like something like finn than others And and what we did at first is because I think maybe we weren't a hundred percent confident in finn or we you know didn't sort of have that all-in mindset is We sort of flip-flopped a little bit at first. We you know still try to follow the same structure that we have before so you search for
an article, you find one great, you don't, you go through to Fin, it answers it great, it doesn't, you go through to the team. What we did is just Fin from the beginning in the prompt message, if you want to speak to a person, just say speak to a person and it will go straight through. And the really, really positive thing about that is it means that...
Conor Pendergrast (07:03)
Brilliant.
Tobi Davis (07:10)
You know, your users are self-categorizing, right? Like there would have been a certain chunk of people around this time that...
probably were never fully gonna be happy with a FIN answer or they wanted to speak to a person to have that, you know, a specific person look at their specific data and tell them what's going on. So we were finding that even if FIN gave the right answer to that person, they were still coming through to one of the team anyway, because they wanted that sort of extra level of confidence. So if you're just putting it in the initial prompt
message on Fin, you know, if someone, if someone wants to wait to speak to one of our team, brilliant. You know, that's what we're, that's what we're here for. but it means that it's going to be a better experience for them because they're not going to sort of get into a conversation with an AI, not fully be happy with the outcome and then go through to a member of our team. Whereas if some, someone's in the other camp and they just want an answer, they just want their problem solved, you know, great. They can get it straight away.
Conor Pendergrast (08:15)
they can get it straight away.
Tobi Davis (08:17)
And so, you know, as part of this, we are taking out a number of conversations that I suppose would be a much higher likelihood of being unsuccessful and just leaving Fin to focus on the people who, you know, who really want that kind of support.
Conor Pendergrast (08:36)
Yeah.
Yeah. So the first step, the first thing you changed was then putting, you updated your workflows and had, had let Fin answer as the first step, but you also had a prompt there and guidance, I presume that says like, Hey, okay. Well, you can go through to the customer. What was the, what was the second thing you did then? What was the, what was the next change you made?
Tobi Davis (08:45)
Mm-hmm.
Yeah, that's absolutely right. mean and the next change and I think something that was super super significant for us was actually taking the time to restructure our entire knowledge base basically, you know, I'll give huge amounts of credits my colleague Lam for this because you know before Before she kind of took it and did her thing with it, you know, all of the articles were Kind of written in the way that I talk, right?
So
like long paragraphs, way more text there than needed to be. Yes, yeah, exactly. And so, you know, that's all well and good if someone is reading your knowledge article from top to bottom, like it's a letter.
Conor Pendergrast (09:31)
An editor's nightmare.
Tobi Davis (09:44)
which realistically people aren't doing and also Fin really struggles to do. So what she did was, you know, just kind of clean everything up basically, you know, we have the information there, but it was in such an inaccessible format that, you know, Fin wasn't able to, ⁓
pull the information that it needed. And so what it would either do is it would not be able to answer the question or it would try and answer the question with what it could infer, but what it could infer wouldn't really make sense or wouldn't, you know, it would be sort of getting muddled up with three different features at a time and talking about buttons that don't exist. This is not even to say that some of our articles were just a video, you know, without any, ⁓
Conor Pendergrast (10:16)
Yeah.
Tobi Davis (10:33)
thing else on there. Yeah, exactly. There were a few. It's like, how to, how to, ⁓ you know, add this thing. And it would just say, watch the video below. And the video would be, you know, great in terms of telling them how to do it. you know, exactly. ⁓
Conor Pendergrast (10:33)
to solve X problem, watch this video. And Fin can do nothing with that.
Mm.
Super. Yep, if someone wants to watch a video...
Tobi Davis (10:58)
But yeah, so what she did was, you know, really sort of restructure that, as I say. So things where we had videos, you making really clear like bullet points about the specific actions in the video, step by steps of sort of guidance that way. What we would do as well is we would have, you know, maybe multiple articles that all kind of talk about the same thing or that we could amalgamate differently or that we could kind of split up
the way that we needed to split up, we would ⁓ utilize audiences. So, you know, for us and the amount of different types of users that we have on our platform, you know, we have teachers, we have subject leaders, head teachers, data managers, you know, all of these people within the school that are using Pupil Progress differently.
Conor Pendergrast (11:30)
Mm-hmm.
Tobi Davis (11:54)
One thing is, if say I teach ⁓ child development and I need to add an assessment, practical assessment for child development and Conor, maybe you teach maths or science or something that has a very different way of assessing, the information that you're going to need to get from Fin is quite different and the way that it works is quite different. Even though you're asking the same, we're both asking the same question, we wouldn't both want to get the same answer because there's
Conor Pendergrast (12:15)
Very different, yeah.
Tobi Davis (12:24)
about us and the job that we do that means that that's slightly different. So, really making that content in the articles easy to read both for Fin and for humans, right? That's the other side to it is beforehand, we were writing something that a person could read. Now everything that we produce is for two audiences specifically for our users who come across those articles and then for
Conor Pendergrast (12:40)
Yeah. Yeah.
Tobi Davis (12:54)
for Fin as well to pull the information from. But then on top of that, making sure that Fin knows
to say what to and okay, if there's something about, you know, I'll give an example here if I just switch over the tab to our audiences. So one thing for us is repeatable subjects and ⁓ non-repeatable subjects, right? Which is language that we use, but basically it's
Conor Pendergrast (13:05)
Mm-hmm.
Perfect. Yeah.
Gusses.
Tobi Davis (13:23)
way that the exam works and the exams work differently for something like history or English than they do for these subjects. know, English will have the same structure every single year and
Conor Pendergrast (13:30)
Mm-hmm.
Tobi Davis (13:39)
Maths will have completely different structure. So the way that you add the assessments is different and so if someone's coming through and they're saying okay, I want to add a pass paper for You know, whatever subject it is depending on what subject they teach the answer that That Fin will give them the information that it will pull from in terms of what it's pulling from our help center Will be different
Conor Pendergrast (14:07)
Super. Okay. So I think there's three takeaways here for people. The first and most important was just let people interact with Fin. It sounds obvious, but let them interact with Fin and also make it very clear that you can talk to a human at any point as well. And so that's the first thing. The second thing is make sure that your knowledge center, your public help articles are written so that
Tobi Davis (14:11)
Mm.
Yes.
Conor Pendergrast (14:36)
people and Fin AI Agent can consume them, can interpret them correctly and kind of understand them and then share that information with whoever needs it. And that includes, if you want to have a video, have a video, but people also don't want to watch a full video, like a four minute video, just to learn something that could be in a bullet pointed list or that you want to come back to and scan again. So have those separate from the video.
Tobi Davis (14:53)
Yes.
Conor Pendergrast (15:01)
and excuse me, on the same page and alongside the video, because then someone who wants a little bit more detail, like Fin includes those footnotes that you can click into and the references, which is a great way of gaining more confidence in what Fin is saying, can go through and then see the video and be like, actually, I do need a visual guide to how to do this. So that's the second point. And then the third one is your audiences that you're showing here. But where are you applying these audiences? Are you applying these to the help center or in other places?
Tobi Davis (15:24)
Mm-hmm.
Yes, so
predominantly the audiences are applied to the Help Center.
So particular articles, you we might have two very similar versions of the same article, but you can only see one if you teach certain subjects and you can only see the other if you teach other subjects. What we also do sort of on top of audiences. And again, this is something that we were speaking about beforehand in terms of something like custom answers or what used to be the resolution bar is slightly different.
Conor Pendergrast (15:37)
Yeah.
Yeah.
Tobi Davis (16:07)
you know, is maybe something like in here where we only want this answer to show for subject leaders rather than teachers or somebody else. ⁓ So we don't always necessarily add the audience.
Conor Pendergrast (16:15)
Of course. Yeah.
Tobi Davis (16:24)
in the same way. Sometimes we'll do it sort of based on particular criteria. This can sometimes be a nicer way of doing it as well, I think, because of the level of specificity to our users and how they use Pupil Progress. know, sometimes we might want to drill down sort of even further for a specific bit of guidance. Say, for example, something that has come up recently is languages. One of the exam boards, they've changed
all the specifications for the languages. So you're sort of getting uber-specific where people are, say, trying to, they're still using the last specification, but they want to add to the assessments from the new one.
Conor Pendergrast (16:58)
Right.
Tobi Davis (17:10)
And then you could add a new audience to do that, but you could also, it's just this one specific thing, this one specific message that we want to make sure they get. And so sometimes we'll be more tight on ⁓ setting the audience this way instead. So as I say, predominantly that's on our articles, but also our custom answers as well.
Conor Pendergrast (17:29)
course.
Hmm.
That's, that's very interesting. And so, so customer answers, you, you're using these mostly, so they're, I don't, are they deprecated by intercom yet or they're, not deprecated, but they're, I think they're definitely not gonna, it doesn't seem like they're going to do anything. So probably something like content snippets would be where someone would start doing this sort of thing, or maybe internal articles, or even just like really, really, really narrow public help articles maybe as well. What?
Tobi Davis (17:46)
No, but it feels like...
Yeah.
Yeah, that's exactly
it.
Conor Pendergrast (18:08)
What's really interesting, Tobi, I think, like from what we talked about the three things and the 20 percentage point lift that you've got, I talk about
I talk about content data and actions being the three levers that you can pull for, for Fin, where your focus has been on the content, both in terms of customer answers and the public health articles, as well as data being the attributes, the company attributes and the user attributes that fitting into your audiences, but nothing on the action sides in terms of either data connectors or the, what we always call tasks and is now called procedures. ⁓ and you've done, had such great impact with those.
Tobi Davis (18:28)
Yes!
Mm-hmm. Yeah, and you know, it's one of those where it's like
The other side to this, right, is that everything is constantly evolving. Who knows? I didn't really think it would get much higher than 40, 50%, but who knows? I'm sure there's more things that we can do. mean, obviously, we've started recently using things like the Topic Explorer and these other tools that we CX score and things that we didn't have before. But I think our...
Conor Pendergrast (18:58)
Yeah.
super.
Tobi Davis (19:16)
our sort of philosophy with it has always been to give our users as much agency as possible. And as I say, I really believe that a ⁓ positive or a great customer experience is a completely different thing for every single different person and even the same person at different times they look for it. And so what we try to do is keep that central to everything in terms of
Conor Pendergrast (19:36)
Hmm.
Tobi Davis (19:46)
You know for me at this moment in time a good customer experience is I need to really sit down and chat with someone You know get on a call maybe and and and and talk out this issue Sometimes it is I just need to do this thing as quickly as possible and and and move on and so I think by focusing on content and really making sure that Okay, you know we're not what we're not trying to do is is say that Fin with
you know, we wanted to take all of our support or we think everybody should get support through Fin because at the end of the day, it's not always right. It's not right for every person. It's not right for every support interaction. But we focus on the people who do want to get support that way and maximize that. And then also the people who don't want to get support that way, make it easy for them not to. And then you kind of have happy people in both camps because, you know, the people who are getting their
Conor Pendergrast (20:40)
Yeah.
Tobi Davis (20:45)
resolutions from Fin, they're getting super tailored information because we're putting so much focus into content and audiences. And then as I said at the beginning, you're not getting the negative resolutions from the people that kind of don't really want to be chatting to Fin in the first place. And they're not sort of having to spend their time doing something that they don't want to do. So yeah, that's what's worked for
Conor Pendergrast (21:06)
Yeah.
Brilliant.
Tobi Davis (21:15)
and as I say, who knows, maybe we'll speak again at another point and there'll be something else going on. Yeah. Yes, yeah, exactly. We're at 150%, yeah.
Conor Pendergrast (21:19)
in six months when you're at 95 % resolution rate and have done it through something totally different.
Tobi, this has been really helpful. hope people take away a lot from this and find ways of applying it to their own companies and their own support organizations and improving their customer support experience. But in the meantime, if anyone would like to find out more about you and Pupil Progress, where would they go?
Tobi Davis (21:46)
Yeah, you're more than welcome to send me a message on LinkedIn, send me a connection request, more than happy to answer any questions or help the CX community in any way that I can. But yeah, if you want to know more about pupil progress and what we do for schools, then do just pop us into Google and let it do its magic.
Conor Pendergrast (22:11)
Excellent,
thank you. Okay, and if you, dear watcher viewer person, would like to find out about future episodes of Support Stack,
you can go to customersuccess.cx/daily and that will get you signed up to my daily, week daily in fact newsletter where you can hear all about support stack and all of my other ideas and suggestions and tweaks on how you could improve your ⁓ intercom customer support experience. Other than that, have a lovely day, bye.