Testing Fin? Here’s Why It Shows the Wrong Status Page | Ep. 5 with Jen Weaver

Testing Fin is an essential step before any changes go live — but what happens when the test results themselves are misleading? In this episode of Support Stack, Conor Pendergrast is joined by Jen Weaver from Tettra to investigate a specific and surprisingly common issue: why Fin sometimes shows the wrong status page during testing, even when there is no real problem with the site.

The episode opens with a real example from Jen's Intercom setup. While testing Fin's responses to queries about Slack being down, Jen noticed Fin was referencing a major API outage that did not actually exist. The culprit turned out to be a data connector configured to return an "example response" — a static, sample dataset generated by Intercom — rather than pulling live information from Tettra's actual status page.

Conor walks through the fix step by step: navigating to workspace settings, then integrations, then data connectors, locating the "unresolved instance" connector that links to the status page, and switching the test response setting from "example response" to "live response". There is a subtle scroll required to reveal the "retest request" button, but once confirmed and saved as live, Fin immediately begins pulling accurate, real-time data.

The practical implication is significant. When the data connector is set to "example response", every test run through the Fin testing suite, the guidance pages, or via a Fin task will return answers based on fictitious data. In Tettra's case, this meant Fin was confidently telling testers there was a major outage — purely because the example data said so. Switching to "live response" resolved the issue instantly, with Fin correctly reporting that the status page showed no incidents.

The episode closes with a simple but important reminder: before running any meaningful Fin test session, check your data connector settings and confirm they are returning live data. It is a small configuration step, but it has a significant impact on the reliability of every evaluation you carry out. If you are working with Intercom, this is exactly the kind of field-discovered fix that saves hours of confusion.

Join my daily email list to get emails about future episodes.

Episode transcript

Conor (0:00)
Hey, I'm Conor Pendergrast from CustomerSuccess.cx.

Jen (0:03)
I'm Jen from Tettra.

Conor (0:05)
And today it is episode 5 of Support Stack.

Conor (0:16)
Jen, what are we looking at today?

Jen (0:19)
Yeah, so we were digging around in my Intercom space and came across this. So this is the test conversation, but it gave a link as I was testing answers — Slack is broken, right? That was to an API outage and a link that isn't our status page. And so I was asking you, like — what? What is this? How do I fix this? It's not going out to customers, that's my understanding. But it does make it hard to test Fin's answers if they're sending the wrong weird links. So can you walk me through this?

Conor (0:59)
Yeah. So what we're seeing here is that there's a really helpful connection that you've got set up to your status page. So it means that Fin has the ability to check your status page if there aren't any outages. So for example, if there's a big site issue, instead of just being like, "I don't know what's going on," Fin can give the proper context.

Jen (1:24)
Oh, it's brilliant.

Conor (1:25)
But when you're doing testing, if you've got this test data set up, it just makes it really difficult to properly test the flows that you want to test — properly test what Fin will say when the site isn't down. So in this case, what we wanted to do was test and see: okay, how is it troubleshooting error messages? So that's exactly — this is really unhelpful here because it's just gonna give the same "there's a major API outage" when there isn't, it's just the test data. So we want to see what it's like without that. We can actually fix that though. You can change that. So if you go into your workspace settings — that's on the left-hand side with that cog in the bottom left — click settings, and then you're gonna go to integrations and then data connectors. Your data connector in here is called unresolved instance. So this connects to your status page. So if you click onto that, under what you're actually looking to do is test response.

Jen (2:26)
Okay.

Conor (2:26)
And so test response is where you can choose between having the example response, which is what Intercom has generated for you to see what it looks like. But in this case, you're actually going to click live response instead so that you can get just what's currently there — which is, thankfully, because Tettra is doing great, there's only that information. So if you just slightly — here's a little tricky bit — slightly scroll down within this page, retest request, click that. That was hidden. Make sure that you've got — yeah, it is a bit hidden. And now in the top right, you can press — yeah, that's it, correct. So now you can do save and set live. So the next time you go and do a test, either through the guidance page or through the Fin testing suite or for something like a Fin task, then it'll get the live information, which is that there's no site issues, and you can run that again. So if we run that again now, and if we just give a generic question — exactly, Slack is down, Slack is broken — it'll go off, and instead of telling you "my God, there's a major API outage," it'll just say like, cool, the status page looks fine, and here's other information instead. And so it's just a really helpful way of getting the correct response. So there you go. It's got the correct status page and everything.

Jen (3:49)
Yeah. Yeah. Yeah, because then a lot of the time they'll click through and then subscribe there as well.

Conor (3:58)
Awesome. Thank you. Well, thank you very much, Jen Weaver. If people are interested in your work — your incredible work — how would they find you? How would they find what you do?

Jen (4:10)
LinkedIn, connect with me on LinkedIn. And then if you sign up for my newsletter on LinkedIn, it has my podcast Live Chat with Jen, which is every two weeks and it is for support leaders, and also my every Friday Support Snapshot, which is a roll-up of the podcasts of the week.

Conor (4:30)
Fantastic. There's something unique about Live Chat with Jen though. Can you tell people exactly what's unique about it?

Jen (4:37)
Um, it's me. I'm unique. I don't know.

Conor (4:41)
What is it? I mean, do you not focus on — or have you changed the format now? Because it was like, here's one thing that we do, here's how you do X.

Jen (4:52)
Yeah, it's kind of a cousin to this series because it's very tactical and actionable. It's like, how do you do a thing as a support leader? And I've gotten to the point now where I'm getting requests too — could you find a support leader to talk about how they do this thing? And then we go through the steps of how they did that and share it with the audience.

Conor (5:16)
Yeah, like genuinely, I think that's one of the things that I most like about Live Chat with Jen — people come on and they talk not about general life stuff for 60 minutes. It is very directed. It is very clear. It's got a problem that people are talking about that they want to fix. And then it moves on to how they did it, step by step. I really, really enjoy the format. And yes, genuinely, that's one of the major inspirations behind this show — behind Support Stack. There's a real space and a real need for, in our very complicated, very fragmented world as support leaders, we're dealing with more and more narrower, neater products. And it's hard to know how to do them. And when you've got shows like yours which are really geared towards those step-by-step processes, I think it really, really helps to give people a bit more confidence and a couple of ideas on how they can do it. Thank you very much for doing what you do. I really, really appreciate it. And now that I've finished glowing about Jen — if you, dear viewer, want to get future episodes of Support Stack, you can get them by going to customersuccess.cx/daily. That's where you'll find my weekdaily email list, and I will email you every weekday with a short and targeted email that talks about Intercom, about being a customer support leader, and not much about what I had for breakfast — very little about that.

Jen (6:55)
Thank you very much. It is a great email. You should sign up for it.

Conor (6:59)
I appreciate that, thanks Jen. Okay, cool, bye.

Conor (7:02)
Awesome.