Show Notes
Successful businesses run experiments. Constantly. Harvard Business School professor Stefan Thomke found that companies like Amazon, Google, and Booking.com run thousands of experiments per year. They don't guess—they test. And they grow faster because of it.
Scott's not Amazon. But yesterday, his team ran an experiment—and he's sharing it.
The problem: Churn. One of Scott's SaaS businesses tracks churn monthly. The report shows the number, but not the why.
Churn applies to everyone: Training companies, land investors, service businesses. Anytime you have to go get new customers, you have churn.
The excuse: Someone on the team quoted Alex Hormozi—"small businesses go out of business." Scott's response: "You can't do that."
The experiment: Scott gave AI access to the data. The insight: customers with reviews cancel 58% less. One package didn't allow reviews—and had the highest churn. They turned on reviews for that package yesterday.
The hypothesis: Churn will drop. They're measuring over 30, 60, 90 days. Scott will report back.
The Slack example: Slack found that the more colleagues you added early, the more likely you were to become a paying customer. That's why they push "bring your team."
The lesson: What can you do early to help your customer succeed? When your customer succeeds, you succeed.
The challenge: What experiment could you run this week? Change one thing. Measure before and after. True or false—did it work?
Got a business question? Ask Scott here: scotttodd.net/ask
📜 Full Transcript (Click to expand)
Scott Todd (00:11.384)
There's something that successful businesses do that struggling businesses completely ignore. The answer is that they run experiments. Constantly. They're constantly running experiments. Harvard Business School actually did a study on this, and Professor Stefan Tomsky, Tomkey found that companies like Amazon, Google, Booking.com literally ran.
thousands of experiments a year. They don't guess. They test everything. And because of that, they grow faster. Now I'm not Amazon. I know you're not Amazon. We don't have thousands of engineers that can constantly run different tests. But yesterday in my business we began an experiment and I want to tell you about the experiment. I want to tell you why we're doing this
Because I think it might help you see something in your own business. I'm Scott Todd and I've built multiple seven-figure businesses since leaving my corporate Fortune 300 VP job. And this channel is dedicated to helping you build a company that you love. Now, one of the companies that I have is a SaaS product. It's a software as a service product. And one of the challenges of running any company.
Including a software as a service is this thing called churn. Now, you may not be familiar with churn, and let me break that down for you. Churn is essentially where your customers turn over. And it can apply to literally every single business, even though not every single business measures their churn rate. But think about this for a minute. The more customers that you keep every single month, basically the better and faster the company is gonna grow.
So every month I get a report, and the report is basically a data point. Here's your churn rate. Okay, so we calculate the churn rate every single month for this business and we look at it and we try to make improvements from it. But basically, one of the things that the report doesn't offer us is insight into the why or the potential why this is happening. And look, before you go back and say to me, hey, hold on a minute, I don't have a software as a service.
Scott Todd (02:37.809)
I you know, like churn does we don't even measure churn in my business. I want you to hear me out. You see, churn doesn't just does doesn't appear for, you know, certain businesses. As I mentioned, it appears for every single business. I'm talking about even businesses that you might even think about. I was talking to a a B2C company, training company, and ultimately they had one product that they sold to their consumers.
So their consumers would leave, you know, buy from them and leave and never come back. That is churn. You might say, well, how's that churn? They fulfilled a product. Because tomorrow you have to go out and get new customers. Anytime you have to go out and get new customers, you have some sort of churn in there, which most people don't talk about. One of my companies is a land investing company. And when people stop paying, guess what? That is churn. If they default on the loan, that is churn.
I have to go replace them again. And so all of this really stacks up in almost every single business that you that you want to think about. So I want you to stay tuned. Okay. So as we started digging deeper into the numbers, the one thing that was lacking was hey, what's the driver of this? Why are people leaving? And the team that operates that business, they had lots of ideas. well, it's because of this. It's because
these are small businesses and they turn over. And in fact, I kid you not, I kid you not, someone on my team literally quoted Alex Hermozy and saying, this business is all about small businesses and even Alex Hermozy says small businesses are tough because they go out of business. I'm like, you can't do that. No, that doesn't work. See what we need to make sure that we're doing is that we understand the true driver of this.
Why, why is this happening? And is there some data point that would help us slow that down? Is there something that's missing? So about a week ago, I gave AI access to the data. I said, here's the database, here's the exact same stuff that the team looks at. I want you to go through and I want you to analyze every single potential data point. And it did. Okay, like it's just cranking away at analyzing this data.
Scott Todd (05:04.539)
And then one of the things that we came back with and noticed is we started looking at data points. And one of the things that jumped off the page here is that customers with reviews cancel less. It's a very simple thing, right? Like a customer who has a review cancels less. In fact,
The data points that if they have a review, just even one review on their account, their cancellation rate is 58% less. That is statistically significant. Okay, like it is it is a direct correlation. So, okay, that's interesting. So now it changes the question. If reviews are important, and then that leads to engagement.
And that leads to stickiness, that also leads to lower churn rate. How do we help people get reviews? You see, all of a sudden the question changed within the team's mindset. This whole idea of the data point changed the way that they were approaching the problem, because they were approaching a problem is almost like writing it off, like there's nothing we can do about it. But when we notice that there's this one data point that if we could help them do this.
Then we would retain more of our customers. Now all of a sudden we have something we can rally around. So the experiment that we started yesterday is that we decided to turn on reviews for a package. Yeah, we had one package that did not allow reviews to be collected. And we're like, hold on a minute. This also has the highest churn rate. Okay. the team always looked at that and said, well.
if they want to be able to collect reviews, then they should move to a a higher tiered plan. But see, in a way we were setting them up for failure. Okay, in a way we were setting ourselves up for failure. So what we did was we turned on this this this ability for them to collect reviews. And now the team is driving to say, hey, not only are we turning this on, but we're also going to go back and
Scott Todd (07:21.794)
Begin to help them and push them to go get this review because it helps us, but it also helps them because it adds to their credibility. Now, we're measuring this over the next 30, 60, and 90 days. Like we're gonna measure this on a monthly basis. And I'm not just telling you this because I want you to see what I'm doing. I'm gonna bring you along. I'm gonna show you what the numbers are. We're gonna give this 30 days. 30 days from now, I'm gonna report back and tell you about this experiment.
And the hypothesis here is that the the churn rate's gonna drop. That's what we're we're basically saying. We're gonna do an all out push to move on this one metric, and we're either gonna be right or we're gonna be wrong. No worries. I'm gonna bring you along. And if I'm wrong, if the team is wrong, if we're wrong, if I'm wrong, no problem, we're wrong. But we tried something. And you see, this is where the visibility trap
begins to work in our favor. Now, if you aren't familiar with it, I talk about the visibility trap in prior episodes. It's one of the things that, you know, maybe you don't have access to the data or you don't have the complete data picture. In this case we had a data point, but we didn't really have the complete data picture. We had to dig deeper to kind of get the insight into the number. You see, the the data was always there.
It was always there, but we had to go deeper to to dig in and look at it. This is where we're like, hey, we have to leverage the the visibility trap here. We're in a trap. So my question to you now is what data do you have that potentially you don't have the whole picture? Right? Like what what data do you have that you're not acting on? And the bigger challenge that I would would challenge you with is this.
Is there something in your business that if your customers do this one thing, they are more likely to succeed? And remember, if your customers succeed, you succeed. And if your customers succeed and you succeed, that means that your business is succeeding. So is there something that you could help your customers do early on to get them a quick win that helps you move the needle?
Scott Todd (09:45.663)
And we've seen this play out in other businesses. For example, Slack, the communications tool, Slack determined that the more people that they could the more of your colleagues that you could get onto Slack in the early days, the more likely it was to stick and the more likely that you were to become a paid customer of Slack. So that's why the push is always like, hey, get your team with you. Come on, bring your team with you, bring your team with you.
Because that's a data point that they measured and said, Hey, if you do this, you're statistically more likely to succeed with our product.
So going back to land investing, because a large part of my audience is land investors, real estate investors, is there something you can do early on that would help your customer succeed and stay with you? You have to figure that out for your business. But you should be looking at every business problem with this same lens. What can I do early on to help my customer succeed? Because remember, when your customer succeeds, you succeed. So what are you gonna do?
What experiment could you run beginning today or this week? I'll give you a week. What can you run? And it can be simple. It can be something about maybe test the color on a button on your website. If you change the color of this button, what happens? Measure the before, measure the after, give it some time and see if your hypothesis is right. Don't run a bunch of different tests on that button at the same time or whatever you're doing.
Run a test, give it some time, report back. Did your hypothesis work or did it not? And I will leave you with this one last thing. Remember that a hypothesis has to be true or false. So in my example, it either has to reduce churn, true or false. Does it reduce churn? True or false? And if it's a true, it stays. If it's a false, we revert back.
Scott Todd (11:49.894)
And see you can do that now in your business without a team of engineers. Just change one thing, one metric, test it. And for me, I promise I'm gonna come back to you in the next thirty days and report how our churn test is going. And in the meantime, I will see you in our next episode.
There's something that successful businesses do that struggling businesses completely ignore. The answer is that they run experiments. Constantly. They're constantly running experiments. Harvard Business School actually did a study on this, and Professor Stefan Tomsky, Tomkey found that companies like Amazon, Google, Booking.com literally ran.
thousands of experiments a year. They don't guess. They test everything. And because of that, they grow faster. Now I'm not Amazon. I know you're not Amazon. We don't have thousands of engineers that can constantly run different tests. But yesterday in my business we began an experiment and I want to tell you about the experiment. I want to tell you why we're doing this
Because I think it might help you see something in your own business. I'm Scott Todd and I've built multiple seven-figure businesses since leaving my corporate Fortune 300 VP job. And this channel is dedicated to helping you build a company that you love. Now, one of the companies that I have is a SaaS product. It's a software as a service product. And one of the challenges of running any company.
Including a software as a service is this thing called churn. Now, you may not be familiar with churn, and let me break that down for you. Churn is essentially where your customers turn over. And it can apply to literally every single business, even though not every single business measures their churn rate. But think about this for a minute. The more customers that you keep every single month, basically the better and faster the company is gonna grow.
So every month I get a report, and the report is basically a data point. Here's your churn rate. Okay, so we calculate the churn rate every single month for this business and we look at it and we try to make improvements from it. But basically, one of the things that the report doesn't offer us is insight into the why or the potential why this is happening. And look, before you go back and say to me, hey, hold on a minute, I don't have a software as a service.
Scott Todd (02:37.809)
I you know, like churn does we don't even measure churn in my business. I want you to hear me out. You see, churn doesn't just does doesn't appear for, you know, certain businesses. As I mentioned, it appears for every single business. I'm talking about even businesses that you might even think about. I was talking to a a B2C company, training company, and ultimately they had one product that they sold to their consumers.
So their consumers would leave, you know, buy from them and leave and never come back. That is churn. You might say, well, how's that churn? They fulfilled a product. Because tomorrow you have to go out and get new customers. Anytime you have to go out and get new customers, you have some sort of churn in there, which most people don't talk about. One of my companies is a land investing company. And when people stop paying, guess what? That is churn. If they default on the loan, that is churn.
I have to go replace them again. And so all of this really stacks up in almost every single business that you that you want to think about. So I want you to stay tuned. Okay. So as we started digging deeper into the numbers, the one thing that was lacking was hey, what's the driver of this? Why are people leaving? And the team that operates that business, they had lots of ideas. well, it's because of this. It's because
these are small businesses and they turn over. And in fact, I kid you not, I kid you not, someone on my team literally quoted Alex Hermozy and saying, this business is all about small businesses and even Alex Hermozy says small businesses are tough because they go out of business. I'm like, you can't do that. No, that doesn't work. See what we need to make sure that we're doing is that we understand the true driver of this.
Why, why is this happening? And is there some data point that would help us slow that down? Is there something that's missing? So about a week ago, I gave AI access to the data. I said, here's the database, here's the exact same stuff that the team looks at. I want you to go through and I want you to analyze every single potential data point. And it did. Okay, like it's just cranking away at analyzing this data.
Scott Todd (05:04.539)
And then one of the things that we came back with and noticed is we started looking at data points. And one of the things that jumped off the page here is that customers with reviews cancel less. It's a very simple thing, right? Like a customer who has a review cancels less. In fact,
The data points that if they have a review, just even one review on their account, their cancellation rate is 58% less. That is statistically significant. Okay, like it is it is a direct correlation. So, okay, that's interesting. So now it changes the question. If reviews are important, and then that leads to engagement.
And that leads to stickiness, that also leads to lower churn rate. How do we help people get reviews? You see, all of a sudden the question changed within the team's mindset. This whole idea of the data point changed the way that they were approaching the problem, because they were approaching a problem is almost like writing it off, like there's nothing we can do about it. But when we notice that there's this one data point that if we could help them do this.
Then we would retain more of our customers. Now all of a sudden we have something we can rally around. So the experiment that we started yesterday is that we decided to turn on reviews for a package. Yeah, we had one package that did not allow reviews to be collected. And we're like, hold on a minute. This also has the highest churn rate. Okay. the team always looked at that and said, well.
if they want to be able to collect reviews, then they should move to a a higher tiered plan. But see, in a way we were setting them up for failure. Okay, in a way we were setting ourselves up for failure. So what we did was we turned on this this this ability for them to collect reviews. And now the team is driving to say, hey, not only are we turning this on, but we're also going to go back and
Scott Todd (07:21.794)
Begin to help them and push them to go get this review because it helps us, but it also helps them because it adds to their credibility. Now, we're measuring this over the next 30, 60, and 90 days. Like we're gonna measure this on a monthly basis. And I'm not just telling you this because I want you to see what I'm doing. I'm gonna bring you along. I'm gonna show you what the numbers are. We're gonna give this 30 days. 30 days from now, I'm gonna report back and tell you about this experiment.
And the hypothesis here is that the the churn rate's gonna drop. That's what we're we're basically saying. We're gonna do an all out push to move on this one metric, and we're either gonna be right or we're gonna be wrong. No worries. I'm gonna bring you along. And if I'm wrong, if the team is wrong, if we're wrong, if I'm wrong, no problem, we're wrong. But we tried something. And you see, this is where the visibility trap
begins to work in our favor. Now, if you aren't familiar with it, I talk about the visibility trap in prior episodes. It's one of the things that, you know, maybe you don't have access to the data or you don't have the complete data picture. In this case we had a data point, but we didn't really have the complete data picture. We had to dig deeper to kind of get the insight into the number. You see, the the data was always there.
It was always there, but we had to go deeper to to dig in and look at it. This is where we're like, hey, we have to leverage the the visibility trap here. We're in a trap. So my question to you now is what data do you have that potentially you don't have the whole picture? Right? Like what what data do you have that you're not acting on? And the bigger challenge that I would would challenge you with is this.
Is there something in your business that if your customers do this one thing, they are more likely to succeed? And remember, if your customers succeed, you succeed. And if your customers succeed and you succeed, that means that your business is succeeding. So is there something that you could help your customers do early on to get them a quick win that helps you move the needle?
Scott Todd (09:45.663)
And we've seen this play out in other businesses. For example, Slack, the communications tool, Slack determined that the more people that they could the more of your colleagues that you could get onto Slack in the early days, the more likely it was to stick and the more likely that you were to become a paid customer of Slack. So that's why the push is always like, hey, get your team with you. Come on, bring your team with you, bring your team with you.
Because that's a data point that they measured and said, Hey, if you do this, you're statistically more likely to succeed with our product.
So going back to land investing, because a large part of my audience is land investors, real estate investors, is there something you can do early on that would help your customer succeed and stay with you? You have to figure that out for your business. But you should be looking at every business problem with this same lens. What can I do early on to help my customer succeed? Because remember, when your customer succeeds, you succeed. So what are you gonna do?
What experiment could you run beginning today or this week? I'll give you a week. What can you run? And it can be simple. It can be something about maybe test the color on a button on your website. If you change the color of this button, what happens? Measure the before, measure the after, give it some time and see if your hypothesis is right. Don't run a bunch of different tests on that button at the same time or whatever you're doing.
Run a test, give it some time, report back. Did your hypothesis work or did it not? And I will leave you with this one last thing. Remember that a hypothesis has to be true or false. So in my example, it either has to reduce churn, true or false. Does it reduce churn? True or false? And if it's a true, it stays. If it's a false, we revert back.
Scott Todd (11:49.894)
And see you can do that now in your business without a team of engineers. Just change one thing, one metric, test it. And for me, I promise I'm gonna come back to you in the next thirty days and report how our churn test is going. And in the meantime, I will see you in our next episode.