7 Steps to Designing a Data-Informed Product Loop

Product, at its core, is really an overlap of dozens of disciplines. Not just product management but DevOps, analytics, engineering, consumer psychology, application development, and so on.

The teams responsible for these disciplines all rely on a sound strategy and accurate data to build and ship out usable products to customers. But too often, they jump from their high-level strategy and goals straight to designing their product features. They don’t have a workable method for leveraging data along the way to make informed product decisions, and they don’t test the impact to see if their strategy is working.

The solution is a data-informed product loop: a problem-solving cycle that allows product teams to model, measure, design, and experiment to inform their product strategy. With a data-informed product loop, there’s no jumping to feature ideas, and there’s no data siloing. The most high-performing teams move along the loop swiftly, regularly measuring the impact of their decisions and recalibrating their product strategy.

1. Establish a strong product strategy to explain how users benefit

Your product strategy is your view of how you’re going to win. Strong strategies get to the crux of why your product should exist and how it will make a difference to your users with coherence and clarity.

I really like referring to Richard Rumelt’s definition of strategy from his book “Good Strategy Bad Strategy.”

The “kernel” in your strategy is your unique way of understanding a problem—and then hypothesizing the solution. For example, Amazon’s approach is to offer low pricing and convenience to their customers. That’s the kernel that distinguishes their products.

A “kernel” doesn’t just distinguish your product—it benefits your customers by offering something they can’t find elsewhere in the market. That’s the true purpose behind your product strategy. You want to establish what it is about your product that will help customers in ways no other company can in order to grow your business.

Let’s say a fintech company has a mission to deliver a transparent view of customer finances, so they align their banking with their personal ethics. The strategy here is purposeful banking. They’re taking a stance that aligning ethics and values with banking will drive product growth.

For your own strategy, think about your personas, the wider market, and perceived gaps. Form your own opinions on the challenges customers are facing and how your product will benefit them.

2. Select and refine a usable model

Now that you have your strategy, you need to plug it into a workable model—a method of communicating your strategy to the rest of your team. Models also provide a framework for plugging in the data you’ll uncover later on in the loop.

The models you use and socialize throughout your team are as important as your strategy. Models help orient teams and encourage more effective conversations. But you can’t copy off every model you see. You’ll need to develop some of your own.

As George Box once said, “All models are wrong, but some are useful.” The idea is to figure out what’s usable for your company’s strategy and workshop from there. You can borrow from standard models and evolve yours into a bespoke model that fits your specific strategy.

Standard models

Standard models are tried, true, and tested. They’ve already been developed, and they can be applied to many scenarios, product strategies, and businesses. You’ll find them useful for organizing your thinking.

There’s likely a model in existence for any basic metric you want to measure or behavior you want to frame. A great example is AARRR Pirate Metrics, a model that PLG companies can use to track their customer journey. Dave McClure developed this framework and broke it down into five basic categories: Acquisition, Activation, Retention, Referral, and Revenue. They can be universally applied to any business to analyze user behavior without starting a new model from scratch.

But don’t let standard models tie you down. Based on the factors, ideas, and goals that are specific to your strategy, you can work with existing models and develop them into a bespoke model.

Bespoke models

You have to be prepared, as a company, to apply your own unique characteristics and spin on things to actually make the model useful. Custom modeling does the trick.

Looking at our fintech example, perhaps a traditional banking business model would show that more money in accounts leads to more investing. But since this fintech company has a unique belief that tying ethics and morals to spending insights will yield results, they need to develop a model that fits their strategy more closely.

Aggressively borrow from standard models when it makes sense, so you’re not doing extra work, but dig deeper when it comes to the ideas and beliefs particular to your company. Incorporate those into a usable model.

3. Layer in metrics and measurements to effectively make decisions

Once you’ve decided on a model that captures your product strategy, you can identify which metrics you’ll use to measure progress, test your hypothesis, and make decisions about shipping your product.

Adding metrics gives you the ability to measure the variables and factors in your selected model. Metrics lead to data-informed decisions, keep your teams aligned, and clearly communicate progress.

Let’s assume that Pirate Metrics is a relevant starting framework for your product strategy, and you’re using that as a standard model. Your team now needs specific benchmarks to understand progress toward goals at each stage of the framework.

Lean Analytics is a framework you can use to map specific metrics to the AARRR stages, depending on where they’re most helpful along the customer journey. For instance, SEO keyword rankings are good metrics for setting expectations and measuring progress at the acquisition stage, while transactions are a strong metric for the revenue stage.

Our hypothetical fintech company might be using the North Star framework, with primary bankers as their north star. The specific measurement for their model then becomes the number of unique users with direct deposit activity and spending activity in the last 30 days—which indicates primary bankers.

4. Identify leverage points that yield results

Leverage points are specific places where, by focusing a little bit of extra energy on experimenting, your team can realize outsized results instead of continuing with the status quo. Identify those points in your model so you can prioritize where to focus your team’s efforts.

You can identify leverage points by analyzing the data from the metrics you established in the prior step. They might point to a specific customer journey stage, usability issues that pile up together, or a variable that potentially yields 2X improvement.

Our hypothetical fintech company might discover that funding their accounts is a point of leverage. They believe that there are more opportunities around funding efficacy, and if they increase the amount of funding, it will yield outsized results. They’ve identified funding amounts as an area for experimentation.

5. Place your bets through designing and experimenting

Design and build experiments that influence your leverage points. Use the data and metrics from your model to place bets on new product features and then test how they actually perform on a cohort of users.

Use and analyze data to design and run experiments. This might look like adding new features, running campaigns, or testing out systems. The important thing is to focus on experiments with the highest probability of impact and learning. That’s where the bet lies.

Look at your prior data, prior intuition, and what might be happening along your customer journey and model it out with real data to design a safe experiment. In a situation where funding efficacy is the leverage point, you might look at where customers are landing first—and where funding is slowing down.

Map out what other options they have, and analyze where they’re dropping off. Maybe you discover that customers are dropping out before they even get to the funding options, so you decide to run an experiment that accounts for this user journey. You design and test a new feature earlier in the journey to see if you can improve funding efficacy.

Real experimentation takes discipline and dedication. You won’t always experience successful experiments, so there should be signs of failed tests to mix in with the good. The way to make progress is through a culture of review, learning, and follow-up on every experimentation result.

6. Measure the impact on your strategy

When your feature or experiment ships, you can set about understanding how it impacts not just immediate metrics and leverage points but also the factors that really matter: sustainable, long-term, differentiated growth.

At the end of the day, you want to assess if your test had a positive effect on the leverage points and the results you’re seeking. Product teams can measure impact in many different ways: through A/B testing, controlled testing, straightforward usability testing, and beta testing.

Our fintech company might measure the impact of their experiment on funding sources and funding effectiveness, which was their leverage point. Those are the immediate results and metrics they hoped to test.

But they also want to know how their experiment impacted retention and the likelihood of someone becoming a primary banker. That’s their north star metric and, ultimately, what ties back to their product strategy.

7. Integrate learnings into the product loop

Although technically the last step in our cycle, it absolutely cannot be skipped. Integrate the results of your experiments into your product strategy, models, measurement systems, and future experiments—and have a plan for sharing these learnings across your company.

Even the most data-driven companies that we speak to say they run experiments all the time, but they don’t share those results or impacts outside of their own teams. That renders the cycle ineffective because you’re not sharing the learning or growth process throughout the company.

Through your learnings, you’re deciding where to pivot your strategy and where to continue progressing. But you have to communicate that to every team member and share that knowledge instead of keeping it siloed for the next iteration of modeling and experimenting.

Get started with the North Star Statement Exercise

Download the North Star Statement Exercise to brainstorm your own north star, opportunities, and investments. Create a usable model for your company to build a data-informed product loop.