Metrics and Data for Product Managers | Product Manager Competencies

February 8, 2024
product management
5 min read

By Arun Singhal – ex- Director of Product Management Google

Today, product management has become a lot more data driven and data oriented. A lot of product managers are not good at this skill especially in their early career, so it’s very important to build this skill. They need to be able to strategically navigate through data based product problems and make informed decisions about their product and drive success.

Key Takeaways:

Northstar metric is the main metric that aligns with the core goal of your product, like user satisfaction and revenue generation.
A/B experimentation involves the art of running experiments, interpreting results, and making data-backed decisions.
For product managers, understanding of various metrics and how they may interplay with one another is essential.
Product managers should be able to extract data and analyze it.
Understanding how to influence specific metrics is a key skill for successful product managers.
Problem solving metrics movements involves troubleshooting issues, understanding the interplay of metrics, and providing informed responses to stakeholders.

In this article

Add a header to begin generating the table of contents

1. Northstar and counter metrics

As a product manager, it is very important to know what your North Star metric is. This is the main metric that aligns with the core goal of your product, like user satisfaction and revenue generation. However, it is important to note that you don’t make product decisions only on the basis of Northstar metrics alone. It is essential to include counter metrics in the same as well, as they can act as checks and balances for you. For example, suppose you have an online product and you are trying to get more and more customers to sign up and buy the same. But people are not buying it, because you are getting the wrong set of customers. So your counter metric is your conversion, which tells you that just optimizing for northstar is not the right thing because you are not getting the right value out of that customer. Hence, having a clear understanding of the North star and counter metric is very important.

2. A/B experimentation

For many products, AB experimentation is the bread and butter. Learning how to do it, how they run, how to interpret data from AB metrics is very important, especially in industries like consumer internet space where rapid testing and iteration are very common. Different aspects like what is experimental holds, what does traffic do, what does it mean to take the traffic from 1%-5%, how do metrics change between them, how to read and interpret those metrics are very important for A/B experimentation. It is a tricky skill which takes time and learning but this is an important skill for product managers to learn.

3. Understand Interplay of various metrics

As a product manager, you should realize that your product and business will have multiple metrics. You should know how metric A interplays with metric B and be prepared for questions like- If metric A is increasing does metric B decrease? If they are both increasing, why is that happening? So understanding of those metrics and how they may interplay with one another, how the movement of one may impact the movement of the other is important for a good product manager to know. In many places, a common pitfall is that there will be a data analytics team who may be assigned this work, so it might be assumed that a product manager does not need to do it. But it is crucial for the product manager to understand why things move because that has an impact on the product. So while you may rely on other people to do some of that analysis, to understand the metrics and movement and why certain things may happen will differentiate a good product manager from an average product manager.

4. Extract data and analyze

Another skill that good product managers have is extract data and analyze it. Many organizations and groups may have a data and analytics team which may run the reports up, build dashboards and so on. But they will only build standard stuff. It is necessary that you don’t shy away from learning this skill. Many product managers feel that if they are not from a technical background they don’t need to learn it.

So as a product manager, if you’re able to do some of the analysis yourself and extract that data when needed is a surefire way to learn that skill, become self-dependent and gain a competitive edge.

5. How to move certain metrics

E-commerce conversion is one metric that you can use. If someone from the company comes up to you and tells you that the company is doing something else in the product, so you need to move a particular metric, you should be aware of what you need to do. It may or may not be the right thing to do, but you need to know to be able to make those choices and to push back when it’s the wrong thing to do or if you don’t believe in it. Typically you will make a change in the product and measure the metric. How those interplay with your product capabilities and what you can do to move those metrics is a good skill to have as a product manager.

6. Problem solve metric movements

When metrics deviate from the norm, it is the role of product managers to diagnose the problem. Let us take an example of YouTube to understand this skill. You are doing a regular monitoring of your metrics and lets say your sign-up or conversion rate drops. Let us say your watch minutes drop for a few days. Inevitably in such cases, the product manager will be asked to answer the reason behind the same. It’s a tough skill, but as a product manager you have to be able to interpret that data. You need to know if there is an internal factor affecting the same, such as if you or some other team launched something that may have had an impact. It could also be an external factor such as an India-Pakistan match happening which people prefer watching over YoutTube. This skill involves troubleshooting issues, understanding the interplay of metrics, and providing informed responses to stakeholders. It requires a blend of analytical thinking and a deep understanding of the product’s ecosystem.

Hence, the role of product managers in understanding and interpreting data and metrics is very important today. It is the expertise in these skills that differentiates a good product manager from an average product manager. Be it northstar and counter metrics, A/B experimentation, understanding interplay of various metrics, extracting data and analyzing, moving certain metrics, or problem-solving metrics movements, in today’s evolving product management domain, it is crucial for product managers to learn and implement these skills in their tasks.

Frequently Asked Questions

1. What metrics do product managers use?

Product managers use metrics like on-time delivery quality, customer adoption, revenue generation, customer lifetime value, and so on.

2. Why are product manager metrics important?

Product manager metrics are important because they help in serving as warning signals and also help in aligning the product team to make better decisions in product development.

3. What is A/B experimentation?

A/B experimentation involves the art of running experiments, interpreting results, and making data-backed decisions. It involves comparing the performance of two or more versions of a product to see which appeals more to the users.

4. What is meant by the Northstar metric?

This is the main metric that aligns with the core goal of your product, like user satisfaction and revenue generation. It is the one metric that is the most predictive of a company’s long-term success. For example, in AirBnB, northstar metric would be the number of nights booked.