How to crack product metric questions in PM interviews

By Max Serrano on November 12, 2019 How we wrote this article

Metric questions are common in interviews at tech companies like Facebook, Google, Amazon, etc. For instance, your interviewer might ask, "What are things LinkedIn should measure on a regular basis?"

These questions can feel really unsettling at first. But the good news is that if you know how to approach them they can become fairly easy to answer. So, let's step through our suggested techniques for answering metric interview questions as well as an example to help you prepare for your interview.

Here’s an overview of what we will cover:

What is a metric interview question?
How to answer metric interview questions
Example metric questions with answers
List of sample metric interview questions (from Meta, Google, Amazon, etc.)
How to practice metric questions

Click here to practice 1-on-1 with FAANG ex-interviewers

1. What is a product metric interview question? ↑

Metric interview questions test if candidates can perform data analysis and select key metrics that matter most to the success of a product. Employers like Facebook and Google use these questions to evaluate critical thinking and communication skills. There are two types of metric questions: Metric definition questions and Metric change questions.

Metric definition questions focus on your ability to define metrics that provide clarity on the health of a product or feature. Here’s an example question: “What metrics would you use to determine success for Facebook Sponsored Posts?” There are many different metrics you could be tracking (e.g. impressions, clicks, return on ad spend, etc.) and your interviewer will want to hear you select the most important ones using a rigorous process.

Metric change questions test if you know what to do when a key product metric (e.g. traffic, revenue, engagement, etc.) is going up or down for no apparent reason. For example, “Facebook Newsfeed engagement is down 10%. How would you report this issue to the CEO?” There are many different reasons why this decrease might be happening and your interviewer will want to see you take a bulletproof approach to find the root-cause of the issue.

2. How to answer metric interview questions ↑

A common mistake candidates make when answering metric questions is to provide an unstructured answer. In this next section, we’ll walk through two step-by-step approaches you can use to avoid that pitfall — one for Metric definition questions and one for Metric change questions.

2.1 Metric definition questions: GAME method

Metric definition question example:

“What metrics would you use to determine success for the Facebook Newsfeed?”

Our recommendation is to use the GAME method to answer Metric definition questions. Let’s walk through each of the four steps one by one:

Goals
Actions
Metrics
Evaluations

1. Goals: First, you should start by making sure you understand the product properly and agree with your interviewer on specific user and business objectives. Agreeing on the goals upfront is extremely important because what you measure will change entirely depending on what you’re trying to achieve with the product.

For instance, imagine your interviewer has asked you, “What metrics would you use to measure the success of the Facebook Newsfeed?” The Newsfeed is one of Facebook’s main features and it fulfills many objectives for the business. It’s a primary driver of user engagement, a strong revenue generator, and the main place where Facebook can test new personalization features. So, how do you determine what the most important objective is?

There’s no single right answer to that question but you could begin by saying, "My understanding is that there are several use cases for the Newsfeed including engaging users, generating ad revenue, and testing new personalization features. To stay healthy, Facebook needs strong user engagement, so I would assume that this is the primary business goal. Is that what you have in mind too?"

2. Actions: Next, you should think about all the actions users can take in the product. Listing each action will help you focus on available metrics and avoid those that aren’t trackable.

The easiest way to do this is to think about what it really means for a user to be “engaged.” You should aim to list every relevant action (e.g. creating a post, viewing a post, commenting, liking, sharing, etc.). However, it’s also a good idea to avoid going into too much detail. For example, simply list “liking” as a category of action instead of “liking a friend’s post, liking a group’s post, liking an event, liking a comment, etc.”

Next, you need to prioritize your list of actions. Remember to use the goals you established in the previous step to explain your thinking. For instance, you might say something like, “There are a number of user actions which reflect engagement but I think the three most important ones are likely to be posting, commenting, and sharing.”

3. Metrics: Once you have a prioritized list of actions, it’s time to define associated metrics for each one. In this step it’s important that you define the metrics you will measure with precision.

Let’s imagine that in our previous step, after discussing with your interviewer, you prioritized the “comment” and “share” actions. Talking about “comments” and “shares” isn’t sufficient as there are many different ways in which you can track these actions.

In order to measure engagement based on these actions, here are some metrics that would make sense to track:

"Comment" metrics
- Comments per thousand sessions
- Comments per thousand posts seen
- Comments per thousand posts clicked
"Share" metrics
- Shares per thousand sessions
- Shares per thousand posts seen
- Shares per thousand posts clicked

4. Evaluations: After defining metrics, you should conclude the discussion by answering the initial question with the metrics you recommend. Finally, evaluate the metrics you have selected by highlighting trade-offs and limitations.

For the Facebook Newsfeed example, you could summarize by saying something like, “So, in order to increase Newsfeed engagement, I would first look at comments and shares per thousand sessions. These metrics would give us an idea of meaningful engagement for the average user.”

The last key step is to show that you understand the strengths and weaknesses of your recommended metrics.

So you could wrap up your answer by saying something like, “We should keep in mind that not all commenting and sharing is positive. Perhaps we should monitor the sentiment of these actions to make sure we're creating a positive experience overall. Also, commenting and sharing represent deep engagement. If we are interested in shallower engagement we might need to consider other metrics (e.g. Number of sessions per user per day, Number of likes per user, etc.) to measure engagement for users who don't comment or share but still use the product frequently.”

2.2 Metric change questions: IGotAnOffer method

Metric change question example:

“Facebook Newsfeed engagement is down 10%. How would you report this issue to the CEO?”

We developed the IGotAnOffer method below to help you give a clear and thorough answer to Metric change questions. Let’s walk through each of the three steps one by one:

Define the metric change
Explore possible root-causes of the change
Conclude

1. Define the metric change: Many candidates skip this step and start listing ideas in an unstructured way. This is a big red flag for interviewers. Before starting to answer the question you need to define the metric change in detail. This often entails asking for three pieces of information:

The exact definition of the metric we’re talking about
The time period over which the metric has changed
The characteristics of the user segment impacted by the change (e.g. device type, country, etc.)

Let’s imagine your interviewer asks, “Facebook Newsfeed engagement is down 10%. How would you report this issue to the CEO?” To clarify your understanding, you might then ask something like, “What exactly do we mean by engagement here? Is that a direct measurement like number of sessions, or a combination of more detailed metrics (e.g. likes + comments + shares per user, etc.)? And over what time period has that changed occurred? Finally, do we know if all users are impacted or only a particular segment?”

Here let’s imagine your interviewer tells you that the number of Newsfeed sessions for all users has gone done 10% since yesterday. Only after you’ve clarified that information should you start exploring possible root-causes.

2. Explore possible root-causes: Interviewers want to see that you’re thinking of every possible problem which could cause the metric change you are investigating. In order to ensure you’re thinking comprehensively and communicating clearly, you should:

Create a MECE framework for possible root-causes
Brainstorm potential root-causes to the problem within this framework
Discuss the different root-causes identified with interviewer

For our drop in engagement example, you could split your investigation between internal and external factors:

Internal factors: possible problems internal to Facebook that can explain the drop in sessions
External factors: possible problems external to Facebook that can explain the drop in sessions

Note that the framework above is MECE: Mutually Exclusive and Collectively Exhaustive. The root-cause of the problem has to be internal OR external, as there is no other option.

Once you’ve created your framework you need to brainstorm ideas and possible causes within each of the areas you identified. For example here you could populate your framework with the following points:

Internal factors:
- Data accuracy (e.g. This metric change was measured incorrectly)
- We broke the product (e.g. Someone in the organization made a change that broke the Newsfeed)
- We made the product worse (e.g. The quality of content in the Newsfeed has degraded)
- etc.
External factors:
- User habits (e.g. User changed habits overnight for some reason, like a protest)
- Referrers (e.g. Google had an outage)
- Competition (e.g. Snapchat released a new feature that captured a portion of our normal Newsfeed traffic)
- etc.

The interviewer will most likely stop you to talk about some of your proposed root-causes, which is perfectly normal. Listen carefully to what they say, as they will often provide clues that will point you in the right direction.

3. Conclude: Finally, you should conclude by summarizing your findings related to the initial question you were asked.

Let’s imagine that you discovered Facebook suffered an outage which caused the drop in sessions. In this case you might summarize by saying something like, “Based on what we’ve talked about, I would explain to the CEO that Newsfeed sessions dropped 10% since yesterday because of a major outage this morning. I would also make it clear that we came to this conclusion by ensuring it could not be any other cause, and double-checked that the duration of the outage aligns with the 10% drop.”

3. Example metric questions with answers

Now that you know how to approach metric questions, let's look at some full examples.

Try answering the question by typing your response in the comments at the end of this article, be sure to follow the appropriate method above. Play both the role of the interviewer and candidate. This is a great opportunity to gain some practice for your PM interviews.

The key is to answer the question without seeing other people’s answers. To do so, scroll down directly to the bottom and leave your answer before reading other candidates’ proposals.

3.1 YouTube traffic went down 5% — how would you report this issue to the executive team?

Answer:

You will find our proposed answer to the question below. Before taking a look at it, make sure you go to the bottom of the page and answer the question by yourself in the comments section. There are only so many opportunities to prepare for metric interview questions!

This is a metric change question, so we’ll use the IGotAnOffer method described above.

Step one: Define the metric change

First we want to completely understand the question and define the metric, as “traffic” is a bit vague. Here are some questions that immediately come to mind and will help confirm our understanding:

What exactly do we mean by traffic here? Website visits, or plays on videos, or time spent watching? etc.
Is this impacting all user segments? Or maybe only a particular device type, or country, or browser? etc.
When did the dip start and how long did it last?

For the rest of this example let’s imagine the interviewer tells us that the average time spent watching per session is down 5% month-over-month worldwide on mobile only.

Step two: Explore possible root-causes

Now that we know the exact problem we can create a framework to identify the root-cause of the time spent watching per session decrease:

Internal factors: possible problems internal to YouTube that can explain the drop in average time spent watching per session
External factors: possible problems external to YouTube that can explain the drop in average time spent watching per session

Note that the framework above is MECE: Mutually Exclusive and Collectively Exhaustive. The root-cause of the problem has to be either internal OR external, there is no other option.

Within that Internal / External framework, let’s brainstorm different root-causes that could drive down the average time spent watching per session:

Internal factors:
- Data accuracy (e.g. We should confirm our reporting tool is working as expected)
- Context (e.g. This could be an expected seasonal drop)
- Access to the product (e.g. We might have had a major outage)
- Product changes (e.g. We could have shipped some code that introduced a bug)
- Product quality (e.g. Recent content might have been bad enough to drive away traffic)
External factors:
- User habits (e.g. Video consumption is down across the industry)
- Referrers (e.g. Facebook made a change to limit linking to YouTube videos)
- Competition (e.g. Instagram TV released a new feature that captured our users)
- Society (e.g. There’s less active users because of civil unrest in a country that typically drives a lot of traffic)

We can also dive deeper on some of the root causes. Here’s an example of how the conversation could go.

Data accuracy: Have we checked this dip compared to a similar metric — for example, time on page? Is there anything else indicating that our reporting tool might be broken?

Let’s imagine the interviewer confirms that we’re seeing a similar dip with time on page that leads us to believe the data is accurate. They also confirm that we’ve double-checked that our reporting is working as expected.

Context: Is there any history of a similar dip? We’re close to major holidays, perhaps that could lead to less product usage overall.

The interviewer let’s us know the only time we’ve seen a drop of more than 1% this time of year was due to major outages, but we have had 100% uptime in the past few months. This information also addresses our next factor, as access to the product hasn’t been disrupted.

Product changes: Is it possible we shipped some code that introduced a bug? Or, did we release any significant feature changes?

Let’s imagine the interviewer says that the user interface for the video player was recently changed on mobile. We would then ask questions to explore this such as, “What exactly was changed in the UI and was any drop to be expected as a result?”

After discussing with the interviewer, we learn that the UI change involved making the “Send video to device” button two times larger, and reducing the “Full screen” button by half its original size. The interviewer also explains that the “Send video to device” button can be used to play a video that’s on your mobile phone on another device such as a Chromecast, Roku, SmartTV, etc.

This is interesting, but we need to investigate further. At that point, we could form a hypothesis and say something like, “Have we noticed a change in the frequency at which the ‘Send video to device’ and ‘Full screen’ buttons are being used on mobile? Maybe mobile users are having a harder time tapping the ‘Full screen’ button now that it’s smaller, and are tapping the ‘Send video to device’ button by accident because it’s too big? Tapping ‘Send video to device’ would end the session and therefore decrease the average time per session. And if the smaller ‘Full screen’ button is harder to tap, it might frustrate some users and lead them to end their session early.”

Let’s imagine that the interviewer confirms that this is likely one of the drivers of the decrease. Before making any final conclusion, we would continue exploring the remaining elements (i.e. Product quality and External factors) to confirm that they aren’t also a driver of the change we are observing. Let’s imagine we quickly discuss this with our interviewer, and they confirm there’s nothing significant to report related to those remaining possible factors.

Step three: Discuss and conclude

Now that we’ve stepped through our framework, we would conclude by saying something like, “Based on what we’ve talked about, I would explain to the executive team that our time spent watching per session has dropped by 5% month-over-month on mobile because of a change in the video player UI, which has negatively affected the user experience. Mobile users are having a hard time tapping the new, smaller ‘Full screen’ button and are ending their session prematurely. They are also now more often tapping the larger ‘Send video to device’ button unintentionally which also ends their session early.”

Finally, we could enrich our answer by saying something such as, “We’ve also double checked that no other drivers were causing this engagement dip. A good next step here would probably be to revert the UI changes, think through why we made these changes in the first place, and brainstorm how we can achieve the goals we were aiming for in a different way.”

3.2 Example metric question mock interview: "Instagram engagement is down 10%. What do you do?"

Watch how Damien, ex-Facebook PM, has a clear flow of questions to tackle this problem. Pause the video as you go along in order to construct your own answer.

3.3 Example metric question mock interview: "Pick 3 key metrics for YouTube"

Watch how Mark, ex-senior product manager at Google turned interview coach, gives a best-in-class answer to this metric definition question. Pause the video as you go along in order to construct your own answer.

4. List of sample interview questions: Metrics ↑

If you would like feedback on your answer to any of the questions listed here, you can leave your answer below and our team will get back to you. If you'd like to learn about the other types of questions you may face, you can also visit our ultimate guide to product manager interview questions.

4.1 Practice Metric definition questions

Here is a list of metric definition questions that were asked in PM interviews at Facebook, Google, Amazon, Microsoft, and LinkedIn, according to data from Glassdoor.com:

What metrics did you use to measure the successful launch of your product?
Imagine you are the PM of the Facebook Newsfeed — how would you measure retention?
What metrics would you use to measure the success of Facebook’s “Save Item” feature?
How would you determine post ranking in the Facebook Newsfeed?
Tell me what metrics you would look at as a product manager for Instagram ads
How would you measure the success of the new YouTube Player UI?
What analysis would you use to understand if we should increase the price of an Amazon Prime Membership?
How would you determine the negative value of an abusive posting?
What are the things that Netflix should measure and analyze on a daily basis?
How would you measure the success of Apple's WWDC event?

4.2 Practice Metric change questions

Here is a list of metric change questions that were asked in PM interviews at Facebook, Google, Amazon, Microsoft, and LinkedIn, according to data from Glassdoor.com:

Users are no longer signing up for our email list — what would you do?
There's been a 15% drop in usage of Facebook Groups — how do you fix it?
You have just localized an ecommerce site in Spain and now see that traffic has reduced — what could be the reasons?
The usage of Facebook Event’s “Yes I’m going” dropped 30% overnight — what data would you look at to try to isolate the issue?
You are looking at YouTube’s Daily Active User data worldwide and notice a 10% jump compared to yesterday in Indonesia — what happened?
You are the PM of Facebook 3rd Party Login, and you see your numbers are declining 2% week-on-week — what do you do?
Reddit traffic went down 5% — how would you report this issue to the executive team?

Now that you have a list of sample questions to work with, it’s important to consider how you will practice with these questions.

5. How to practice metric questions ↑

With a lot to cover, it’s best to take a systematic approach to make the most of your practice time.

Below you’ll find links to free resources and four introductory steps that you can take to prepare for product metric questions.

5.1 Study the company you're applying to

Get acquainted with the company you’ve applied to. In many cases, the product questions you’ll be presented with will be based on real-life cases the company is facing. If you’re applying to a specific team, study up on their products, the user, etc.

Take the time to find out which products you’ll most likely be working with, based on the job description, and research them. Look up relevant press releases, product descriptions, product reviews, and other resources in order to discuss what’s most important to the role: the company’s product.

If you'd like to learn more about a specific company's PM interviews, then we'd encourage you to check out our guide for that company below :

5.2 Practice by yourself

Of course, first of all, you'll need to take time to practice building your answers within your chosen framework. We recommend you start by doing this on paper. You can also watch some mock interviews on our product management YouTube channel. That way you can see what an excellent answer looks like.

Once you’re in command of the subject matter, you’ll want to practice answering questions. But by yourself, you can’t simulate thinking on your feet or the pressure of performing in front of a stranger. Plus, there are no unexpected follow-up questions and no feedback.

That’s why many candidates try to practice with friends or peers.

5.3 Practice with peers

If you have friends or peers who can do mock interviews with you, that's an option worth trying. It’s free, but be warned, you may come up against the following problems:

It’s hard to know if the feedback you get is accurate
They’re unlikely to have insider knowledge of interviews at your target company
On peer platforms, people often waste your time by not showing up

For those reasons, many candidates skip peer mock interviews and go straight to mock interviews with an expert.

5.4 Practice with ex-interviewers

In our experience, practicing real interviews with experts who can give you company-specific feedback makes a huge difference.

Find a product manager interview coach so you can::

Test yourself under real interview conditions
Get accurate feedback from a real expert
Build your confidence
Get company-specific insights
Learn how to tell the right stories, better.
Save time by focusing your preparation

Landing a job at a big tech company often results in a $50,000 per year or more increase in total compensation. In our experience, three or four coaching sessions worth ~$500 make a significant difference in your ability to land the job. That’s an ROI of 100x!

Click here to book product manager mock interviews with experienced PM interviewers.