What Are Vanity Metrics (And How to Identify Them?)

But you need to identify them. Otherwise, you could be tracking or prioritizing metrics that don’t actually help you improve your app’s performance.

In some cases, only looking at these vanity metrics can make you collect misleading insights about your target user. This blog post will cover what vanity metrics are in detail and how to identify them, along with examples.

What Are Vanity Metrics?

A vanity metric is any metric that may seem impressive on the surface but doesn’t paint a complete picture of how you can improve your business performance.

For example, a metric like new users seems impressive, but it doesn't inherently improve business performance. A metric that improves business performance might be user retention because while you do want to acquire new users, you also want them to stick around (and pay, and continue to pay).

How Vanity Metrics Can Sometimes Fool You

Eric Ries, who first coined the term “vanity metrics,” says vanity metrics can fool you if you only measure vanity metrics. As a result, it can make it challenging to find a clear direction on what you can improve about your app:

“When the numbers go down, we invariably blame someone else,” says Ries. “Over time, this allows each person in the company to live in their own private reality. As these realities diverge, it becomes increasingly difficult for teams to reach consensus on what to do next.”

For example, InList, a mobile app for booking exclusive nightlife events, fell into the trap of only measuring its new downloads when it first launched. According to Gideon Kimbrell, founder of InList, in a Forbes article:

“I was thrilled by the number of downloads our app was getting upon its initial release,” says Kimbrell. “Its popularity validated the hard work we had done in choosing the right brand identity.”

“The high download numbers certainly helped us secure partners, contracts, and additional investors as well,” continues Kimbrell. “But after a few months, we began to notice something: Many of these downloads weren’t actually leading to fully qualified, revenue-generating users.”

InList realized their product team needed other forms of data as well to make their vanity metrics more meaningful. Only then could InList have a better strategy to improve its app:

“To have any value at all, [Vanity metrics] needed to be presented alongside percentages of users converted, retention rates, and other key performance indicators that could tell us not only where our users are coming from, but also which ones were staying on for significant lengths of time,” says Kimbrell.

How to Identify Vanity Metrics With 3 Questions

Not sure whether the metric you’re tracking is a “vanity metric?” Ask yourself these three questions to find out:

Is The Metric Actionable?

Each time you look at a metric, you must ask yourself if it’s possible to make a business decision or product initiative based on it. If not, the metric itself won’t be enough to indicate where your app can improve. You’ll need other complementary metrics to make a business decision about it.

Does The Metric Depend on Luck?

A quality metric needs to be something you have control over and not dependent on luck. Otherwise, the data you’re measuring becomes inaccurate and not replicable.

Let’s say that you notice a massive spike in the number of new users who sign up for your app overnight. Your product team is over the moon about your success. However, this spike in new users isn’t thanks to your genius marketing team. Instead, it’s because Elon Musk made a tweet about your app to his followers.

The significant spike in new users might be great news for your app. However, that doesn’t mean that you can expect this type of growth consistently since the increase is mainly thanks to Elon’s tweet (unless you create a partnership).

Does The Metric Reflect Business Performance?

A metric needs to be as accurate as possible and genuinely reflect your business performance. Otherwise, it could mislead your team into believing things about your mobile app or strategy that aren’t true.

For example, a million followers on a social media page may seem like a significant number and a major milestone, but it doesn’t paint the whole picture or always reflect business performance. That’s because the account could easily have bought the followers for $2,000.

Examples of Vanity Metrics For Mobile Apps and SaaS Companies

Still not sure whether or not you’ve been focusing only on vanity metrics the whole time? Here are examples of vanity metrics that, by themselves, don’t help you understand the whole story behind your app performance:

New Downloads

Your mobile app’s long-term success doesn’t just depend on acquiring new users but also on other factors, such as your retention rate. While it’s vital to get new users to your mobile app, Looker founder and CTO Lloyd Tabb doesn’t believe that the metric paints the whole picture:

“The number of people who download your product has no correlation with your company’s survival,” says Lloyd. “How many apps go on viral spins and wither months — even weeks — later?”

Traffic

It’s not enough to just get tons of traffic towards your mobile app landing page. While driving high traffic is essential to boost awareness around your mobile app, you also want to convert these visitors into actual customers.

If you’re generating traffic and few of your visitors are converting, there are many reasons why that could be the case. For example, you could be attracting the wrong users in your marketing campaigns. Or, your landing page copy doesn’t do a good job of converting the value of your app.

Total Number of Users

While your total number of users is a good metric to track, it doesn’t help you fully understand your users' behavior. You also want to measure other metrics such as Monthly Active Users (MAU) and Daily Active Users (DAU) to monitor engagement. For example, to make your total number of users metric as practical as possible, you must also be able to answer the following questions:

• How does the total number of users help you identify which user became a paying subscriber or just stayed a free-trial user?
• How many of those users are actually active on the platform?
• What segment of those total number of users ends up churning after a certain amount of time?

That said, your total number of users can tell you other valuable things about your mobile app. For example, your revenue could tie into the number of users you have.

Social Media Likes and Followers

You may be active on social media to promote your app, and your marketing team works hard to grow your community. An active social media presence is an excellent way to boost awareness of your mobile app. That said, likes and followers qualify as vanity metrics because they’re easy to manipulate.

Also, these metrics don’t tell you how many of these followers become paying users or engage with your app. To better understand your growth on social media, you need to track other vital metrics, such as the click-through rate to your app landing page.

Another critical metric to look for, as well as likes and followers, is your engagement per follower. It’s great to have a lot of followers on social media, but it’s meaningless if few of them actually engage with your content.

Examples of Actionable Metrics

On the other hand, these metrics below are actionable and provide good clues on the overall performance of your mobile app. When you combine them with vanity metrics, you’ll have all the data you need to understand the full picture of your app’s performance:

Churn Rate

Your churn rate measures how many users ditch your app after a specific period of time. It’s a metric you want to measure alongside your new registered users, so you can get an idea of how many unique users you can get versus how many you lose.

You must track this metric frequently because a churn rate that’s too high could be a sign that there’s something wrong with your user experience. Use the formula to calculate your churn rate:

Number of users you lost over the past 30 days / Number of users you had at the beginning of the month 100 = monthly churn rate*

User “Stickiness”

“Stickiness” refers to how many users “stick” to your app and remain loyal. It’s an important metric to monitor since your app’s long-term success depends on how good you are at retaining existing users.

Use the formula to calculate your user stickiness:

Daily Active Users (DAU) / Monthly Active Users (MAU) = User “Stickiness”

Average User Session

There’s no point in getting new users if they’re not actively using your app. Average user session helps you determine how long users spend within your app and if they spend enough time to see its value.

The ideal average user session, of course, will depend on the type of app you manage. If you’re working on a music streaming app like Spotify, the longer the user listens to music on your app, the better.

If you’re managing a ride-booking app like Uber, then your average user session should be pretty short. The user should be able to hop on the app quickly, schedule a ride, and get picked up as fast as possible.

Measure your average user session with the formula below:

Sum of all users’ individual sessions / total number of sessions in a day = Average User Session

Net Promoter Score (NPS)

The issue with many new users is that the metric alone doesn’t show how satisfied users are with your app, which is where the Net Promoter Score (NPS) comes in. The score measures how likely users are to recommend your app on a scale of 1 to 10.

A score of 10 on the NPS means the customer is highly satisfied with your app. A score of 1, however, means they’re extremely unlikely to spread the word about your app to others.

To get the most out of your NPS survey, make sure to also include open-ended questions where the user can provide additional feedback about your app to identify areas of improvement.

Don’t Let Vanity Metrics Trick You

“Vanity metrics aren’t useless,” says Looker founder Lloyd Tabb in an interview with First Round Review. “They have their use case but are points of comparison for other people to evaluate you.”

Once you go through your data to better understand your mobile app performance and user behavior, it’s time to make improvements. For example, that could include adding features to your app such as activity feeds and chat messaging to increase app engagement.