Outspeaking.com

Measure, Experiment, and Iterate Based on User Engagement

When Dropbox analyzed early user cohorts, they discovered multi-device users retained at 4x the rate of single-device signups. This insight reshaped their onboarding! Cohort analysis turns scattered usage data into a roadmap for prioritizing product bets with measurable retention impact.

Quick Take: Cohort analysis divides users into groups by behavior, letting you measure retention rates of 40-60% (typical for SaaS benchmark) and identify which changes actually improve customer lifetime value.

Eric Ries's book The Lean Startup suggests an experiment-driven, iterative approach to discovering the right customers, products, and business models for startups. His oft-stated suggestion is to measure the right things, then analyze those measurements to devise experiments to improve the business strategy. (This is the scientific method and empirical reasoning applied to business processes.)

One such experiment is cohort analysis.

Cohort Analysis

The technique of cohort analysis divides users into several distinct groups (cohorts) based on specific and measurable actions. Each cohort represents a portion of the group as a whole. A cohort graph plots those cohorts and those percentages over time, so that you can see how those cohorts grow and shrink relative to each other over the course of your measurements.

Good analysis requires you to identify these cohorts carefully. By definition, a cohort is a group; by implication, everyone in a cohort has something in common. A thoughtful analysis will resemble a sales funnel. Your goal is to improve the percentage of users who travel through each stage of this funnel, because each stage represents greater user engagement and, presumably, more revenue.

Common examples of cohorts include users reached through a specific campaign, users who exhibit a specific threshold of use, and users with distinct demographic characteristics.

Graphing these cohorts over time allows you to see the effect of changes and specific experiments intended to improve user engagement. Does simplifying the sign-in process help convert visitors into registered users? Does changing the text of the pricing plan lead to more paying customers?

Interactive Cohort Calculator

What To Do Next

  1. Instrument events: capture signup, activation milestone, feature adoption, revenue, and cancellation.
  2. Define 2-3 actionable cohorts (for example, activated within 24h, invited a teammate, completed onboarding checklist).
  3. Run a baseline retention export for the first six months and note drop-off inflection points.
  4. Hypothesize 1 change (tightened onboarding, reminder email, UX friction removal), implement it, and measure.
  5. Re-measure same cohorts after two cycles; compare absolute retention lift and LTV delta.

Tip: Avoid changing multiple funnel steps simultaneously - single-variable experiments produce credible signals.

Using Cohort Analysis Effectively

Two obstacles may present difficulties to using cohort analysis effectively. First, you must instrument your processes to measure user engagement and conversion. The more data you collect, the better. If you overlook a customer segment of any kind, you've lost actionable data—clearly a substantial problem to overcome! Choose each metric to measure with care.

Second, you must choose your cohorts well. Dividing data by the level of engagement is essential to producing effective reports, even if the line between "this customer has spent $90 this month" and "this customer has spent $100 this month" may be thin. The better you define your cohorts, the better you can analyze them.

The results may sometimes be horrifying; the change you expected to have a positive effect may have had the opposite effect. Worse, it may have had no effect at all. This is a victory for cohort analysis—if you have the discipline and the courage to re-examine your assumptions, make a new business hypothesis, and repeat the cycle of change, measure, and review.

Only when you can describe the questions you're trying to answer and form a statistical model of how user behavior supports or negates your hypothesis do you have the framework with which to make sensible decisions.

Success is no guarantee, especially when your business is searching for a business model, but cohort analysis offers the opportunity to measure your progress toward your goals. The better your organization of thoughts and applications, the stronger your feedback cycle and the more you can learn from cohort analysis.

Frequently Asked Questions About Cohort Analysis

What is cohort analysis?
Cohort analysis is a method of dividing users into groups (cohorts) based on shared characteristics or behaviors, then tracking how these groups perform over time to measure engagement, retention, and revenue.
How do you calculate retention rate?
Retention rate = (Users active at end of period / Users at start of period) x 100. For example, if 600 of 1,000 users return after one month, your retention rate is 60%.
What's a good retention rate?
SaaS typically sees 40-60% month-1 retention, ecommerce 20-40%, and subscription services 60-80%. Rates vary by industry and product maturity.
How does churn rate differ from retention?
Churn rate is the inverse of retention: Churn = 100% - Retention%. A 60% retention rate means a 40% churn rate for that period.
What's customer lifetime value (LTV)?
LTV estimates the total revenue a customer generates over their entire relationship with your business. It's calculated as (Average Revenue per User) / (Churn Rate).

Glossary of Core Terms

Cohort
A group of users sharing a defining characteristic at a starting point (e.g., signup week, feature adopted, acquisition channel) tracked longitudinally.
Retention Rate
Percentage of users remaining active at the end of a period versus the starting cohort size.
Churn Rate
Percentage of users (or revenue) lost in a period; inverse of retention.
Activation
The moment a new user experiences core value (varies by product; often correlates strongly with long-term retention).
LTV (Customer Lifetime Value)
Estimated total revenue generated by a user over their relationship, influenced by ARPU, gross margin, and churn velocity.