Product metrics
North-star metrics, KPIs, funnels, A/B test design and reading results.
Download all 57
Start here 2
The general, start-from-zero entry for a need in this category: it works from a blank start, asks for what it needs and hands off to the specific entries. Example: Plan a trip.
- Write an analytics tracking plan
Writes an analytics tracking plan with consistently named events and properties, when each fires, the question it answers, privacy notes and QA steps. Use when instrumenting a feature.
- Choose metrics for a product
Picks the few metrics worth tracking for a product, from one main metric to journey steps and guardrails, with exact definitions, targets and a weekly review. Use when unsure what to measure.
Specific 50
One situation, audience, tool or format inside the category. Example: Plan a road trip with children.
- Review launch results
Reviews a launched feature against its success criteria, separates real signal from noise and novelty, and recommends whether to iterate, scale or roll back, with the reasoning.
- Analyse a conversion funnel
Analyses a conversion funnel step by step to find the biggest leak, the segments where it differs, likely causes and the experiments or fixes worth trying first. For PMs and growth teams.
- Define an activation metric
Finds a product's activation moment from usage and retention data, defines an activation metric with an action, threshold and time window, and plans how to validate it.
- Define feature success metrics
Defines success metrics for a feature using HEART and goals-signals-metrics, with baselines, targets, guardrails, decision rules and the event data needed. Use before building or launching.
- Design an A/B test
Designs an A/B test plan with a hypothesis, primary and guardrail metrics, minimum detectable effect, sample size, duration, randomisation unit, stop rules and an analysis plan.
- Estimate a feature's impact
Sizes a feature's expected impact before building it, with explicit reach, adoption, effect and value assumptions, a low-base-high range and the cheapest way to tighten the estimate.
- Monthly open-source growth review
Runs a monthly growth review for an open-source project, from collecting public numbers to finding the leakiest funnel stage, judging last month's bets and choosing next month's, with approval gates.
- Build a growth experiment backlog
Builds a ranked growth experiment backlog from a funnel and ideas, with hypothesis, metric, effort, expected impact, minimum sample and run time per test, and flags untestable ideas.
- Define a north star metric
Proposes a north star metric with input metrics and guardrails, tests it against the value users actually get, and shows the rejected candidates. Use when setting product goals.
- Diagnose a metric drop
Investigates a drop in a product metric with a structured tree (data and tracking, segments, platforms, releases, external factors), ranks the hypotheses and gives the queries to run.
- Analyse feature adoption
Analyses how a feature is adopted - exposure, first use, repeat use and depth by segment - to find where users drop off between seeing it and relying on it, and what to change.
- Analyse free-to-paid conversion
Analyses conversion from free plan or trial to paid by cohort, time to convert, upgrade triggers, limits hit and segment, finds where and why free users stall, and proposes tests that respect value.
- Analyse paywall performance
Analyses a subscription app's paywall from views to trial starts to paid by placement, plan mix, price, offer and platform, finds the weakest step and ranks what to test next.
- Build growth accounting
Breaks active users or accounts into new, retained, resurrected and churned each period, computes the quick ratio and shows which flow is really driving or dragging growth.
- Define product-qualified leads
Defines product-qualified lead criteria from usage and firmographic data, tests them against past conversions for precision and coverage, and sets the handoff rules to sales.
- Measure notification impact
Measures the real impact of push, email and in-app notifications with holdouts, opt-outs, unsubscribes, uninstalls and long-term retention rather than open and click rates.
- Plan retention improvements
Turns a product's retention curves into a ranked plan of levers - activation, habit, re-engagement and resurrection - with the experiment, metric and plausible effect for each.
- Product analyst
Acts as a product analyst who turns product questions into decision-ready analysis, checks the data before the result, separates correlation from cause and says how sure each answer is.
- Set up first product analytics
Sets up minimal analytics for a new product - the first ten or so events, five metrics, user identity, consent and a starter dashboard - so early decisions rest on clean data.
- Analyse net revenue retention
Analyses net and gross revenue retention by cohort and segment, splitting expansion, contraction and churn to show where revenue is gained or lost and which segments carry the number.
- Audit event tracking data
Audits existing product analytics for missing, duplicated and misnamed events, broken properties, identity gaps, bot traffic and platform drift, with fixes ranked by the decisions they affect.
- Challenge a feature-retention claim
Tests a claim like "users who use feature X retain twice as well" for selection, reverse causation, tenure and survivorship, and says what analysis or experiment would show whether X causes retention.
- Define account-level product metrics
Defines B2B product metrics at account level - active accounts, seat utilisation, breadth and depth of use, champion and admin activity - with rollup rules from users to accounts and risk signals.
- Define AI feature metrics
Defines product metrics for an AI or LLM feature - task success, acceptance and edit rates, retries, fallbacks to manual, sampled quality, harm signals and cost per successful task - with guardrails.
- Define API product metrics
Defines metrics for an API or developer product - time to first call, active apps, per-customer error rates, integration depth, production adoption and expansion - with data sources.
- Define an engagement metric
Defines an engagement metric that fits the product's natural usage rhythm - the qualifying action, the active-user window, stickiness ratios or usage-day histograms - and checks it against retention.
- Define mobile app KPIs
Defines KPIs for a mobile app - installs to activation, day 1, 7 and 30 retention, crash-free users, store rating, notification opt-in and uninstalls - with formulas, platform splits and data sources.
- Define public service KPIs
Defines KPIs for a public or charity service - completion, take-up by channel, cost per transaction, satisfaction, time to outcome and failure demand - split by user group to show who is left out.
- Define search quality metrics
Defines metrics for in-product or site search - zero-result rate, click position, reformulation, abandonment and time to success - with event logging and a weekly review routine.
- Design a pricing experiment
Designs a pricing test - new-visitor split, geo or time split, or painted door - with revenue per visitor as the metric, fairness and legal checks, exposure rules and how long to run it.
- Find power user behaviours
Builds a power user curve from usage data, finds what the most engaged users do differently and separates behaviours that may cause engagement from ones that only mark it.
- Write an experiment readout
Turns a finished experiment's results into a one-page decision record for stakeholders, with a forwardable summary, the result against the prediction, trust checks, the decision and limits.
- Choose a product analytics tool
Compares product analytics tools such as Amplitude, Mixpanel, PostHog, Heap and GA4 against a team's questions, data stack, privacy needs, skills and budget, and recommends one with a trial plan.
- Explain SaaS metrics on your numbers
Explains SaaS metrics such as MRR, ARR, NRR, GRR, churn, expansion and quick ratio by calculating them step by step on the user's numbers, with checks and common mistakes.
- Measure and shorten time to value
Defines time-to-value milestones for a product, measures the time to reach them by segment, and finds the steps, waits and handoffs that delay new users or accounts from reaching value.
- Quiz me on metric pitfalls
Runs a quiz game of short product scenarios that each hide a metric trap, such as Simpson's paradox, survivorship or a shifting denominator, and explains each one after the answer.
- Set up a weekly metrics review
Sets up a weekly metrics review meeting for a product team - the metric deck, owners, commentary format, deep-dive rotation, agenda and how actions are tracked - so the meeting drives decisions.
- Write activation and funnel SQL
Writes SQL for activation rate and ordered conversion funnels from an event table, with conversion windows, strict step order, time zones, internal users, bots and late-arriving events handled.
- Review an A/B test setup before launch
Reviews an experiment's setup before launch - randomisation unit, exposure logging, metric definitions, interference, power and stopping rules - and lists the fixes needed for trustworthy results.
- Review an open-source project's weekly growth numbers
Turns a week of an open-source project's public numbers (traffic, referrers, downloads, stars, issues, contributors) into what changed, the likely cause and one action for next week. Use every week.
- Set plan limits from usage data
Analyses how usage is distributed across customers to set plan limits and value-metric thresholds, showing who each limit affects, the revenue at stake and the risks of each cut-off.
- Check a KPI for perverse incentives
Reviews a proposed KPI or target for ways people could hit it while harming customers, quality or other teams, then adds counter-metrics, review rules and a safer wording.
- Choose metrics for a two-sided marketplace
Chooses metrics for a two-sided marketplace by stage, such as liquidity, match rate, time to first transaction and take rate, with definitions, balance checks and health measures for both sides.
- Define metrics for an internal tool
Defines a small metric set for an internal tool or process change - adoption, time on task, rework, time saved as capacity, staff ease - with baselines and ways to measure without analytics.
- Define KPIs for a physical product
Defines KPIs for a physical product line - sell-through, return rate, field failure, rating trend, warranty cost and margin per unit, attach rate - with formulas, sources and alert thresholds.
- Explain a change in NPS
Explains whether a change in NPS or CSAT between two periods is real, checking margin of error, response rates, segment mix and survey changes before pointing to the reasons behind it.
- Plan experiments with low traffic
Works out what a low-traffic product can learn from experiments - detectable effect sizes, bigger bets, more sensitive metrics and non-experimental evidence - instead of underpowered A/B tests.
- Set metric targets from a baseline
Sets a commit and a stretch target for a product metric from its baseline, normal variation, seasonality and the realistic effect of planned work, so targets sit outside noise and inside reach.
- Find leading indicators for a product outcome
Finds early user or account behaviours that predict a lagging outcome like retention, conversion or revenue, checks the link holds across cohorts and segments, and picks which to manage week to week.
- Set up growth metrics for an open-source project without telemetry
Defines the handful of public, telemetry-free metrics that show an open-source project's adoption and community health, with collection commands, a weekly archive and leading versus vanity signals.
Advanced 5
Expert or deep work that assumes the user can judge expert output. Example: Plan croissant lamination.
- Define guardrail metrics
Defines a standing set of guardrail metrics for experiments and launches that catch harm to revenue, performance, trust or support load, with thresholds, owners and actions on breach.
- Design a holdout experiment
Designs a holdout or long-term experiment that measures the cumulative impact of a feature, programme or channel, with group size, duration, contamination risks and a decision rule.
- Segment A/B test results
Slices A/B test results by segment without p-hacking - pre-registered segments, interaction tests, multiple-comparison control and shrinkage - and says which segment effects are real enough to act on.
- Diagnose a sample ratio mismatch
Confirms whether an experiment's group sizes really differ from the planned split, traces the cause through assignment, exposure logging, redirects, bots and filters, and says if results are usable.
- Set up an experimentation programme
Sets up how a company runs product experiments - who can launch them, a review step, pre-registration, standard metrics, a results archive and decision rules - sized to its traffic and maturity.
Not: general spreadsheet or statistics work (data-analysis domain).