Knowledge Base

Tools, ideas and methods that help you become a better product manager

4 Disciplines of Execution

The 4 Disciplines of Execution is a model for focusing work on important goals despite the daily interruptions and distractions. In the book by the same name, Chris Mcchesney, Sean Covey and Jim Huling explain it as an “Operating system for achieving the goals you must achieve”.

A/A Testing

An A/A test, (also known as an uniformity trial), is an experiment where two populations are exposed to the same experience (in contrast to an A/B test where the groups are exposed to different experiences). The technique is primarily useful to validate the testing infrastructure and cohort assignment for A/B testing.

A/B Testing

An A/B Test (also known as a Randomized Controlled Trial) is a controlled experiment aimed at measuring the impact of a product change, with the intent of proving causality. Essentially, an A/B test is a reliable way of judging, with high probability, whether an observed user behaviour is caused by a specific modification in how a product looks or how it works. When A/B tests are applied systematically, they can be used to release valuable product changes with confidence, and to prevent low-value or damaging changes from being released.

AIDA Model

The AIDA Model (Attention, Interest, Desire, Action) is the earliest documented mnemonic for customer engagement models, originally used for advertising and marketing. The AIDA acronym emerged somewhere between 1898 and 1907 (see the section on the origin of AIDA).

Acquiescence Effect

The Acquiescence Effect (also known as Acquiescence bias, Yea-saying or Agreement effect) is an observed tendency for people to choose positive answers (such as Yes or Agree) more frequently than negative answers, regardless of the question, in the range of 10-50%. This effect is important for user interviews and research because it skews the data from opinion surveys and provides false confidence. The effect is also important for anyone involved in product management and stakeholder alignment, as it can generate a false sense of agreement.

Air Sandwich

An Air Sandwich plan (also known as an Underpants Gnomes Plan) contains lofty objectives and a detailed feature roadmap, but nothing in between. Such plans are typically misleading because there is no clear way to track progress apart from measuring effort, no way to challenge assumptions and no clear strategy to deal with unexpected problems.

Attitudinal Metrics

Attitudinal metrics measure user attitude, how users feel or what they say about the product. They are usually collected through surveys, interviews or focus groups. In product management, attitudinal measures are often used to track perceived usability and customer satisfaction.

Beachhead Market

A Beachhead Market (also called a Beachhead Segment) is an initial market segment that a new product can capture, allowing it to expand and grow so that it can start capturing customers in other market segments. Using a beachhead segment allows new entrants to break into an established market with an early version of a product that is not at parity with market leaders.

Big Rocks Theory

The Big Rocks Theory is an approach to time management and prioritization that suggests prioritising around a small number of important things to ensure that they get done, rather than trying to achieve the maximum number of tasks in a specific period.

Bowling Alley Framework

The Bowling Alley Framework (also called Straight-Line Onboarding) is a user journey optimization technique from Product Led Growth, helping to reduce the “time to value” and increase conversion and retention during the initial user interactions with a product.

Christmas prioritisation

Christmas Prioritisation is a method for splitting a large initiative with complex objectives into a sequence of smaller and simpler milestones based on urgency. It is named based on a mispronunciation of the name of Chris Matts, a UK-based business analyst who combined prioritisation methods with ideas from financial trading models to create Real Options.

Commander's Intent Statement

A Commander’s Intent Statement is a concise and clear outline of the purpose for an initiative or a mission, complementing a specific execution plan by providing the wider context and the thinking behind the plan. The goal of a commander’s indent statement is to align everyone involved on the key objectives, and to ensure that people can use their own initiative and deviate from a specific plan if needed, but still achieve the ultimate goals.

Community Driver Taxonomy

In The Embedded Entrepreneur, Arvid Kahl proposes an Audience-Driven approach to discovering problems and product opportunities, engaging with communities and observing them. He proposes a taxonomy of communities based on what drives them, noting that different strategies apply to discovering product opportunities and engagement based on the key reasons why communities form.

Community Suggestion Forum

A Community Suggestion Forum is a digital board for community feedback and proposals that allows community members to submit new ideas, vote for each other’s proposals, leave comments and discuss suggestions. Create a Community Suggestion Forum Use Votito to host a community forum with public suggestions, voting and comments. Collect feature requests and idea proposals from your community, allow people to vote and comment on suggestions, and engage with groups of users easily. Get started free, in just a few minutes. Create a Community Forum

Confidence Meter

The Confidence Meter is an estimation tool that helps product managers evaluate the reliability and credibility of proposed improvement ideas based on the strength of the supporting evidence.

Cost of Delay

Cost of Delay (CoD) is a popular factor for work prioritization, applied in queue-based or iterative work scheduling. It can be powerful for rough (order of magnitude) comparisons, but also misleading if applied naively. This article explains the key ideas behind CoD, how to calculate it, applicability and limitations in the context of product work, and a brief history of the method.

Dialectical Bootstrapping

Dialectical Bootstrapping is a technique for improving the accuracy of estimates in situations where multiple independent experts cannot be consulted easily. The process involves a single person making two judgements, the first providing an anchor and the second attempting to be as different as possible from the initial estimate, while still staying plausible. The final estimate is the average of the two.

Dichotomous Questions

Dichotomous questions (also known as Forced Choice) are questions that offer exactly two possible answers. The offered answers are mutually exclusive (with an unambiguous split), and there is no middle option (such as “Don’t Know” or “Not Applicable”). This article explains the key things you should know when using dichotomous questions in research.

Earlyvangelists

Earlyvangelists are the ideal initial customer segment for a new product, proposed by Steve Blank in The Four Steps to the Epiphany. According to Blank, those are the customers that product managers can best rely on for feedback and early sales, and for spreading the word about the early product versions.

Eisenhower Matrix

The Eisenhower Matrix is a prioritisation filter for work based on urgency and importance. It helps schedule work and priorities by categorizing tasks into four categories:

Expectation Ratings Survey

The Expectaton Ratings (ER) survey is an attitudinal research method that captures the relationship between perceived and effective usability. It is particularly useful for prioritising user experience improvement tasks and redesign ideas.

Experience Mapping

Experience Mapping is a technique for visualising customer issues and building a shared understanding about potential product opportunities. It can be useful as a first step in deciding about potential impact or outcomes to explore for further product research or planning. In particular, it’s common to run an experience mapping session before creating an Opportunity Solution Tree, but this method can also be used with other road-mapping or prioritisation techniques.

Experimentation Growth Model

The Experimentation Growth Model (also called the Experimentation Maturity Model and the Experimentation Evolution Model) describes the typical phases that product teams and organizations follow while developing technical and organizational capabilities for experimentation (such as A/B testing), in particular in the context of online software products. It can be useful for teams and organizations both for self-assessment and as a set of guidelines to improve their experimentation capabilities.

Five Stages of Growth

The Five Stages of Growth model provides guidelines for focusing on a specific type of work at different times during a product lifecycle, and can be helpful to select objectives and product development ideas that could create a big impact.

GIST model

GIST is a hierarchical planning model developed by Itamar Gilad while working at Google, and documented in his book Evidence Guided.

Given-When-Then

Given-When-Then (also known as Gherkin) is a format for business-oriented acceptance tests and executable specifications.

Growth Pyramid

The Growth Pyramid (previously called The Startup Pyramid and The Startup Growth Pyramid) is a model for product evolution after the product/market fit. It provides guidelines and milestones that a product needs to achieve before significant growth.

Guardrail metrics

Guardrail metrics (sometimes called diagnostic metrics) help keep a process on track by detecting unintended impacts, unwanted side-effects and avoiding short-term solutions that might cause long-term damage.

HEART framework

HEART is a framework for user-centered metrics developed at Google to track progress towards goals for consumer (B2C) web applications. It helps product teams decide how to measure the impact of design changes, set the Overall Evaluation Criteria for A/B tests and track user behaviour changes at large scale. The HEART framework is user-centered instead of business centered, so it focuses primarily on the delivered value part of the value exchange loop.

Hierarchy of Effects

Hierarchy of Effects systems capture customer engagement flows in an idealised sequence of stages, in order to plan marketing campaigns, design product funnels and develop metrics to monitor user progress towards product goals. Some notable examples are AIDA, REAN and AARRR.

Hierarchy of Powers

The Hierarchy of Powers is a model that sizes up product opportunities and economic competition in relation to different types of economic power, to help guide product development and investment decisions. The model is specifically useful to predict growth opportunities and what a product needs to achieve in a specific market environment to compete successfully, and to drive alignment between different types of forces influencing product growth. Combining and aligning different types of power allows a product to achieve “escape velocity” and beat the market.

ICE Score

The ICE Score is a prioritisation method for quickly comparing different ideas based on three factors: Impact, Confidence and Ease.

Idea Bank

An Idea Bank is a collaborative space for collecting, organizing, and prioritizing product ideas. The main difference between an idea bank and other types of product idea management methods is the level of commitment. Accepting a proposal into an idea bank is, according to Itamar Gilad from Evidence Guided, “a weak form of commitment”. The team owning the idea bank will review the idea, without committing to anything else.

Impact Estimation Tables

Impact Estimation Tables (IET) are a tool for impact/effort scoring. They can be used to compare alternative design ideas, to analyse key risks, to estimate and to help with planning, prioritisation and tracking iterative delivery work.

Impact Mapping

Impact mapping is a lightweight visual and collaborative planning technique that helps to focus product delivery on outcomes and promotes the idea of optionality, with many different ways of achieving the same outcomes. By clearly stating value for the delivery organization and the impact for users/customers, this method helps to visualise the assumptions about the Value Exchange Loop and avoid Air Sandwich plans.

Kano Survey

The Kano Survey is a research tool for discovering customer expectations related to product features, according to the Kano Model. Kano surveys can be useful to help understand customer needs and preferences, choose between alternative options, uncover hidden customer expectations about basic product functions and potential exciters, and help with prioritising product development work. They can also help to prevent over-engineering by identifying unnecessary features, or improvements that would not contribute to customer satisfaction. At the same time, the Kano Survey is imprecise, and should not be used as the key factor for prioritization, but as a supplemental tool to evaluate assumptions and direct further research.

Levels of Evidence

The Levels of Evidence model (also called the Hierarchy of Evidence) ranks trustworthiness of different types of information. It was originally created for critically analysing medical research papers and therapy studies. The model is interesting from a product management perspective as a way to systematically apply a confidence score for prioritisation systems, such as ICE or Impact Estimation Tables.

Likert Scale

The Likert Scale is a common attitudinal survey format, requiring survey participants to select an option between polar choices (such as “Strongly agree” and “Strongly disagree”). The most common scales have 5 or 7 points, but others such as 4 or 3 point scales are also used.

Mathiness

Mathiness is a term for calculations and formulas that may look and feel like rigorous mathematics but lack true analytical rigor or validity, and often disregard logical coherence or factual accuracy.

MoSCoW prioritization

MoSCoW Prioritization is a fast and rough prioritization technique that was popular from the mid-1990s to the early 2000s in the context of time-boxed development (particularly iterative delivery), and is still in use in project work based on the DSDM process.

Net Promoter Score

The Net Promoter Score (NPS) is a popular but disputed and widely misused customer satisfaction index. This article explains the key aspects of running NPS research, its limitations and applicability, for teams that want to use this method in practice.

Novelty Effect

The Novelty Effect (also known as Novelty Bias) is a short-term psychological response to something new, unexpected or unfamiliar, causing a temporary change in behaviour. The Novelty Effect explains why shallow engagement gimmicks (for example gamification) look good in research but produce no significant long-term results or behaviour change. It can significantly skew the results of A/B tests, and may need special treatment to detect and resolve. From a more positive perspective, it provides a temporary window of opportunity for products to lead users to a first valuable experience in early engagement, as the boost in interest can cause increased tolerance of UX issues and a temporary willingness to spend more effort.

Now-Next-Later Roadmap

The Now-Next-Later Roadmap (NNL) provides an overview of product delivery in three time horizons: immediate, relatively soon and long-term. Each delivery item in the roadmap documents two key aspects: the problem to solve, and how solving that problem relates to an existing product objectives.

Objectives and Key Results

Objectives and Key Results (OKR) is a popular framework for setting goals to align teams and drive towards common outcomes. It’s based on a hierarchical breakdown of higher level goals (objectives) into component activities (key results), which become objectives on the next level of the hierarchy. In that way, company-wide objectives can be connected through several levels of metrics hierarchies to key results for a team or even an individual contributor.

Opportunity Solution Tree

An Opportunity Solution Tree is a visual representation of different ways a product can achieve a business outcome, showing various product opportunities, solutions and assumptions. By clearly visualising underlying assumptions and product opportunities, it helps to avoid Air Sandwich plans.

Overall Evaluation Criteria

The Overall Evaluation Criteria (also called the Overall Evaluation Criterion, or a Fitness Function) is a quantitative measure of the objective of a product experiment. An OEC can range from a single metric to a set of complementary metrics, and it can be specific to a single experiment, related to a set of experiments, or focused on longer-term aspects of product success.

Palchinsky Principles

In the book Adapt, Tim Harford lays out three key principles for adaptive planning, explaining them as the problem-solving approach of Peter Palchinsky. Palchinsky was a civil engineer in charge of several huge infrastructural projects in Imperial Russia and the Soviet Union, who in the early 20th century intuitively applied what would today be called iterative delivery, systems thinking, optionality and experimentation with feedback cycles. The three Palchinsky principles are important because they succinctly capture the essence of most modern successful product development approaches.

Patton's Customer Outcomes

Jeff Patton, in the article The Mindset That Kills Product Thinking, proposed a simple five stage progression of customer outcomes, particularly suitable to consumer-oriented software products:

Performance Path Map

A Performance Path Map is a sequence of key skills users need to acquire on the path from a beginner to an expert in a specific domain. The map is useful as a design aid and a prioritisation tool for product management. It is also useful a motivator for users, helping with customer retention and turning advanced users into active promoters.

Persona Spectrum

A Persona Spectrum is an extension of the classic product management concept of a user persona, allowing for a wider discussion about the contexts in which users take on the role of the persona.

Pioneers - Settlers - Town Planners

Pioneers - Settlers - Town Planners (PST) is an organizational model for aligning teams based on the maturity of the product or service they provide, instead of the usual organizational division by business units (such as finance, marketing, IT). Pioneers invent, Settlers turn inventions into products, Town Planners turn products into utilities for Pioneers.

Pirate Funnel

Pirate Metrics capture a typical customer lifecycle for consumer products in five stages: Acquisition, Activation, Retention, Referral and Revenue. The name of the method comes from the initial letters of the five stages (AARRR) and its resemblance of how stereotypical pirates talk in cartoons and comedy movies.

PreMortem Exercise

The PreMortem Exercise is a technique for discovering critical flaws in a plan, by pretending that a plan failed and then trying to perform a list the reasons why it could have failed, then performing a post-mortem analysis on those theoretical failure causes. It was developed as part of the Decision Making Games for the U.S. Marine Corps by Gary Klein and colleagues.

Product Opportunity Assessment

Product Opportunity Assessment is a structured questionnaire designed to identify good product opportunities, help with delivery focus and understand what would be required for a product to succeed in exploiting the opportunity. It can also be used to identify poor opportunities and help a product team avoid or park them, to prevent the delivery organization from wasting time.

Prospect Awareness Scale

The Prospect Awareness Scale is a categorisation tool helpful for identifying different types of content, on-boarding assistance and engagement for different types of users, based on their awareness of a problem, potential solutions, and specific products or services.

Purpose Alignment Model

The Purpose Alignment Model (also called Purpose Based Alignment Model) is a set of heuristics for handling different types of work, based on a combination of two factors: mission criticality and market differentiation.

QUPER model

QUPER (QUality PERformance) model is a method for defining release plans and targets for cross-functional, system-wide quality aspects of a system (such as usability, performance, scalability and similar…). It aids with prioritisation and roadmap planning.

REAN model

REAN is a customer journey model useful for defining product marketing funnels and metrics for user engagement analytics. The model consists of 4 stages (Reach, Engage, Activate and Nurture) and takes the name from the initial letters of each of the stages.

Real Options

Real Options methods help to structure iterative delivery staged investments and prioritisation based on the financial derivative option valuation techniques. A “real option” is the right, but not an obligation to take a certain action in the future (such as implementing a feature or committing to a milestone release date). Unlike financial derivatives based on which the method is named, real options are not traded or sold. Instead, the thinking behind derivatives trading with options is used to evaluate when to take a certain action, and if to take it at all at that point.

Research Engagement Platform

A Research Engagement Platform is a tool that helps teams reach out to potential research participants, speeding up the preparation for structured research activities such as surveys, interviews, focus groups, and user testing. Such tools typically alow teams to collect and manage lists of potential research participants, segment participants by interest, send out invitation e-mails or communicate to groups of potential respondents, and track responses and engagement to inform and facilitate future activities.

Single Ease Question

The Single Ease Question (SEQ), also known as the Task Ease Question, is a post-task attitudinal survey looking at the ease of completing a task. It’s particularly useful as a diagnostic measure, complementing larger post-scenario surveys. It’s simple to administer and score, and easy for participants to answer, so it can be a useful tool for comparing perceived usability before and after a design change, and for diagnosing UX problems in larger workflows.

Stakeholder Mapping

Stakeholder Mapping is the process of visualising the influence of various stakeholders on a product, in order to ensure that the needs of important stakeholders are considered for product planning, to manage and avoid risks arising from potential negative influence, and to gather the required political and organizational support for an initiative to succeed. Stakeholder analysis is an essential component of product development, complementing the insights gained from customer analysis, as it can help to ensure internal organizational support and help to identify various operational, financial, legal, and market-related risks.

Suck Zone

The Suck Zone is the period between the first exposure to a product and the point where a new user feels confident enough to perform basic tasks. It is useful for analysing retention and churn and designing strategies to help users become more competent and interested in continued use.

System Usability Scale

The System Usability Scale (SUS) is a post-test attitudinal survey measuring perceived usability. It provides a numeric score, so it can be used for benchmarking, relative comparisons of different systems, to evaluate the effects of changes to a system, or to track trends and changes over time and to detect unexpected impacts.

Technology Acceptance Model

The Technology Acceptance Model (TAM) is a 12-question survey useful for predicting technology product adoption and usage frequency. Slightly modified, it can also be used to predict Likelihood-to-Recommend scores.

Three Horizons Model

The Three Horizons Model places product bets and investments in three different categories based on how quickly they are expected to deliver a return on investment. It enables companies to plan and approach product development in different ways for different horizons, effectively allowing high risk bets to proceed in a controlled way but not stifle them based on standard operating procedures and expectations for mature products.

Tog's paradox

Tog’s Paradox (also known as The Complexity Paradox or Tog’s Complexity Paradox) is an observation that products aiming to simplify a task for users tend to inspire new, more complex tasks. It’s one of the key reasons for the symptom of requirements changing after delivery in enterprise software products, and for feature creep in consumer products. Tog’s Paradox also explains why it’s futile to try to completely nail down requirements for a software product, as the product itself will have an impact on the users, causing them to demand new functions.

Usability Metric for User Experience

The Usability Metric for User Experience (UMUX) is a standardised attitudinal survey, particularly useful for tracking the perceived ease of use and perceived usefulness of products when the physical space for survey questions is limited (such as inline forms on web sites and mobile applications).

User Learning Effects

User Learning Effects are changes to user behaviour and habits caused by interactions with a product. Users modify their behavior based on their experiences with a digital product or service, which can cause bias in A/B testing and influence long-term behaviours beyond the time frames for typical experiments. Considering user learning effects during A/B testing is important to prevent short-term gains that cause long-term losses, and to increase the relevance of shorter experiments. A/B tests need to account for both immediate user reactions and for how users adapt over time.

Value Exchange Loop

The Value Exchange Loop (or Value Exchange System) is a mental model clarifying how the two main types of product value co-evolve. Melissa Perri defined the model in the book Escaping The Build Trap, as a way to stimulate product management and delivery teams to gain closer understanding of users and customers.