Testing hypotheses – the only way to create knowledge
David Deutsch, an astrophysicist and evangelist of the multiverse theory, explores “How is knowledge created in the universe?” in his book The Beginning of Infinity. Whether the knowledge creator is evolution or a human, Deutsch concludes that all knowledge is generated by one process:
- Formulate a hypothesis
- Run an experiment
- Gather results, solicit peer feedback, analyse outcomes
- Reject, refine, or confirm the hypothesis and then repeat
When we build products, we’re creating knowledge about:
- Which customer segment has a real need
- What product features will meet that need and drive purchase
- How best to market and distribute the product
- Which business model and pricing will achieve the desired ROI
Every piece of product knowledge – big or small – comes from hypothesis testing.
"Borrowing" Knowledge vs Creating
Sometimes the fastest way to learn isn’t starting from scratch, but leveraging what others already know.
Imagine you’re launching a neobank (like Emerge.nz) in NZ.
Instead of building a full prototype, you study unit‑economics benchmarks: acquisition cost, conversion rates, operating expenses, average transaction size—of similar neobanks in Australia, U.K., and Germany. By aggregating those insights, you can sketch optimistic and pessimistic financial models for your own launch.
You didn’t generate this data yourself—it was created by other teams—but by collecting and adapting it, you gain valuable knowledge far more cheaply than by building and testing everything in-house.
The key metric for product teams: Hypotheses Tested per Time Unit
A key skill for product managers and founders is focusing on what matters most—and executing ruthlessly.
It’s easy to get lost in analytics, endless customer interviews, or feature bloat—hopping from meeting to meeting. Three months later, you’ve shipped a couple of features that neither drive revenue nor delight customers—instead, they burn cash through complexity and maintenance.
Remember: 9 out of 10 hypotheses fail. You can’t predict which idea will unlock your product’s core value. The only way to find out is to test more hypotheses faster.
Without a clear metric for hypothesis throughput—weekly, monthly, or quarterly—day‑to‑day distractions will derail you. Make “Number of hypotheses tested per quarter” a Key Result under your top Objective 😄
Knowledge Isn’t Complete Until It’s Shared
Suppose a PM tests different renewal‑reminder emails sent one week before subscription expiry. The version highlighting lost features drives significantly higher renewals. That’s a great insight into loss‑aversion.
Now the PM has two things to do:
- Scale it: Roll out the winning email to all customers and embed the approach across marketing channels.
- Evangelise it: Share the finding company‑wide so other teams—marketing, product, leadership—can apply the same principle, spot patterns, and refine strategy.
Creating knowledge through hypothesis testing – and ensuring it spreads – is the engine of lasting product success.