Split-testing is great for linear optimization; making our landing pages, conversion rates, and retention metrics incrementally better day-in day-out. But it’s also amazing for testing big hypotheses, like what our customers really want to get out of our product. If you’re not doing both, you’re missing out.
Of all the tactics I have advocated as part of the lean startup, none has provoked as many extreme reactions as continuous deployment, a process that allows companies to release software in minutes instead of days, weeks, or months. My previous startup, IMVU, has used this process to deploy new code as often as an average of fifty times a day.
Over time, here’s my experience with what happens. People get used to the rhythm of five whys, and it becomes completely normal to make incremental investments. Most of the time, you invest in things that otherwise would have taken tons of meetings to decide to do.