Startups require a constant balancing act between execution and learning
Some people, when they start to realize the power
of using data to inform their decisions, become obsessed with
optimization. I think this idea is particularly appealing to those of
us from an engineering background. By reducing the decisions we have to
make to a series of quantitative questions, we can avoid a lot of
real-life messiness.
Unfortunately, most decisions that confront startups lack a definitive right answer. Sometimes an early negative result from an experiment is a harbinger of doom for that product, and means it should be abandoned. Other times, it’s just an indicator that further iteration is needed. The only way to get good at these decisions is to practice making them, pay attention to what happens, compare it to what you thought would happen, and learn, learn, learn.
This has given rise to another school of thought, one that sees quantitative analysis, models, and anything involving spreadsheets as inherently anti-innovative and, therefore, anti-startup. But this is wrong, too. Spreadsheets, and predictive modeling in particular, have an important role to play in startups. It’s just very different than what it looks like in other contexts.
Let’s first take a look at what happens when spreadsheets go horribly wrong. For a change of pace, I’ll take an example from a startup inside a large enterprise. Imagine a general manager that has read The Innovator’s Dilemma and related books, and is therefore trying hard to help her organization make a transition to a new product category via disruptive innovation. She knows the internal politics are tricky, but she’s navigated them well. She has a separate team, with its own culture and office, and a mandate straight from top management to innovate without regard to the company’s historic products, channels, or supply chain. So far, so good.
Still, this manager is going to spend the company’s money, and needs to be held accountable. So somebody from the CFO’s organization prepares an ROI-justification spreadsheet for this new team. Because this is a new skunkworks-type project, everyone involved is savvy enough to understand that the initial ROI is likely to be low, much lower than projects that are powered by sustaining innovation. And so the spreadsheet is built with conservative assumptions, including a final revenue target.
Everything that’s happened so far seems reasonable. And yet we’re now headed for trouble. No matter how low we make the revenue projections for this new product, it’s extremely unlikely that they are achievable. That’s because the model is based on assumptions about customers that are totally unproven. If we already knew who the customer was, how they would behave, how much they would pay, and how to reach them, this wouldn’t be a disruptive innovation. When the project winds up getting canceled for failing to meet its ROI justification, it’s natural for the entrepreneur to feel like it was the CFO – and their innovation-sucking spreadsheet – that is the real cause.
And yet, it’s not really fair to ask that the company’s money be spent without anyone bothering to build a financial model that can be used to judge success. Certainly venture-backed startups don’t have this luxury – every business plan has a model in it. Just because entrepreneurs tend to forget about these models doesn’t mean their investors do. Companies that reliably fail to make their forecasted numbers are exceptionally prone to “management retooling.”
I think the problem with this approach is not the presence of the spreadsheet, but how it’s used. In a startup context, numbers like gross revenue are actually vanity metrics, not actionable metrics. It’s entirely possible for the startup to be a massive success without having large aggregate numbers, because the startup has succeeded in finding a passionate, but small, early adopter base that has tremendous per-customer behavior. Similarly, it’s easy to generate large aggregate numbers by simply falling back to non-disruptive or non-sustainable tactics (see Validated learning about customers for one example). And in a corporate context, a result in which the startup proves that a particular innovation is non-viable is actually very valuable learning.
The challenge is to find a way to use spreadsheets that can reward all of these positive outcomes, while still holding the team accountable if they fail to deliver. In other words, we want to use the spreadsheet to quantify our progress using the most important unit: validated learning about customers.
The solution is to change our focus from outputs to inputs. One way to conceive of our goal in an early-stage venture is to incrementally “fill in the blanks” for the business model that we think will one day power our startup. For example, say that your business model calls for a 4% conversion rate – as ours did initially at IMVU.
After a few months of early beta at IMVU, we discovered that our actual conversion rate was about 0.4%. That’s not too surprising, because our product was pretty bad in those days. But after a few more iterations, it became clear that improvements in the product were going to drive the conversion rate up – but probably not by a factor of 10. As the product got better, we could see the rate getting closer and closer to the mythical “one percent rule.” Even that early, it became clear that 4% was not an achievable goal. Luckily, we also discovered that certain other metrics, like LTV and CPA were much better than we initially projected. Running the revised business model with these new numbers was great news – we still had a shot at a viable business.
That’s hardly the end of the story, since there is still a long way to go between validating a business model in micro-scale and actually building a mainstream business. But proving your assumptions with early adopters is an essential first step. It provides a baseline against which you can start to assess your long-term assumptions. If it costs $0.10 to acquire an early adopter, how much should it cost to acquire a mainstream customer? $0.50? $1.00? Maybe. But $10.00? Unlikely.
Think back to the conflict between our Innovator’s Dilemma general manager and her nemesis, the CFO. The resolution I am suggesting is that they jointly conceive of their project as filling-in the missing parts of the spreadsheet, replacing assumptions and guesses with facts and informed hypotheses. As the model becomes clear, then – and only then – does it make sense to start trying to set milestones in terms of overall revenue. And as long as the startup is in learning and discovery mode – which means at least until the manager is ready to study Crossing the Chasm – these milestones will always have to be hybrids, with some validation components and some gross revenue components.
This model of joint accountability is at the heart of the lean startup, and is just as applicable to venture-backed, bootstrapped, and enterprise startups. As with most startup practices, it requires us to do a constant balancing act between execution and learning – both of which require tremendous discipline. The payoff is worth the effort.
(image source: virtualgalfriday.com)
