Estimation Sucks. Let's fix it.
The problem with estimation is that everyone is trying to get better at being right. Let's instead get better at being wrong.
There is little worse than being at the end of a project that’s gone well past its original deadline.
You feel stressed, your team feels stressed, and conversations with your stakeholders suck.
These feelings had me questioning my effectiveness and ability as an engineering leader, and I was tired of it.
In some conversations about estimation, folks will talk about adding buffers or finding ways to better capture the work, or confidence intervals and ranges. These are all in pursuit of finding the right answer.
The thing is, we’re always wrong.
By focusing on how we could be less wrong, we turned our remote-first team around from 1 release every 4-6 months to at least 1 release per month for close to 2 years in the middle of the pandemic.
Now you should use whatever mechanism works for you and your team/s, but I’ve found it hard to go back to ”traditional” estimation systems since.
Working small, and using empirical data to manage stakeholder expectations meant that:
The team celebrated frequently and felt a sense of momentum
The team were less stressed
Focussing on a smaller piece of work at a time meant fewer bugs in production
Our stakeholders were able to be confident in our release windows
Our conversations with our leadership group were able to be much richer
We were able to create tight feedback loops around our processes and how we refined the work to ensure we gradually reduced that deviation from the mean
We stopped caring about deadlines
We were able to pivot as needed
We had more opportunities to improve our system of work
And I didn’t need to play planning poker for the billionth time.
Besides being an amazing team (which to be clear is where most of the success should be attributed), there were 2 key ingredients, and 1 theorem that made this happen, namely:
Slicing
Cycle Time (The time taken for a card to move from in progress to done)
Little’s law
This isn’t a post about queuing theory but Little’s law gives me an equation I lived and died by. Effectively it tells us that we can determine the mean response time by taking the mean number of items in the system and diving that by the mean throughput of items in that system, practically, it means I put the # of stories left in, and I get my delivery date out.
Here’s what it looks like:
Min Days remaining = Number of Stories / (WIP / Cycle Time )
Using this, at any given point, I could understand where we were on our trajectory and could manage our stakeholders appropriately. You can take this away now, and apply it to your teams and be able to understand where your work might land, there’s just 1 annoying problem.
Deviation from the mean.
The equation above gives us the lower bound of our delivery range, if we take the standard deviation of the mean, we get our upper bound, the latest we’re likely to deliver this work. Its equation looks similar to the last
Max Days remaining = Number of Stories / (WIP / Cycle Time + STD Deviation )
Now, if your team has a lot of variation in their work sizing, you’re probably faced with an upper bound that’s very far away from your lower bound.
In a nutshell, this is why estimation sucks, too many variables, of infinite possible range.
So how do we solve this?
We slice.
Specifically, we focus on a single estimate. Is the work around 1-3 days? If it is, great, it can move into the ready-to-go backlog (assuming it’s otherwise refined). If it’s not, determine if we can slice it further, or just accept that we can’t eliminate all variation.
Through this acute focus on slicing our team consistently had 2 days of variation in their cycle time. This was completely acceptable to us, up to 2 days of variation on a story wasn’t worth solving for and it still gave us a small enough upper bound to work with.
By focussing our efforts on same sizing everything, we only had to estimate once and let the data do the rest.
When you make the work small, you ultimately make your estimates, less wrong, because you have constrained the maximum possible time spent on the work
Were we late sometimes? Sure. But when a train comes every 5 minutes, no one cares if it’s late. No one batted an eyelid if we were a day or two past our original date, often because we were able to communicate this well ahead of schedule using our metrics, affording us a greater amount of trust.
This slicing at a story level can be applied all the way up your value chain, at an epic, and initiative level, in fact, aggressively slicing out chunks of value from a large initiative was how we were able to start building a consistent stream of value.
There’s a lot more to this story, including how we approached refinement, team safety, and collaboration, but that will have to wait for another time.
In the meantime, as a teacher I know is fond of saying, “The best predictor of future behaviour, is past behaviour”, and that’s something I found to be very true in our approach to delivery, and very true in the usefulness of estimates in knowledge work.
If you’re keen on trying the approach with your team, I recommend starting with slicing, then tracking your story cycle times. Use this data to start improving your slicing, and your delivery.
P.S Tell me about one estimate success story you had in the comments!
Further information and inspiration
Slicing Heuristics - a fantastic talk from Neil Kilick which was where I started my journey
Is it done yet? - Michele Playfair talks about this (and the importance of story mapping) far better than I could ever hope to, it also contains some additional tools to help you be more predictable.
Demystifying Project Estimation - Nicola Ballotta and Jordan Cutler examine a number of approaches to estimation across projects, this article inspired a discussion between Tom Hill and myself on the value of slicing & cycle time in delivery
What I’m reading
The Embedded Entrepreneur - I’ve been really getting into the idea of a micro-saas, and this book by Arvid Kahl is an incredibly practical read on how to go about validating and building an audience.
The Hard thing about Hard things by Ben Horowitz was a phenomenal book on perseverance and decision making by a CEO, it was recommended to me by a friend, and I discussed one key take-away on LinkedIn
Elixir in Action - I’ve been enjoying getting back into Elixir of late, and I find Saša Juric does an amazing job breaking down the underlying why behind how Elixir wants you to program.