Alex LauI've given a lot of absolutely terrible estimates for software over the years. (But let's be honest, can anyone really say they haven't?) One factor I've grown to appreciate is "The Trailblazing Tax."
The trailblazing tax is the idea that trying to trailblaze a new pattern in code is an order of magnitude more difficult than making a change that abides by an existing pattern.
Let's say you're working on an application and are prompted to estimate the time it takes to create a new report to track how many new users signed up. If there's already a reporting subsystem in place, there's a reasonable chance that all of the "new" logic you'll need to add for the request is copying some existing files and writing the database query itself. If you wanted to, you could find an existing report and list out the exact new files and layers of logic that would be needed for this new report.
However, let's suppose there's no reporting framework in place already. In this case, there is so much more to think about:
- What format should the reports be generated in: PDF, CSV, etc. Engineering-wise this begs the question of if you need to add a new library as a dependency to your project.
- Who has permissions to see these reports?
- How are users accessing reports? Are they being delivered via email or are they able to download them directly in the browser? Maybe this answer depends on how long the report takes to generate and the system has to have both.
The list goes on and on... the more novel this functionality is, the more questions will certainly arise as the developer embarking on the feature request will have. Additionally, there are the "unknown unknowns" of not knowing what roadblocks will be stumbled upon throughout this process.
Yes, this example compares having everything already in place to having nothing already in place. You may even be thinking to yourself, "of course writing a new subsystem from scratch is going to be harder." And you're right, that notion on its own isn't too surprising I suppose.
But where the danger lies is where we as developers find ourselves estimating "just one more thing" in terms of adding new functionality to a system. We often think things will be easy when there's one seemingly innocuous change.
Developers give happy path estimates because that's what has the most concrete information. They don't account for non-happy path factors because it's hard/impossible to predict what those are.
A happy path estimate doesn't account for a new package that has installation errors and is incompatible with other dependencies in the project until they've dug in and started writing the actual code. Maybe there's email logic in place that would let you send a report for small files, but once they hit a certain size a mail server would reject them.
I know when I'm trying to conjure these types of scenarios during an estimation meeting, I feel like I come across as overly-paranoid. And I can't say that I blame anyone for thinking that I'm paranoid when everything that I'm talking about is theoretical.
Spike Tickets
In my experience, the best solution to the problem of hard-to-estimate tickets is to prove out subtasks via "spike tickets" - i.e. tickets whose sole purpose is to have a timeboxed research task of proving out an unfamiliar concept.
Revisiting the new bits of functionality from the reporting example earlier, some spike tickets could cover:
- Generating reports in new formats like PDF and CSV
- Creating a basic permissions framework for who can see reports
- Sending out even one report over email and testing various file sizes with this email attachment
The beauty of this approach is that it minimizes unknowns for a task. Rather than biting off a giant chunk of functionality, spike tickets offer bite sized bits of research effort.
Wrapping Up
Unknown factors have a compounding effect on scope creep for a task. Have you ever seen a task blow up to 2 or 3 times its original estimate? I'd be willing to bet that there were multiple unknowns at play that made the task much harder than it initially seemed.
Software is always going to be hard to estimate, there's no getting around that. Trailblazing code is a necessary part of software development, but acknowledging and planning around it helps keep estimates in check before they get out of hand.
Do you have any strategies you use for handling hard-to-estimate code? Leave a comment below!
