Jacob Kaplan-Moss

Managing Technical Debt

I did an “Ask Me Anything” session at TinySeed the other day, and one of the startup founders asked a question about managing technical debt. The company had built a successful product, had found product-market fit, but in doing so had incurred some tech debt. Their codebase was brittle, under-tested, and under-documented. The founder wasn’t sure how to balance moving forward and continuing to build new features with going back and paying down that tech debt.

This is a super common question – nearly every time I work with a founders something along these lines comes up. Startups – particularly ones that are bootstrapped or self-funded – don’t often have the luxury to produce tight, well-architected, carefully tested and documented code. And I think that’s fine! Traction is everything; small startups should be prioritizing landing customers and growing their product. A perfect codebase but no customers kills the company, but sustainable income pays for the time to clean up a crufty codebase. Startups are a place where the “debt” metaphor fits pretty perfectly: you’re incurring future work to get to market quicker. I’m all for it.

But once you find yourself with those customers and that creaky codebase … what next? How do you start to pay down that debt? Here’s my playbook for managing tech debt:

  1. Measure: there are no great measures for tech debt, but there are several serviceable ones. Pick one, be consistent, and track it over time.
  2. Agree on a tech debt allocation, a percentage of time Engineering will spend working on reducing tech debt.
  3. Monitor and adjust regularly.

For details, read on:

Measuring tech debt

There’s no broadly-accepted technique for measuring tech debt. This is probably because it’s a pretty loosely-defined term, meaning different things in different places. And whatever it means, it tends to be really hard to measure. But to manage tech debt you have to have some way of measuring it – even a poor measure is better than none.

A few options:

  1. Use your issue tracker. If your organization has good discipline around using an issue tracker (e.g. GitHub Issues, Jira, etc.) to track work, you can lean on that. Create a “tech debt” label, apply it to any tickets related to reducing or cleaning up tech debt, and use those tickets to measure debt. You can simply use raw ticket counts, or if you do any sort of estimation you can aggregate estimates. You can now track the rough amount of tech debt in your backlog, and monitor as it goes up or down over time.

  2. Use DORA metrics to measure tech debt indirectly. You probably should be using DORA metrics anyway to measure your engineering team’s performance, and if you’re not – why not start? They can’t measure tech debt directly, but they do measure the effect of tech debt on your engineering practices, so you can see the second-order effects.

    Tech debt is most strongly correlated with CFR: more tech debt usually leads to a higher failure rate for changes. Tech debt also tends to reduce DF and increase LT, and sometimes increases MTTR.

    Again, this won’t measure tech debt directly, but using these metrics as indirect measures of tech debt tends to work well enough to be useful.

  3. Ask your staff. The simplest method is to simply poll your engineers regularly and ask them how bad tech debt is in their estimation. The absolute numbers don’t mean much – “tech debt is at 7 out of 10” doesn’t tell you anything on its own – but tracking change over time can be enough information to make judgement calls about investment in tech debt.

However you measure, you have to be consistent. Pick a technique, commit to it, and follow that same technique over time. This playbook relies on a stable measure of tech debt that you track over time. If you change up how you measure, it won’t work.

Commit to a time budget for tech debt

Next, agree to a fraction of time your engineering team will spend on technical debt. This could be per sprint, per quarter, whatever – just a allocation of time to be spent decreasing tech debt. The key is that you need agreement across the organization – Product, Sales, Support, etc., anyone with an input into how Engineering spends its time. Everyone has to agree to allow Engineering to set aside this time for tech debt work. If you can’t get this agreement (and get people to stick to the agreement), this won’t work.

At first, the specific percentage will be something of a guess. You won’t yet know how much time you need to spend to make a dent. I usually recommend starting around 10-20% – that’s enough time to be meaningful and to see impact, but not so much that it makes it hard to get agreement.

I don’t recommend “tech debt sprints” – e.g. spending 2 weeks a quarter just working on tech debt, or similar. Reducing tech debt works better when it’s a long-term, ongoing, incremental commitment. And you’ll have less trouble keeping stakeholders happy if you continue to deliver product features alongside your tech debt work.

Measure and adjust over time

Now the pieces are in place: you’re measuring tech debt, and you’re reducing it over time. Next, every few months, you can sit down with your stakeholders and review your time commitment against your metrics. If your numbers aren’t improving as fast as you want: increase the time investment. If you’re making great progress, perhaps you can give some time back to other functions.

This’ll be a constant push-pull. As mentioned above, there aren’t fantastic ways of measuring tech debt, so there’ll be a bunch of noise in any measurement you use. Plus, it’s not like you’ll be able to completely avoid introducing new sources of tech debt even as you clean up older ones. And, you’ll likely also have deadlines for features, large customers that need specific things, and other inputs into your engineering prioritization. So you’ll make some progress, adjust the allocation downwards to meet a deadline, backslide for a while, increase the allocation, frustrate your Sales team at the slow progress on features, decrease the allocation, …

But as long as you’ve got the system in place to consistently measure your progress, a mechanism for setting aside some time, and the commitment from your stakeholders to review and adjust that carve-out regularly – you’ll make progress.