Oct 8

Technical debt assumes that you have an existing system and you know already about the areas where the duct tape is becoming thin. Let’s discuss the problem, the conflict around technical debt a little. It’s basically always the same battle: the guys maintaining the system probably have good reasons why they want to pay back on the accumulated technical debt, whereas the product owner believes more functionality is more important.

This is a typical clash of interests between people from different tribes and usually from different departments: the developers report to a technical manager, the product owner to a business guy. The IT manager is typically under pressure to minimize costs, in particular costs for support and maintenance, so he’s interested in paying back on technical debt during projects. The business manager, however, is under pressure to convince new clients with new features. So, that there is a difference in how technical and non-technical people set priorities between technical debt and new features doesn’t come as a surprise: naturally, to the business side this is all about technical issues which they don’t consider to be their concern. It’s only natural that they believe that tackling such technical issues should be solved by the technical folks “somehow”. After all, it were these guys which build the system with all these problems in the first place, and now they want to spend more time and money on this? Although I’ve exaggerated here a little maybe and this line of argument is way too simplistic (given that often debt is accumulated over time), it’s still quite popular. Such product owners seem to believe that they are only responsible for developing new functionality. Corporate culture can contribute to this: some companies have a culture where every new glitter is taken for a star and gets way more attention than the cash cow functionality that keeps existing clients happy.

So some product owners will see technical debt as a separate issue which needs to go on a maintenance budget, a budget that somebody else is responsible for, e.g. the IT department. The problem with that idea is that in a culture where building new features is the only thing that counts, typically no one ever gets around to clean up the technical debt. All what will happen is that you’ll have some people that fix bugs with this maintenance budget. So you don’t actually ever spend the money on technical debt but only on the results of it which of course doesn’t address the root of the technical issues. Also, if you finally do get an approval for a refactoring project, these are really the most boring and horrible types of projects to work on. So it’s not likely to see a lot of happy, motivated people on such projects — and sure enough the most competent people are probably assigned to more valuable projects anyway. As a result and because there is no business pressure on such refactoring projects, as soon as something else comes up, such refactoring initiatives are abandoned or at least down-sized to a minimum. Peter Seibel describes in a recent, lovely written article the need for an Engineering Effectiveness team at Twitter in which he lays out in some details how hard it is to really put up with technical debt.

Let’s drill down on technical debt from the point of view of requirements. Wait a minute, this doesn’t sound right — how could technical debt be a requirement? Obviously, it isn’t, but there is a close relation. Technical debt can come in different flavors: e.g. you really should install a second machine and a load balancer so you’re prepared for failure or we really need to rewrite module foobar.clx to finally get rid of all the spaghetti code that’s slowing us down to oh, we really need to support responsive design, we’re getting more mobile users every day. Now, if you take a look at these three simple examples, they are related to quality requirements: the first is about availability, the second about maintainability and the last about usability. Take a look at the ISO 25000 standard for software product quality page listing all the quality attributes one might want to consider. Technical debt is always tied to some quality that your system should offer, but doesn’t.

A reason for this is that quality attributes are often not made explicit during requirements gathering, regardless of whether you’re following an agile approach or not. The less technical the product owner, the more often they seem to assume that performance, scalability, security and the other -ilities will come out just right by magic. Technicians, on the other hand, know perfectly well that they will not. It’s worse if they don’t that: they will build something that might fulfill the functional requirements but not the non-functional requirements. “Works on my machine” might be fine for a naive developer, but nobody will be happy if takes 5 minutes to load the page for the gazillions of parallel requests in production. There is a reason why software architecture is mostly concerned with quality, if you don’t plan and build for scalability, it’s unlikely you end up with a highly performing system. Take a look at the picture: this bridge wasn’t planned to cope with the amount of traffic it sees nowadays, some time in the 80s somebody decided it would be okay to have three lanes instead of two and didn’t think through the long term consequences.

Rheinbrücke A40 Duisburg-Neuenkamp
The bridge of the highway A40 over the Rhein in Duisburg. It is damaged so bad that trucks are no longer allowed to cross it.
Picture made by kaʁstn Disk/Catstitched by Daniel Schwen, licensed under CC BY-SA 3.0 de over Wikimedia Commons

Of course, technical debt can also accumulate over time, one might rightly point out: sure, one machine was enough for the requirements at the start, but nobody followed up on the increase in users and nobody cared (or had time) to clean up the code in module foobar.clx back then (nor in the following six years of quick bug fixes and minor one-off patches that have grown on it like leeches). This is a sign that nobody actively had an eye on how the world in and around the systems changes and to take action early on. For code quality, Uncle Bob points to the Boy Scout rule which says that you should always clean up (regardless of who messed it up) — another way of trying to ease maintainability by paying back on technical debt every day.

The over-aching point here is that you, as a technician and you, too, as a product owner need to think about the quality requirements of your system and to do that over the entire life cycle. What was a fitting solution at a time might turn into technical debt over time. This means means your system now doesn’t hold up to the quality requirements you and your clients needs and no matter what was the cause for it, you’re better of fixing it now than accumulate even more technical debt.

Isaac Sacolick describes in his article on How to get an agile product owner to pay for technical debt how to address the common problem of technical debt heads on, mostly from a managers perspective via process and by making people (mostly the technical lead) responsible. However, I think it makes a lot more sense to try to ensure a common understanding between the developers and the product owner. One way to go about this is to use quality scenarios: as a member of the dev team, ask your product owner how many users the system needs to handle. Ask her if she thinks it’s okay if somebody might find a security issue earlier than you. Or if she thinks it’s okay when a seemingly small change will take three weeks because the code is an intangible mess that nobody understands since Dieter left? These sort of questions hopefully open up to discussions based on business value, ROI and cost of delay. You are then using the language product owners understand and you might also learn something along the line (no we expect so many users to do X in the future, so investing in module Z doesn’t make sense).

Now, granted that doesn’t always work. An old colleague of mine always denounced the old system he worked on as being exactly the way it was ordered. Isaac’s article might offer some advice here, of which the most important one is probably that paying back technical debt is way easier if you have a CIO or someone with similar power supporting you.

Posted by Holger Schauer

Defined tags for this entry: , ,
Apr 15

Last week, I visited this years Goto Zürich 2013, which is a two-day conference with tutorials wrapped around it. Zurich Topic areas range from lean start-up over technology to a leaders track. I spent most of my first day on the so-called leaders track as here most talks revolved around adopting agile and lean methods, but also went to some more technical talks in other tracks, too. I’ll summarize most of the talks and my impression below. Btw., the slides to most of the talks can be found on the conference website, more precisely on the schedule overview for the two days.

The conference started out with a keynote by Scott Ambler on his work on Disciplined agile delivery which provides a framework on where decisions are needed when implementing agile methods in a larger setup, drawing from many different methods like Scrum, XP, Kanban, the Scaled Agile Framework and many others. The first part of his talk was mainly concerned with the question if agile holds up to its promise of delivering ‘better’, which he tries to answer with regular surveys. These seem to confirm that team size, location, complexity and methods used do have an impact. Not surprisingly, small team size are more successful than larger, co-located teams better than distributed ones and simple projects are much more likely to succeed than complex ones. And of course, there are still projects using waterfall-like approaches that succeed, while the difference between an iterative and an agile approach are rather minimal. Still, however, the number of project failures are always high, even for simple projects there are more failures than one might expect.

The first talk here was about Spinning by Ralph Westphal, which is well-known e.g. for his association with the Clean Code Developer initiative. His premise was that today’s business reality is a constant change of priority which is at conflict with assuming bigger time boxes during which developers can focus on some goal. His answer is that we should ‘seize the day’ (as Dan North might have put it) and deliver ‘value’ daily. It’s important that this value should be something worthwhile to the customer and the customer should be able to give feedback daily as well. I’m wondering whether it is really always possible to accomplish this, e.g. fixing a bug might require analysis well beyond a day. But even if you might not always hold up to the idea, it might still be a worthwhile guideline for organizing work. Another question which I don’t have an obvious answer for is the question who should be in charge to decide what should be worked on on any given day? Ralph required a thorough triage to be carried out to avoid wasting time, but it’s unclear whether this job belongs e.g. in the hands of the product owner.

Dominik Maximi spoke next about ‘Hostile waters’, e.g. how company culture might influence your chances and approaches to introducing agile in an organization. He made the important point that every company culture exists because it is (or was in the past) successful to work like that. This needs to be respected when you want to change something fundamentally. He then gave a nice overview of the Schneider model on how to classify company culture. Non-representative survey results indicate that ‘agile’ has a similar characteristics to ‘collaborative’ or ‘cultivation culture’, but doesn’t fit in so nicely with a ‘competence’ culture or ‘control’ (no surprises here). Changing the mindset of a company might take up to 7-10 years. Dominik finally discussed John P. Kotter’s work on change steps to implement agile.

Continue reading "Goto conference Zürich 2013"

Posted by Holger Schauer

Defined tags for this entry: , , , ,