The Great Productivity Debate

McKinsey just released a study on measuring developer productivity, and stepped into a minefield of protest from influential developers like Kent Beck or Joel Spolsky. The criticism universally was that McKinsey has read the literature on DORA and SPACE and other metrics but not actually gained any direct understanding — a doctor who’s read the books but hasn’t actually touched a patient.

Developer productivity is close to my heart. As a CIO, increasing output efficiency was a key goal of much of my transformation efforts — especially in environments where developers were not in plentiful supply to start with. I believe that there are significant improvements to be realised in developer productivity in real-world projects, which to me would mean reductions in team size for constant output, reduced turnarounds on feature requests, things like that. Productivity to me would mean ten people with better tools and processes outperforming a hundred. Productivity is output per unit of labour.

I’m not really going into the details of what McKinsey got right or wrong (a link to the debates below) but I do agree they got more wrong than right.

McKinsey backed their study with some tall claims of results seen across some 20 companies that used this:

20 to 30 percent reduction in customer-reported product defects
20 percent improvement in employee experience scores
60-percentage-point improvement in customer satisfaction ratings

I find these claims a bit incredible. The first one is about quality, not productivity — which are often inversely correlated. The second could have been achieved with free food and paid vacations. The third, similar to the first, is about quality rather than the productivity of a programmer (and anyway, to achieve this large an improvement you must have started with a real lemon in the first place).

My take is, however, that all this is somewhat beside the point. Somewhat focused on my experiences in corporates (mostly in India, but a few across the world) that I’ve seen and worked with, I propose that there are fundamental ways in which boring corporate IT setups differ from the exciting world of Kent Beck (Facebook) and Joel Spolsky (Stack Overflow).

McKinsey follows the same mindset that so many managers love — measure more, and results will improve. This is true only where no new tools and techniques are available. Let’s take an example — Google Maps vacuums up driving metrics by the zillion and if you’re planning to drive from Milan to Rome, can point you to the fastest route — bypassing congestion, avoiding detours, making you the most productive driver possible (if doing it in a Ferrari, employee experience will be wonderful too). However, if just getting from Milan to Rome is your goal, planes or trains are usually cheaper and faster.

The most critical improvement to productivity is better tools, not better measurement. Schumacher in a Ferrari could not reach Rome faster than a sixty-year-old on a train. Once the best tools are used, measurement will indicate some scope for further incremental improvement, which will be much smaller than the jump achieved by using better tools.

And the corporates I’ve worked with are far behind the state of the art in their adoption of modern tools.

Yes, some programmers are smarter and more hardworking while some are lazy and dumb. However, unless your hiring process is woefully erratic the average is, well, the average — moderately competent, moderately hardworking. Most companies do fairly mundane stuff — think SQL queries and Linux restarts rather than neural networks or optimising compilers— so genius is not going to be of much use anyway. You need legions of people competent enough to catch a train, not the one Schumacher with his Ferrari.

Some important productivity metrics in software today are DORA and SPACE. In particular, DORA has been around for a while and there’s a fair bit of evidence that it leads to better outcomes. Yet I can tell you for sure, none of the corporates I’ve worked with or seen up close will get much use from DORA, because the underlyings are missing. DORA assumes a complete and mature DevOps environment, and then tries to see which metrics indicate better or worse outcomes.

Take DORA’s “Change Lead Time” metric for instance — this measures how long it takes for code that is committed by a developer to get to production. In a mature DevOps environment, DORA (State of DevOps 2021, page 9) says that it should be a few hours and anything above 24 hours will knock you off the “elite” list. A typical corporate that I’ve worked with — 1 month or more. Often multiple months.

The reason is simple — most corporates don’t implement continuous deployment with automated functional testing. Long UAT cycles are far and away the most common reason for the long time period between code completion and code in production.

I chose this metric deliberately. It involves some big changes in the codechain but the productivity gains are immense. All these changes are mature and well-understood. No reason why any company cannot adopt them except resistance to change.

Other metrics have similar issues. DORA measures deployment frequency, (how often a team pushes changes to production) and says you should do it multiple times a day. Companies are lucky if their teams do it twice a month. Some of the others — like failure rate and time to restore for individual code commits are not even possible in the way traditional companies structure deployments — usually clubbing code from many branches into a single package so pinpointing failure and rolling individual code changes back is impossible.

The one argument teams give is that it is a big jump getting from here to there. It’s a silly argument; it is worth doing because it is a big jump. Also, it’s not Tom Cruise jumping without a parachute across a ravine. This is more like a tourist zipline carrying chubby homely Aunty Sheela to the other side; perfectly safe and well-trodden.

Productivity matters to a company. Even ignoring all the hype about every company being a software company, many spend 5–6% of their budgets on direct development costs and much more on the indirect costs of long project delays. Productivity improvement is at the heart of tackling this; yelling at the project manager is not an effective way to speed up your timeline. What happens in most companies is that they pretend they are highly productive like Facebook, allocate resources and timelines as if they were Facebook and then get upset when reality catches up and it takes ten times as long as promised.

So this is what I am recommending for productivity:

Automated Tests

I have heard innumerable arguments about the impossibility, the time, the cost, the this and that about how automated testing is just not possible for this project. The next one, for sure.

Development shops in many companies pull out every excuse in the book to avoid automated testing. Forget 100%, you will be in luck if 25% coverage happens through automated tests. The theory is great, just not in my backyard.

Automated testing is neither difficult nor expensive and repeatedly shows outstanding productivity benefits. The tools have advanced to the point where writing unit tests is easier than debugging, and faster than writing documentation and there are many strategies that deal with legacy code integration. If you’re not doing automated unit, integration and functional testing no amount of McKinsey billing will improve your productivity much. As a by-product, it improves quality too.

Frankly, if you do one thing to improve productivity, do this — automated unit, integration and functional testing with at least 80% coverage.

Proper DevOps

DevOps is the great multiplier to developer productivity in the present times, but companies routinely pay only the barest lip service to it. Yes, everyone has it but no it is not particularly useful. GitHub is a compliance tickmark, not a collaboration tool for code building. Automation is patchy. Ops is excluded entirely because … someone else’s department.

Be serious about DevOps. Both parts — Dev and Ops. It’s not rocket science (and frankly, India is pretty good at rocket science).

Internet Access

The Internet is the greatest resource developers have in the world, by far, but corporate development shops ensure that their developers are more protected from it than the Chinese from inconvenient political commentary. Most blogs are blocked, most discussion forums off limits and god help you if you want to explore a public GitHub code repository. Completely unnecessary, of course — most (yes, most) of these restrictions have nothing to do with security and everything to do with the fear that developers will “waste time”. We prefer lower-productivity developers who do not waste time.

Software development is not a repetitive exercise — every bit of code has new situations and new twists that require investigation. The reality is, if cut off from the internet, developers spend a huge amount of time figuring out things a 2-minute google search would have answered.

Communication

It’s the key, you’re told by important management gurus. Communications, that is. Slack, Discord and so on have changed the way developers work and communicate with themselves and their stakeholders. Good communication dramatically shortens cycle time and improves quality. Developers in most companies are therefore encouraged to communicate — just not to anyone useful. End users are out, other developers are out, infrastructure people are out, global forums are out, managers — only your vertical’s managers are in. And please don’t use risky stuff like Slack or Discord — just stick to email. Companies love emails zipping back and forth, sometimes with a hundred people on the address list. Microsoft Teams is common, but companies cleverly stay away from things like channels which are too slack-like; better to stick to productivity-killing voice and video calls.

Communication in development is meant to be horizontal but is invariably stuffed into vertical silos. The reason usually given is that “things will get out of control” — except that if the outcome is not great things are already not in control. Instead of imposing a governance structure that prioritises control, impose a communication structure that prioritises communications and outcomes — then figure out control. Don’t be the famous Yes-Minister hospital that was very well managed but without patients (who would only screw up the statistics).

Summary

Sure, the McKinsey study is a bit wrongheaded, but assuming any metrics no matter how good will solve your productivity problem is a bit like me buying a Garmin watch, syncing up the metrics, analysing with Strava and wondering why I’m not completing marathons. Things like waking up early, going for a run regularly … oops!

Development teams are like that. First get the basics right, ensure that teams have the right tools and properly adopting the tools, then worry about metrics. DORA SPACE EEE all these matter only on the edges, after the basics are in place. If your team is still using 1970s tools and techniques to create software, I can tell you (for free) that their productivity is also from the 1970s.

References

The references are all over the place, all related to developer productivity but not necessarily anything I directly reference in the text (because I’ve anyway tried to hyperlink references inline also). Will make for interesting reading on the general topic of developer productivity, though.