diff --git a/content/blog-posts/2019-03-13/be-careful-what-you-wish-for-software-productivity-metrics.md b/content/blog-posts/2019-03-13/be-careful-what-you-wish-for-software-productivity-metrics.md new file mode 100644 index 00000000..ca9d0ecb --- /dev/null +++ b/content/blog-posts/2019-03-13/be-careful-what-you-wish-for-software-productivity-metrics.md @@ -0,0 +1,45 @@ +--- +title: "Simplistic heuristics for software development productivity are damaging" +date: "2019-03-13" +author: "Janis Lesinskis" +tags: + - productivity + - management + - software-engineering + - KPIs + - heuristics +contentType: "blog" +callToActionText: "Looking to improve your understanding of software projects and how this relates to managing smart software teams? Fill in the form below if you would like to discuss how we can help." +hideCallToAction: false +--- + +Recently I've been working on a few open source projects and a few commercial closed source projects. One very insightful comment that has stuck with me was made at Scram where they commented that "nobody ever got paid for removing lines of code" during a code QA phase where they were removing unnecessary code. Ultimately good design involves reducing duplication, the structuring of projects should be based around reducing the total cost of maintenance and one particularly good way to do this is to reduce duplication of code and remove all unused code. In the context of the discussion dead-code removal actually fixed a potential security flaw, *fewer lines of code, directly increased the value of the software being delivered*. + +The software methodology at Scram was explicitly encouraging the removal of duplicated and unnecessary code since this made their software both more secure and more performant. Their deeper understanding of software allowed them to have a better grasp on measuring productivity. Many of the issues with software management and incentive structures come from simplistic heuristics being applied to a complex topic. And sometimes incentives leads to [staff writing minivans](http://dilbert.com/strip/1995-11-13). + +Creating good incentives that are based on the process of creating software requires understanding the software development process. We are mostly structured around giving people equity in projects but even when people are directly rewarded for maximizing the returns on the value fot he software and their incentives align with that of the organization directly they still have to execute on that. And executing effectively on creating software still requires the relevant people to understand the software development process. So we take educating people on the software development process very seriously if there's a substantially software component to the work + +Casey Muratori writes about a powerful software development process called [compression oriented programming](https://mollyrocket.com/casey/stream_0019.html), the idea being that you write code somewhat naturally and then you create the structure when you find duplication. In this way you can quickly write code and use that code as the prototype stage for figuring out where the structure that will enable the reuse must be created. By investigating how the code is used you can find the best ways in which to reduce duplication without having to pay the price of writing that abstraction. By retrospectively analysing how code duplication occurs you are well placed to figure out the structure to deal with the duplication. An integral part of the process is the step of cleaning up the structure of the code as the information about the structure reveals itself to you. + +Writing good libraries is difficult and takes good judgment, you frequently make judgment calls about when to expend current effort to improve future usage and maintenance. At one extreme you get architectural astronauts where so much time is spent on architecting the structure that the actual implementation suffers. At the other extreme is where people give so little thought to architecture that they just copy and paste code around with no thought to reuse. The judgment call that a good software engineer makes is how much effort to place on the structuring of the code, this is not always an easy decision as it is a classic instance of the multiply-vs-addition productivity problem with a large number of non-linearities that must be accounted for. Thankfully there are a number of [Pareto-optimal](https://en.wikipedia.org/wiki/Pareto_efficiency) strategies when considering the value of structure vs effort required and many of these techniques are the core of good software engineering practice. + +At one place I worked at there was an entrenched culture of [copy-and-paste coding](https://en.wikipedia.org/wiki/Copy_and_paste_programming). When faced with some functionality that didn't quite meet the needs what people would often do is just copy the code and make some modifications locally. This of course was great for any given individual who's lines of code created metric just went up but was at a terrible cost to the company. The correct approach is to modify the upstream library code to support your needs, not to copy the internals of the library into your own code and make the modification there as this completely defeats many of the benefits of creating libraries in the first place. Libraries enable reuse of components, *the power of software libraries partly comes from the code that is **not** written*. In this particular example the end result was that many new lines of code were created via duplication but the maintenance cost blew out, in some instances there were more than 20 versions of a very similar piece of code that originated from a copy-and-paste. This approach works somewhat okay up until the point where it doesn't, at which point there are often catastrophic consequences. So how did we even know that there were 20+ versions? There was a bug in the original library code that people were copying instead of actually fixing in the upstream library. Given that a few months had elapsed from the library code being written and the bug being found there was a lot of divergence that had occurred in the places that the code was now being used. Instead of just fixing one function in one library that everyone was using the fix now involved searching through large numbers of code artifacts to find if their code was affected by this or not. This made it hard to track down, *just the time spent tracking down where the changes were made was far greater than the actual bug fix*. But it didn't stop there, with some people now relying on broken behavior this quickly because a nightmarish situation with many people wasting very many hours just because people didn't have the discipline to follow correct procedure and spend a few additional *seconds* writing their code to call a function from a library. + +How did this happen? + +Unfortunately this culture was entrenched before I came on board to fix up the internal libraries. I vividly remember having a conversation with my manager and he instructed me to not put some code I was writing up in a non-complete state because other employees would copy and paste it into their own projects even when told not to do so in the documentation. I remember talking to my friends who were working on high tech production lines at the time about this to which someone said incredulously "at my work if someone took something that was explicitly was marked as incomplete and placed it somewhere later in the production line they would get fired immediately". That people were repeatedly doing something they were told not to do that was damaging to project success, without any repercussions, honestly is a complete failure of management and engineering culture, if this is happening in your organization you really need to get to the bottom of how sure a culture has developed and figure out how to fix it before it gets entrenched. The cost of not doing this is two-fold, your ability to produce value from software is decreased (via increased bugs and decreased velocity) and, potentially even more importantly, such a culture encourages the more skilled engineers (and really anyone that has any self respect for their profession) to leave. Bruce F. Webster wrote a fantastic article about how the effects of software engineering culture impact the retention of talented employees in a blog post called [The dead sea effect](http://brucefwebster.com/2008/04/11/the-wetware-crisis-the-dead-sea-effect/) which describes how allowing a culture of bad engineering practice can create a very damaging downwards spiral of talent. Essentially the root cause of this behavior at this organization was a culture of cutting corners in the internal software quality in order to meet external deadline pressures. The causes of this pattern are very eloquently described in Repenning and Sturman's 2001 paper on process improvement ["Nobody ever gets credit for fixing problems that never happened"](http://jsterman.scripts.mit.edu/docs/Repenning-2001-NobodyEverGetsCredit.pdf). Writing lines of code was taken as a proxy for productivity and delivery of immediate projects was always prioritized beyond everything else (including the ability to keep delivering projects in the future), so time spent on improving the internal quality and removing roadblocks to future development velocity was seen as poor prioritization. This created the shortcuts loop described in the paper, in this particular case the shortcut being taken was the abandoning of correct software engineering procedure where the internal library code is fixed before proceeding. The cost of this is directly realized in the reduction in future capability that comes about in the future, good internal libraries greatly increase a software development team's ability to deliver higher internal quality **and** higher velocity. Some people think that you can trade off some internal quality for increased velocity but this turns out to be impossible as it's really just digging you in deeper into the shortcuts feedback loop. Martin Fowler explains why attempting to trade internal quality for speed is actually impossible and calls this misconception the [tradeable quality hypothesis](https://martinfowler.com/bliki/TradableQualityHypothesis.html). But technical details aside skilled software professionals strongly prefer working at a place that fixes bugs first before making new features, this is a item #5 on the famous [Joel test](https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-steps-to-better-code/) about software companies. The bottom line is that if you don't address those items the best talent won't put up with your projects. + +Generally speaking most people will agree that good software practices are the ones that maximize productivity (provided of course the work is worth doing, some work is not). The tricky thing is that productivity is very hard to pin down in a formalized manner as the appropriate parameters are very dependent on the industry that you are in and also the projects you are working on (most people don't even try though, even firms that claim to do mathematical process optimization frequently don't do this in house). One danger in creating simplistic heuristics for productivity in software development is that people can maximize the metrics without regard for productivity, or worse they game these in spite of productivity. If incentive structures are such that people are getting rewarded for simplistic metrics then you shouldn't be surprised if people do just that, but you foot the bill for all of the error that is in the gap between the heuristic and the reality that heuristic is trying to capture. + +*Making good productivity metrics is hard but gaming bad ones is easy*. The deck is stacked against the manager that tries to make simplistic software metrics. This is ultimately why things like agile exist. + +In any case raw lines of code written is usually a poor metric. If applied simplistically it tends to do the following: + +- Punishes people for using libraries and for working on libraries. +- Encourages people to re-invent the wheel, possibly by copying large amounts of code from elsewhere. +- Disincentives people from removing dead code. +- Encourages people to game the incentives structure by inflating the lines of code count. +- Distracts people from more important metrics of value generation. + +While this metric is obviously deficient the more important point though goes beyond any metric that tries to +If you think that someone is being unproductive and not *providing enough value in their code* then usually this is a people management issue and one that is best solved by other means, not simplistic things like a "lines of code written this week" metric.