.png){kind=small}
One software team doubled productivity by improving architectural health
Software speed
The DoD relies on a portfolio of thousands of custom-built software applications developed over the past 50 years. Traditionally, the DoD has commissioned new codebases and until a system was “done” and then sustained them. Development was slow and costly, and became more-so over time.
In response, the DoD is now aggressively making moves to become faster by adopting modern DevOps processes and driving cloud migration – all to enable greater speed of capability delivery. Speed, as defined by the Agile & Lean process improvement communities, is measured in several ways including:
Fast cycle time: Short delays between dev and delivery
High velocity: More functionality developed per unit time
Low waste & rework: Few design flaws, overhead and risk
Reliability: Few defects impeding proper use in operation
Agile software architecture – speed with discipline
Agile processes are critical to an effort to become faster but will not work by themselves. Our team’s 15+ years of R&D at Harvard and MIT, and recent examination of ~100 DoD programs has shown that speed is only possible in codebases with healthy software architectures – those that are modular, hierarchical, with good APIs, controlled dependencies, and commonality. Unfortunately, Agile development processes do nothing to ensure continued architectural health as codebases grow - Some actually make codebases worse – and can therefore be self-limiting.
To understand why, consider the reason for DoD’s slowdown. Systems were often developed, grown, and sustained without investing in continuous refactoring or re-architecting over long periods of time, leading to significant health problems. Refactoring is often seen as a risk, and intentionally avoided and developer requests to improve systems have often been denied. Hidden costs associated with this degradation were not accounted for. Unfortunately, as complexity grows, speed of delivery, agility, cyber resilience, schedule predictability, quality, safety are harmed in ways that went unmeasured. Complexity and its costs eventually outweigh benefits of continued use or reuse, but this economic/risk tradeoff is not well managed. Eventually, refactoring may become infeasible and recapitalization becomes unavoidable. The decision to replace often made when costs & risks grow well beyond acceptable limits or the system cannot be adapted for a new platform (such as the Cloud.) Recapitalization open happens in a time of crisis and following a protracted period of pain and is typically made much later than would be economically rational.
CodeMRI® Discovery – Linking Architecture to Economics
Silverthread’s CodeMRI® Discovery combines software architecture and financial/risk insight to enable productive evidence-based conversations between executives, PMs, and technologists. These diagnostics provide multidimensional analysis and probabilistic modeling of software architecture, speed/economic risk, information to underpin lifecycle decisions, and granular, actionable information on technical debt and how to fix it. These assessments are based solely on the code and applied graph theory operating on the code; i.e., no engineering opinions and only code-based fact. This analysis has been done on ~100 DoD codebases, informing several multi-million-dollar rewrite/refactor/accept/reject decisions. DoD has used this to be a more informed consumer and provide better oversight.
Analysis of a complex DoD system
In one case, CodeMRI® Discovery was used to help leaders explore the history of a specific DoD system (“System A”), a legacy Java system. Diagnostics were generated on releases going back 10 years and identified significant architecture complexity growth over that time. The most complex version contained nearly 800 files trapped in a “Core” (part of a codebase where modularity is significantly degraded.)
Developers believed growing architecture problems corresponded with decreased productivity and increased fragility they had experienced. CodeMRI® corroborated this belief by providing deep technical insight into System A’s structural problems and using predictive analytics connecting them to speed, quality, waste, and risk problems that would continue if left unaddressed. System A’s project manager believed his development team’s existing challenges could be attributed to architecture problems.
CodeMRI® Modernize – Driving Healthy Architecture
In addition to assessment, the DoD needs the ability to fix challenged systems to improve speed, economics, and cyber risk – all benefits of better architected codebases. The DoD also needs the ability to keep codebases healthy during future development to prevent architecture degradation in the first place. After examining CodeMRI® Discovery results, System A’s team sponsored research to use Silverthread’s CodeMRI® Modernize product and test its ability to automatically generate refactoring plans, test whether System A’s developers could make practical use of those plans to measurably improve a real codebase, and test whether the predicted economic/risk gains would be achieved as a result of that refactoring. The goal of this work was to make modernization and continuous modernization an increasingly viable option, lessening the need for complete rewriting/recapitalizing.
Fixing the System A’s architecture
In late 2018, System A’s team spent 25% of their development labor on refactoring activities in parallel with normal sprint-based feature development. They used Silverthread’s technical health improvement plans to identify and attack high-value refactoring targets with the goal of re-imposing a modular structure. At first, engineers were skeptical of a tool’s ability to provide insight beyond what they already knew. Over time, they became increasingly confident and willing to use this approach. Developers iteratively selected some recommended changes to act on, fixed them, and regenerated new improvement plans to check results and explore what to do next. While the tool proposed a priority order for improvements, engineers were free to pick and choose what to do based on their own knowledge. Several fixes that developers believed would require very significant effort and risk were skipped. Those deemed low-risk were done first. Developers likened the experience to “untangling fishing line” with the benefit of an expert system to guide them. Some knots that were originally very difficult became much more feasible to attack once others around them are removed. CodeMRI® Modernize also caught and prevented new problems from being introduced that normally would have been missed and left to fester. Over 3 months, the team successfully eliminated all critical and emerging architectural issues in System A.
Twice as productive
Improving System A rapidly led to improved development speed, agility, productivity, lower risk. System A’s team reports that the investment in Technical Health improvement rapidly paid for itself in productivity and quality gains, with a strongly positive ROI. The development team says that it is now easier to incorporate new developers into the team. Learning curves had been shortened. The team was able to “throw out” a manual for rookies with tips, tricks, and warnings about side-effects that could be triggered in the previously fragile system. It was no longer needed. APIs now behaved as expected. System A’s team says it is now three times (3x) as productive as they used to be when working in System A. Features are developed and shipped twice as fast, with more predictability and fewer defects.
Improved System A’s Software Economics, Before & After Refactoring
Contact Us
Silverthread’s mission is to advance the state of software measurement practice by quantifying complexity and design quality. Our measurement know-how can establish a more trustworthy foundation for improving software economics.
Comments