All information pertaining to the Agile Solution Factory, including performance metrics, is available in real time using the Agile -- Integrated Data Environment, a key element of which is a program performance dashboa... VIEW ORIGINAL
Another key measure of success for the Agile Solution Factory is the release quality metric (RQM), which assesses the amount of defect-free story points delivered in a release under a service-level agreement (SL... VIEW ORIGINAL
A story-point analysis encompassing 13 RCAS quarterly releases over 43 months illustrates that the Agile Solution Factory achieved a 59 percent increase in productivity, or story points per release, along with... VIEW ORIGINAL
Reserve Component Automation Systems' Agile Solution Factory derives maximum benefit from software development methodology to modernize effectively, fast and securely.
The term agile in software development has become another information technology buzzword, like cloud. Many programs say they are doing Agile, but are they? What does it mean to be Agile enough to realize all the targeted benefits?
Agile is a software development methodology and culture designed to anticipate the need for flexibility and apply a degree of practicality to delivery of the finished product. It focuses on delivering discrete pieces or parts of the software rather than an all-or-nothing solution, in much the same way that software giants deliver updates continually rather than making customers wait for them to release a whole new version of their software suites. For instance, Apple releases multiple small updates that support security and functionality for its iPhone iOS operating system, rather than one or two updates per year. Agile allows for fast and efficient delivery of user capability, enabling access to what the user considers their "minimally viable solution."
For the Reserve Component Automation Systems (RCAS), which is part of the Program Executive Office for Enterprise Information Systems (PEO EIS), it wasn't enough to simply transition to Agile. Rather, the focus has been on optimizing the performance of Agile software development at an enterprise scale by applying a production factory mentality. In September 2013, RCAS made a headlong leap into Agile, executing a monumental transition from waterfall software development.
From this leap came the Agile Solution Factory, the operative word in the name being "factory"--a term that can be defined as "any place producing a uniform product without concern for individuality." In developing the concept of the Agile Solution Factory, consistency of approach remained a driving factor for all RCAS software development processes, tools and products. In a close partnership with the prime integrator, the Agile Solution Factory established itself as a case study of a large government program successfully transforming to Agile and then optimizing performance at an enterprise scale. For the Agile Solution Factory, that journey has never stopped.
The result? The Agile Solution Factory significantly improved the speed at which the user receives software that addresses their missions. As the software became markedly more extensible, modular and secure, RCAS saw customer satisfaction go up dramatically. The Agile Solution Factory's use of advanced automation frameworks like development, security and operations--using the method known as DevSecOps--drives efficiencies and performance gains. Since 2013, RCAS has increased software development productivity by almost 60 percent, reduced implementation costs by more than 50 percent and reduced time to market by more than 70 percent. Software quality has averaged 99 percent defect-free releases. RCAS measures productivity through an assessment of each product team's throughput--the amount of material passing through a system or process--which uses a metric known as story-pointing, discussed in detail below.
More than 80 government programs have participated in technical exchanges and tours of the Agile Solution Factory in Ashburn, Virginia. The purpose of these exchanges includes sharing lessons learned and best practices as they pertain to transforming, optimizing, scaling and holistically integrating Agile across a large government program. During one such tour with the Office of the Undersecretary of Defense for Acquisition, Technology and Logistics in April 2017, a representative commented that it was nice to see what "right" looked like in optimized Agile implementation.
The potential exists to replicate this process and success for other programs. As Robert Pirsig, the author of "Zen and the Art of Motorcycle Maintenance" and a philosopher, wrote: "If a factory is torn down, but the rationality which produced it is left standing, then that rationality will simply produce another factory." So, the question remains, how can the government learn from this example?
THE IMPORTANCE OF CULTURE
Moving away from a traditional software development life cycle approach toward more nimble methodologies such as Agile requires a clear vision and an unwavering commitment to continuous improvement. It requires an organization to evaluate processes, tools, performance metrics and expectations, internally and externally. Perhaps most importantly, it requires a complete culture shift.
Consider for a moment the breadth and scope of the RCAS program. It provides integrated web-based software solutions and support services that enable the U.S. Army Reserve and Army National Guard to manage mobilization, safety, personnel and force authorization activities more efficiently. RCAS provides a standardized, integrated solution that links approximately 10,500 Guard and Reserve units at roughly 4,000 sites in 54 U.S. states and territories.
More than 50 percent of the Army's force structure is in the Guard and Reserve component. To support such a broad swath of stakeholders and their associated requirements, RCAS must react quickly to change and make careful use of available resources. Experience has proven that traditional software methodologies like waterfall were not able to keep pace with a dynamic mission set and the need to adapt quickly to evolving capability gaps.
In 2013, RCAS leadership saw burgeoning demand from end users to receive high-quality software that addresses business value but focuses on decreased time to market. The waterfall model is a development process that flows sequentially through a predetermined series of phases, each requiring extensive documentation before the next can begin: requirements, design, implementation, verification and maintenance. Therefore, the customer must define and document the totality of requirements before the design phase.
This is in stark contrast to Agile, which focuses on prioritizing high-value requirements and prototyping solutions with a focus on speed to market. Agile involves the customer more regularly to provide guidance on the development process. Using waterfall, a contractor could spend months, if not years, in the requirements phase but still end up developing a product that doesn't satisfy the customer's expectations. Agile resolves this.
The product lead for RCAS reached out to industry, and the resulting feedback led to the decision that not only would an Agile framework be necessary, but there would need to be an "all in" approach to the transition. Once RCAS leadership made the decision to "go Agile," successful implementation began with open dialogue and trust between the government and the prime integrator, CACI International Inc.
MEASURES OF LONG-TERM SUCCESS
The Agile Solution Factory introduced many objective performance measurements and analytic practices to provide RCAS leadership with the tools to assess factory performance, including:
• Story-point analysis: A story point is an estimation metric that an agile development team (scrum team) uses to quickly assess the relative size, complexity and risk associated with a particular user story (or requirement). The higher the number of story points, the more difficult or complex the story will be to implement. In the story-point analysis covering 13 RCAS quarterly software releases over a 43-month period, the Agile Solution Factory measured a 59 percent increase in productivity (defined as story points per release), while reducing costs by 51 percent (cost per story point implemented). (See Figure 1, Page xx.) The Agile Solution Factory accomplished this despite a simultaneous staff reduction of 14 percent, thus supporting the driving need to deliver more with less.
• Release quality metric (RQM): The RCAS Agile Solution Factory measures software quality by assessing the amount of defect-free story points delivered in a release. For instance, if a development team completes 100 story points' worth of work, but one particular piece of functionality worth five points is shipped to the field with a defect, then the RQM reveals a 95 percent defect-free solution. RCAS delivered software releases that have averaged 99 percent defect-free functionality over a 46-month period. (See Figure 2, Page xx.)
• Agile Earned Value Management (EVM): EVM is a program management tool used to assess cost and schedule performance, as well as technical progress on projects. EVM provides a set of metrics and insight to support proactive decision-making at a fast pace. By integrating story-point analysis and EVM tools and techniques, Agile EVM delivers objective measures of cost and schedule performance dynamically for current and cumulative periods.
CYBERSECURITY LIFE CYCLE
Rather than waiting to find potential vulnerabilities right before deploying software to the field, the RCAS Agile Solution Factory uses cybersecurity checkpoints throughout the Agile software development life cycle. The factory leverages concepts such as threat modeling, code reviews, enterprise architecture depictions and integrated security tools such as HP Fortify for the end-to-end automation frameworks. For example, by combining HP Fortify with the continuous integration pipeline, developers are notified of potential threats automatically if code contains a known security vulnerability after it has been checked in. This simple validation process ensures that corrective actions are taken before the software release.
When HP Fortify was introduced within the Agile Solution Factory, the team identified all legacy vulnerabilities, added them to the backlog and prioritized them for remediation. As a result, the RCAS Agile Solution Factory gained the ability to continuously enhance system security and avoid the introduction of new vulnerabilities during development. Other automation security capabilities include automated Security Technical Implementation Guide (STIG) assessments within the DevSecOps pipeline to ensure that the baseline operating system, web service and database STIG settings comply with Defense Information Systems Agency requirements during the Agile development life cycle, not afterward.
In addition, this integrated security approach assists in implementing the Risk Management Framework for DOD Information Technology. Many controls within that framework require support for automated static code analysis, threat modeling, configuration management, high-level data flow diagraming and other requirements of the RCAS Agile Solution Factory software development life cycle.
COMPLETE TRANSPARENCY
The RCAS Agile Solution Factory uses an open book concept for full transparency with all stakeholders, continuously building on a foundation of trust. All information pertaining to the Agile Solution Factory, including performance metrics like story-point analysis and RQM, is available in real-time using the Agile -- Integrated Data Environment. (See Figure 3, Page xx.) This customizable portal houses all artifacts created in support of the mission and includes dashboards, automated workflows and tools used by all team members, the government and the contractor. Culturally, this transparency improves decision-making for RCAS and enables an environment that cultivates, expects and rewards open discussions on current status and performance--good or bad--from multiple perspectives.
CONCLUSION
The Agile Solution Factory looks at the creation of software from a holistic point of view. It is not enough, for instance, to simply construct software without also paying careful attention to the repeatability of the process.
As such, successful implementation of an Agile Solution Factory must account for optimizing the efficiency of delivering and installing applications through automation. It must incorporate a level of predictability that mimics what one might expect from any other "factory." It must establish an unabating trust between government and contractor and offer full and complete transparency into the process. In all of this, it also must take note of the inherently evolutionary nature of an iterative software development process.
The promise of Agile is achievable with the right vision, culture and partnership. Today the Agile Solution Factory hosts four different government program application portfolios. The RCAS Agile Solution Factory is constantly evolving and adapting to change across multiple perspectives, objectively measuring and optimizing performance of software and product development, and realizing the benefits of a government and contractor partnership built on trust through total transparency. It faces the challenge of adapting and evolving at ever-increasing speeds as it continues to optimize performance across areas such as productivity, cost efficiency, enhanced cyber posture, product quality and velocity of delivery.
While every government program transitioning to Agile will address unique challenges, the blueprint for transformation can leverage many of the lessons learned, best practices and methodologies of the RCAS Agile Solution Factory and other DOD programs that have transitioned successfully to Agile. These transformation blueprints, in turn, are transferable and repeatable.
RCAS is interested in sharing lessons learned and best practices in transforming to Agile at an enterprise scale and optimizing performance for large government programs. For more information or to tour the RCAS Agile Solution Factory, contact Steve O'Brian at 703-806-3303 or stephen.p.obrian.civ@mail.mil; or Angela Green-Mack at 703-806-3089 or angela.d.green-mack.civ@mail.mil.
MR. STEPHEN P. O'BRIAN is the deputy product lead for Force Management and Readiness Systems at PEO EIS, Fort Belvoir, Virginia. He is a former Air Force officer with more than 20 years of acquisition experience serving in numerous major acquisition programs and with the assistant secretary of the Air Force for acquisition. He holds an M.S. in management from Troy State University and a master of military art and science from the Air Command and Staff College. He is a graduate of the Defense Acquisition University Program Manager's Course and is Level III certified in program management.
MS. ANGELA D. GREEN-MACK is chief of RCAS' Technical Management Division and its chief technical officer. She holds an M.S. in information management from The George Washington University and a B.S. in computer information systems from Alabama State University. She is Level III certified in both program management and information technology.
This article will be published in the April -- June 2018 Army AL&T magazine.
Social Sharing