ABERDEEN PROVING GROUND, Md. (Nov. 4, 2015) -- Imagine taking a test and not learning the results until three or four months later. It's a long time to wait before knowing what you need to work on to improve.
But that's how the Soldiers responsible for cyber network defense at the Army's Network Integration Evaluations (NIEs) typically measured their performance -- through a report governed by formal programmatic test constraints.
Now, spurred by the rising prominence of cyber warfare and changes to the NIE construct, those Soldiers are teaming with mock hackers to get feedback in real time. It's one of several cybersecurity process improvements made possible by the Army's recent shift to holding one annual NIE and one annual Army Warfighting Assessment (AWA) -- a new event featuring Soldier-led evaluations of concepts and capabilities, but without formal system tests for record.
"Instead of saying, 'you didn't catch this, we'll let you know more in a report that comes out three to four months later,' we had our cyber network defenders sit down with the Red Team every three days," said Chief Warrant Officer 3 Greg Olivo, cyber information protection technician for the Brigade Modernization Command, known as the BMC. "It was open book both ways, so I think it was a great learning session for the cyber network defenders and for the Red Team as well. The beauty of this is they'll get a lot smarter -- they'll use the lessons learned during AWA to apply to an NIE."
The cyber Soldiers' interaction with the Red Team -- a group of trained hackers from the U.S. Army Threat Systems Management Office -- occurred throughout NIE 16.1, which took place at Fort Bliss, Texas, and White Sands Missile Range, N.M., in September and October and was the final proof-of-concept for the AWAs, which formally begin in the fall of 2016. More than 12,000 U.S. Soldiers and 1,140 coalition personnel took part in the exercise, which evaluated 78 different concepts and capabilities through realistic operational scenarios and a formidable opposing force (OPFOR).
"Cyber defense is my major concern -- we're facing a nation-state actor when it comes to cyber defense," said Col. Chuck Masaracchia, commander of 2nd Armored Brigade Combat Team, 1st Armored Division, the main operational unit that executes the NIEs and AWAs. "The greatest threat that I face as a brigade commander on the battlefield is not tanks, Bradleys, snipers or IEDs [improvised explosive devices], it's the threat to computer network operations."
AN EXPERIMENTAL ENVIRONMENT
Going forward, the Army plans to execute one NIE per year, focused on meeting integrated program of record test requirements, and one AWA per year, which will provide a more experimental environment to help shape requirements, with an emphasis on joint and coalition interoperability. The AWA will also allow the Army to improve its cyber posture by expanding training opportunities, developing system-of-systems level standard operating procedures and refining unit tactics, techniques and procedures.
"For an NIE the focus is correct -- the focus should be evaluating the system that the Army is thinking about purchasing," Olivo said. "But for AWA, we're working to strike a balance to make sure whatever system we're evaluating is set up correctly and doesn't have vulnerabilities, while at the same time training our cyber security team."
For past NIEs, the Threat Systems Management Office (TSMO) Red Team's mission was to provide each system under test a focused cyber-threat evaluation, which involved very little cooperative interaction with network defenders during the event, said Chip Wurslin and Robert Wedgeworth, the TSMO Cyber OPFOR co-leads for NIE and AWA.
But for NIE 16.1, as the AWA proof-of-concept, the Army devised and implemented a training plan that emphasized iterative and open communications between cyber OPFOR operators and network defenders. Prior to the exercise, the cyber Soldiers were informed of approximately 20 focus areas that would make up their cyber "report card" at the conclusion of the event -- part of a new evaluation rubric designed to quantitatively assess network defenders' ability to detect and mitigate simulated threats.
But they also had the opportunity to improve as they went along. For example, when the cyber network defenders learned they had failed to detect a certain type of malicious activity on the network, they asked the Red Team to deploy similar traffic within the next few days, to see if they could stop it once they were aware of that mode of threat.
"They were not only able to catch malicious traffic, but to change their thinking," Olivo said.
One defender even commented "that he had learned more about defending against Cyber OPFOR effects from this single event than during all previous NIEs combined," Wurslin said. "The evaluation results allowed defenders to focus on areas requiring more attention and helped to better prepare them for follow-on NIE events."
SECURITY AND SPEED
The Red Team collaboration was one of several changes the Army is implementing for NIE and AWA to improve Soldiers' and systems' security posture. While NIE for several years has been the Army's premier venue for Soldier-led operational evaluations of tactical communications systems, the process has traditionally focused on speed rather than security.
"The NIE is set up to get things working right away -- not to get them working securely right away," said Col. Bryan J. Stephens, Cyber Focal director for the Assistant Secretary of the Army (Acquisition, Logistics and Technology), System of Systems Engineering and Integration (SoSE&I) Directorate. "We are now putting the processes in place to do both."
For example, the Cyber Focal worked with Blue Team network defense personnel from the Army Research Laboratory and the 1st Information Operations Command on several steps to formalize and smooth the transition from lab-based risk reduction activities at Aberdeen Proving Ground, Md., to field operations at Fort Bliss. This synchronization ensured that systems' cyber vulnerabilities discovered in a lab setting could be mitigated prior to the start of NIE 16.1. Based on the results, SoSE&I is now implementing a tracking mechanism that will allow NIE and AWA participants to monitor Blue Team findings across past and current events, so the Army can better address consistent cyber trends.
Additionally, to boost security in the AWAs' coalition network environment, SoSE&I is extending by two weeks the Validation Exercise phase that takes place prior to the start of field operations. The extra time to verify that systems are properly configured and secured will reduce cyber risk for U.S. and partner nation units, while again providing better training opportunities for cyber Soldiers.
"Cybersecurity is a team sport, and if one team member is a weak link, the entire team suffers," Stephens said. "With these changes to NIE and AWA, the Army can truly work as a team to improve our collective cyber defense."
Related Links:
Cyberattack No. 1 threat, says brigade commander at NIE 16.1
Senior Department of Defense officials applaud NIE
Cyber warfare capabilities change modern battlefield
Army modernization makes strides as NIE 16.1 concludes
Office of the Assistant Secretary of the Army for Acquisition, Logistics and Technology
Army.mil: Science and Technology News
STAND-TO!: Network Integration Evaluation 16.1
Social Sharing