The U.S. Army Research Laboratory's Human Research and Engineering Directorate has provided manpower and personnel integration (MANPRINT) analytical support to the Network Integration Evaluation since its inception in May 2011.
The NIE is a series of semi-annual, Soldier-led evaluations designed to further integrate, mature and rapidly progress the Army's tactical network. During the evaluations, three Army organizations --the Brigade Modernization Command, the Army Test and Evaluation Command and the System of Systems Engineering and Integration Directorate -- integrate networked and non-networked capabilities with a full brigade combat team to assess capabilities in an operational environment and to determine their implications across DOD's doctrine, organization, training, materiel, leadership and education, personnel and facilities, or DOTMLPF.
MANPRINT is the U.S. Army's Human Systems Integration Program, which falls under the auspices of the Office of the Deputy Chief of Staff, G-1. The G-1 MANPRINT Office establishes policies and procedures for implementation in the system acquisition process in accordance with appropriate Army regulations and exercises primary staff responsibilities for Soldier-oriented research and development in personnel performance and training. MANPRINT focuses design attention on seven domains including: manpower, personnel, training, human factors engineering, health hazards, system safety, and Soldier survivability. ARL is the lead for the first four domains listed for acquisition program categories I and II, and is also the lead for portions of category III programs.
The evaluation was conducted from October through December 2013 on the ranges at Fort Bliss, Texas. ARL researchers participated in three capacities. First, personnel supported ASA(ALT) [Office of Assistant Secretary of the Army for Acquisition, Logistics, and Technology] during preliminary system of systems evaluations at the C4ISR System Integration Laboratory at Aberdeen Proving Ground, and again later during NIE field activities. Secondly, personnel supported ATEC in MANPRINT evaluations of individual systems under test, systems under evaluation, and the Army's capability set fielding. This level of MANPRINT support was of the type usually provided during formal test and evaluation for programs of record. And lastly, personnel from the Fort Bliss field element supported the Brigade Modernization Command. The focus of this third level of support was MANPRINT issues associated with systems-of -systems used within an organizational context in an operational environment. The overall impact of new equipment and systems of systems on the receiving organization also were addressed.
Pam Savage-Knepshield, chief of the Human Factors Integration Division at HRED, is no stranger to NIE. She was boots-on-the-ground collecting data during a previous evaluation. She explains that, "Not only is system integration critical to mission success, so is effective human-system interaction. Collecting and analyzing data to understand the effectiveness of human-system interaction and organizational and human performance is a core competency of our MANPRINT practitioners. Using a variety of data collection methodologies (e.g., observation, interviews, and surveys), our personnel are uniquely qualified to work with Soldiers to identify issues and risks as well as their root causes, and then later work with developers to effectively mitigate them within programmatic constraints."
Early, Pre-Test MANPRINT Risk Assessment
More recently, HRED has expanded its support to earlier phases of the agile process through strategic partnership with ASA(ALT). Here, HRED identifies potential significant MANPRINT risks for tracking throughout NIE through early user interface and integration risk assessment. MANPRINT practitioners review, identify and forecast risks that potentially impact affordability, system performance and maintainability in areas of manpower, training, personnel skills and human factors engineering (specifically system design). Each MANPRINT domain is studied based on the intersection of human-to-system given operational architecture views, operational and supportability requirements (i.e., tools, software, interfaces) and the missions expected to be accomplished.
"In each phase, MANPRINT practitioners work with system engineers, requirements analysts, and logisticians through a formal product supportability integrated product/process team along with the Army Materiel Command sustainment community to characterize the potential burdens of supportability, trainability, and cost of industry and government technical solutions prior to NIE," said Jeffrey Thomas, an HRED research psychologist, at Aberdeen Proving Ground, Md.
System-Level MANPRINT Risk Assessment during NIE
MANPRINT practitioners partner with the ATEC's Army Evaluation Center to evaluate the extent to which systems have effectively addressed MANPRINT considerations through system design as they are used to execute missions by Soldiers of the 2nd Brigade, 1st Armored Division.
"Through this partnership, MANPRINT practitioners integrate with ATEC/AEC evaluation teams during test planning, data collection and analysis activities to report findings on systems under test, systems under evaluation and selected Department of Army capability demonstrations and excursions," said Thomas. "The findings are reported within the operational suitability section of acquisition-sensitive ATEC capabilities and limitations reports and operational-test-agency evaluation/assessment reports. These reports provide system vendors with the critical data needed to improve their systems so they can better meet mission needs in austere environments in which our Soldiers work."
The team spent time in the field during NIE events documenting MANPRINT and integrated logistics support issues and identifying recommendations for manpower levels, personnel attributes, HFE, health hazards, system safety and maintenance attributes.
"This in-field observation and analysis has greatly benefited the Soldier, as system suitability improvements have already been made based upon recommendations from the team," said Jock Grynovicki, HRED. "The team's direct interaction with Soldiers during NIE events and training courses is imperative for understanding the pros and cons of the systems that will be used in operational theater."
The MANPRINT evaluation team contributed to the NIE event for systems under test as well by providing input regarding the fielding decision for a system. Soldier training that is provided to Soldiers prior to the events for these tests has been one area of focus for the team.
"As a result of evaluating new equipment training through classroom observance, recommendations and enhancements for future training packages have been made to better support the Soldier," said Grynovicki.
In addition to providing MANPRINT support to ATEC, HRED provides all HSI/MANPRINT support to Joint Program and Army Program Managers for programs throughout their entire acquisition lifecycle working to identify and mitigate risks well before systems undergo formal test and evaluation.
For example, John Reinhart from HRED's field element in Fort Belvoir provides MANPRINT support to the Program Manager for Soldier Warrior for the Nett Warrior program. Nett Warrior is an integrated dismounted situational awareness and mission command system for use during combat operations. The system provides leaders position location information of friendly forces within the area of operation. The map display on the end user device (a smart phone) can be used for land navigation and to mark friendly and enemy assets. It provides both digital and voice communication capability.
"During the event, I was able to have access to Soldiers in the field, to observe and discuss their experiences using Nett Warrior -- and identifying which system characteristics satisfy or need to be improved as it relates to mission accomplishment. I also participated in the administration of user surveys on Nett Warrior to collect formal survey data from each of the companies that used it," said Reinhart.
Reinhart indicated the user feedback on the Nett Warrior system he obtained and reported informed the product manager about what was useful to Soldiers and what areas needed improvement. He also said that the PM used that information to make improvements to the graphical user interface, system configuration, and basis of issue.
Dr. Christian Carstens, a research psychologist with HRED's field element at Fort Benning, has provided support as a MANPRINT analyst for Nett Warrior at NIE. Carstens said the Army conducted a limited user test of Nett Warrior. The systems were issued to two platoons of Stryker-mounted Soldiers. After completing the training, the Soldiers engaged in an extended series of day and night tactical exercises, each designed to evaluate the efficacy of the Nett Warrior systems. The test team collected both instrumented data (e.g., rate of message completion) and feedback data from the Soldiers.
"Based on what we learned from this and previous tests, the test team was able to make recommendations for improving the system, such as: upgrades to the operating software, improvements to the smart phone for a better visual display at night, reducing the glare on the visual display during daytime use, modifying the location of the smart phone to minimize interference with movement, and improving the incoming message alert system," said Carstens.
Jeffrey Everette, from HRED's field office at Fort Bragg, also provides MANPRINT evaluator support to the AEC. Everette indicated that WIN-T Inc 2 was assessed primarily by data collected from questionnaires administered to the Soldiers, observations made by the Brigade Modernization Command system lead, and AEC evaluators and interviews with event participants.
Another area where ARL supports the process is the MANPRINT evaluation of systems, where MANPRINT analysts familiarize themselves with the systems participating in an event, either at the C4ISR System Integration Laboratory or during pre-NIE training or communications exercises. Then, MANPRINT analysts travel to the various units using each SUE to observe it in use and interview Soldier operators and maintainers. This enables HRED to provide first-hand assessment data directly from Soldier observations on training, usability, and workload for insertion into the capabilities and limitations report that the Army provides for each evaluated system.
"MANPRINT evaluations of [systems] in the NIE environment provide an opportunity for usability analysis extending beyond system MANPRINT assessments," said Richard Tauson, HRED researcher. "First, since the SUEs are not programs of record, there may not be any formally defined requirements. In many cases, the benefits and deficiencies that become apparent in a SUE may lead to identifying requirements that neither the Army nor the vendors have predicted."
For example, many new and legacy systems provide situational displays, but the need for a common, sharable database that these displays could use to exchange information only became apparent when maps and reports could not be transported from system to system.
"Much of our human factors engineering support to program managers is done at the system level; NIE gives us a rare opportunity to take a look at the bigger picture -- when systems-of-systems are in play during a mission -- this is when we can identify issues such as these," said Savage-Knepshield.
Tauson added that in some cases individual systems may appear to be acceptable to the unit, but the cumulative effect of multiple new systems may create problems. He said at the company tactical operations center, the signal support -- maintainer could support the legacy command and control equipment and may have been able to support any given additional SUE. "However, we found that the combined knowledge, skills, and abilities required to support all of the SUEs exceeded what any single Soldier could be reasonably expected to achieve," said Tauson.
Organizational Level MANPRINT Analysis
Another focus at the most recent NIE was at the organizational level during which cognitive load issues associated with mission command were analyzed. Cognitive load refers to the aggregate mental load placed on commanders, staff members, and other users by an increasingly complex mission command work setting.
John Hawley, engineering psychologist from HRED's Fort Bliss Field Element, said the primary objective was to better characterize the nature of cognitive load within the contemporary mission command environment by addressing questions such as: Do commanders, key staff members, or other mission command operators perceive cognitive load to be a problem? What is the underlying nature of the problem? What aspects of contemporary mission command appear to be driving cognitive load? And, going forward, what are some potential solutions to the problem of growing cognitive load on commanders and key staff members?
"Our work is important because effective mission command is a major contributor to successful military operations. Excessive cognitive load on command and staff personnel has the potential to detract from effective mission command performance," Hawley explained. "A recent case study of network-enabled military operations cautioned, for example, that if Soldiers and commanders 'are not adequately trained on network-enabled [mission command] systems and are not proficient in their use in stressful battlefield conditions, then these [network-enabled] systems can be a hindrance rather than a help in combat.'"
Hawley said that based on observations, interviews and database entries, it was determined that cognitive load is an issue for some levels of command and some staff members, particularly during high operational tempo events. The term most often used by NIE participants to describe this overload is 'information overload.'
"We identified the most impacted command echelon and the general sources of cognitive load at those levels. Broad categories of solutions to extraneous load were also identified. The mission command role is intrinsically complex, particularly at the company-battery-troop level. But we do a lot of things (or neglect a lot of things) that add extraneous complexity to this already demanding role. As NIE 14.1 winds down and final reports are being written, our practitioners are already ramping up to support NIE 14.2 and 15.1."
Social Sharing