MCBL experiments with ARL’s Cross Reality COP to support distributed collaboration in virtual reality

By Mission Command Battle LabAugust 2, 2021

A U.S. Army Soldier conducts a quick halt to drop into the XR-Common Operating Picture (COP) virtual reality environment to conduct a terrain fly through before conducting a mission.
A U.S. Army Soldier conducts a quick halt to drop into the XR-Common Operating Picture (COP) virtual reality environment to conduct a terrain fly through before conducting a mission. (Photo Credit: Courtesy Photo) VIEW ORIGINAL

History is filled with examples of battles that have hinged on assembling commanders and staffs in a physical space to collaborate on the challenges presented to them. Whether it was Major General George Meade’s council at Gettysburg in 1862 to decide whether to fight or withdraw, General Dwight D. Eisenhower’s Verdun Conference in 1944 to counter the German offensive in the Ardennes, or General Colin Powell’s battlefield circulation leading up to Operation Desert Shield, face-to-face meetings are always the preferred method to make decisions. What happens when face to face collaboration is not possible? Can technology deliver the same in-person collaboration benefits when the threat, distance, and tempo of operations force dispersion?

One of the central themes in the newly published Army Futures Command Concept for Command and Control (AFC Pamphlet 71-20-9, 14 JUL 21) is that Multi-Domain Operations will require command posts (CPs) to become more survivable and maneuverable to counter enemy threats. Large, cumbersome CPs of the past will need to break into smaller elements to avoid detection and manage electromagnetic signatures.

Teleconferences and videoconferences are nothing new to Army commanders and their staffs, but those mediums have not fully replicated the in-person meetings that critical decisions and shared understanding demand. Thankfully, commercial technology continues to improve. Researchers at the U.S. Army Combat Capabilities Development Command (DEVCOM) Army Research Laboratory (ARL) are hard at work exploring new capabilities that might help Soldiers execute the operations process, even when operating from small, dispersed nodes.

U.S. Army COL Jason M. Railsback, Director of Command and General Staff School's  Department of Joint, Interagency, and Multinational Operations conducts familiarization with Army Research Lab's  Cross Reality-Common Operating Picture.
U.S. Army COL Jason M. Railsback, Director of Command and General Staff School's Department of Joint, Interagency, and Multinational Operations conducts familiarization with Army Research Lab's Cross Reality-Common Operating Picture. (Photo Credit: Courtesy Photo) VIEW ORIGINAL

As part of the Mission Command Battle Lab’s (MCBL) Future Command and Control Information Systems (FC2IS) effort, concept writers and researchers work together to look at new solution approaches to mitigate expected gaps in C2 for the Army’s Aimpoint Force. In order to tackle bite-sized components of the problem, the MCBL executed its first technical excursion with ARL’s Battlefield Information Systems Branch (BISB) from July 12-15. Researchers, analysts, and subject matter experts from across the United States converged in a virtual environment to assess the effectiveness of using ARL’s Cross Reality-Common Operating Picture (XR-COP) to support collaboration, shared understanding, and decision-making.

The military application of virtual reality has the potential to provide an additional means of communication and collaboration for dispersed command nodes.  Lt. Col. John Kallo, CP Branch Chief, Army Capability Manager Mission Command/Command Post, noted during the experiment, “The Army as an institution needs to become more comfortable operating in a dispersed environment more often than collocated.” Unlike current means of remote collaboration, VR allows for the representation of all users in a 3D shared space with the ability to engage with each other and the environment.

Operating from physically separate rooms, the participants utilized a limited-bandwidth cellular network to conduct a division-level back brief for a notional combined joint task force operation. The XR-COP system, integrated with an Oculus Quest 2 headset, allowed users to implement real-time audio and virtual gestures linked to Touch Controllers to convey messages and provide non-verbal context that would otherwise be lost in other remote means of communication.

During the Technical Excursion, participants interacted with maps, graphics,3D terrain models, and a live feed from WINTAK, a tactical mission command system.
During the Technical Excursion, participants interacted with maps, graphics,3D terrain models, and a live feed from WINTAK, a tactical mission command system. (Photo Credit: Courtesy Photo) VIEW ORIGINAL

“As participants collaborated in the 3D environment, it was exciting to watch them actively lean in to get a better view in the virtual space or digitally expand products to better understand the mission,” said Christine Phillips, MCBL analyst and primary observer during the event. This type of experimentation with basic and foundational research is a critical component of the FC2IS effort. By working with researchers early on in their projects, concept writers hope to validate Army concepts and better understand the timeline to deliver capabilities.

Researchers value this interaction since Soldier touchpoints provide crucial user feedback to researchers and inform future use cases and operational scenarios for further research before the projects go final. Dr. Addison Bohannon, branch chief of the BISB, reiterated the point. "The Technical Excursion allowed us to experiment with early-stage technology and Soldiers in an operationally relevant setting. Through this collaboration, DEVCOM ARL can better tailor our research and more quickly deliver innovative solutions that support Army Modernization.”

Users were very receptive, considering the XR-COP system is still early in development. In addition to the formal portion of the experiment, the MCBL provided the opportunity for over 30 additional users to conduct familiarization with XR-COP. An instructor with the Command and General Staff College was surprised when he jumped into the virtual environment stating, “I immediately saw the advantage [of the] capability when I put the headset on. Being able to tie into a Combined-Arms Rehearsal [CAR] and see what is being pointed to on the map is a clear improvement to sitting on the sidelines and only being able to hear what is being said and guessing at what is being pointed at on the map.”

A role player utilizes the Oculus Quest 2 virtual reality headset to interact within a virtual command post during the Mission Command Battle Lab’s Technical Excursion with Cross Reality-Common Operating Picture (XR-COP).
A role player utilizes the Oculus Quest 2 virtual reality headset to interact within a virtual command post during the Mission Command Battle Lab’s Technical Excursion with Cross Reality-Common Operating Picture (XR-COP). (Photo Credit: Courtesy Photo) VIEW ORIGINAL

Several participants highlighted the need to take notes while conducting the VR brief as the headset and controllers inhibited their ability to write without removing the headset. Future VR systems will need to incorporate technologies that enable users to input text, whether through speech-to-text, virtual keyboard, or other novel collaboration capability. It is those kinds of observations and user feedback that researchers require to better explore the problem space.

Most users agreed that augmented and virtual reality could significantly benefit commanders and staff during all phases of the operations process. With continued research and technical analysis, concept writers think it is likely that the future fighting force will be supplemented with some form of VR technologies that will enhance their collaborative abilities while supporting the dispersed nature of future C2.

Future research for XR-COP focuses on integrating external data feeds to provide an accurate, common operating picture. By ingesting data from other mission command systems, researchers and capability developers want to reduce the cognitive burden placed on commanders and staff by eliminating the need to conduct swivel chair operations between disparate systems. As developers improve the user interface and integration of XR-COP, the MCBL Science and Technology Branch will facilitate additional demonstrations and assessments.

As part of the FC2IS effort, the MCBL plans to conduct similar technical excursions to further explore and shape capability research and development leading up to the Command Post 2035 Tabletop Exercise that will take place in December 2021. Interested researchers and program managers can contact Lt. Col. Matthew Maness of the MCBL S&T branch for more information.