Video games inspire Army to embrace augmented, virtual reality

By U.S. Army CCDC Army Research Laboratory Public AffairsAugust 20, 2020

Army researchers run ParaView on HTC Vive to directly interact with their data within a virtual environment.
Army researchers run ParaView on HTC Vive to directly interact with their data within a virtual environment. (Photo Credit: Brian Simmonds) VIEW ORIGINAL

ADELPHI, Md. -- In the modern gaming community, virtual and augmented reality technologies deliver such stunning and immersive experiences that players can easily find themselves lost in a brand-new world in the comfort of their living room.

In much the same way, VR and AR provides advanced visualization capabilities to enhance the analysis and presentation of complex research data.

At the U.S. Army Combat Capabilities Development Command’s Army Research Laboratory, researchers developed workflows that make it easier for scientists and engineers to adopt VR and AR into their day-to-day operations.

“Virtual reality and augmented reality technologies offer tremendous potential as a data analytics tool for scientists and engineers to gain a better understanding of their research data during the exploration and discovery process,” said Dr. Simon Su, a computer scientist at the laboratory’s Department of Defense Supercomputing Resource Center. “The VR- and AR-enabled visualization workflows that we present are assisting researchers uncover various complex 3-D features in their domain-specific research area.”
Running Immersive ParaView on zSpace allows Army researchers to apply 2-D visualization techniques on 3-D data.
Running Immersive ParaView on zSpace allows Army researchers to apply 2-D visualization techniques on 3-D data. (Photo Credit: Brian Simmonds) VIEW ORIGINAL

According to Su, virtual and augmented reality technology progressed immensely over the past two decades thanks to the gaming industry.

During the early 2000s, most people knew very little about this technology, and the few who specialized in its development found it too expensive and inaccessible to implement in everyday life. Back then, an Army-sponsored VR system carried a hefty price tag of $250,000, and the head-mounted display alone cost more than $10,000.

In the 2010s, widespread interest in VR and AR for videogames transformed immersive technology into a lucrative business that investors poured billions of dollars into. Suddenly, VR and AR systems experienced leaps in advancements that not only improved their performance but also made them significantly cheaper.

“The advancement of virtual reality and augmented reality research has mainly been driven by the gaming community,” Su said. “Right now, a commercial VR system has better capabilities, higher resolutions and greater usability—and it’s only around $400. Even high-end versions in the current marketplace are ten-fold cheaper than they were two decades ago.”
Su and his colleagues synchronized the view of multiple Microsoft HoloLens devices so that Army researchers can collaborate and view the same virtual environment.
Su and his colleagues synchronized the view of multiple Microsoft HoloLens devices so that Army researchers can collaborate and view the same virtual environment. (Photo Credit: Brian Simmonds) VIEW ORIGINAL

Within the Army’s corporate research laboratory, Su leads the effort in adapting VR and AR technology to support Army researchers for data-intensive scientific and engineering applications.

Many researchers hesitate to implement AR and VR into their workflow because of the hurdles that come with operating unfamiliar yet necessary equipment. Challenges surrounding hardware and software setup as well as the need to troubleshoot confusing issues often discourage researchers from adopting VR and AR technology in their labs.

In order to lower these critical entry barriers, Su and his team prepare the technology in advance so that Army researchers can explore the benefits of VR and AR at their leisure.

“We approach Army scientists and offer to run their data on a visualization system for them so that they don’t have to worry about what software to use or how to load the data,” Su said. “We solve all that for them so that they can do their job. Their job is not to get the visualization system working but to analyze the data to do their science.”
The Army’s SyncVis application supports both 2-D and 3-D data visualization to provide a unified data exploration experience.
The Army’s SyncVis application supports both 2-D and 3-D data visualization to provide a unified data exploration experience. (Photo Credit: Brian Simmonds) VIEW ORIGINAL

Su and his colleagues created VR and AR applications to improve the usability of various open-source and commercially available technologies.

The team expanded the immersive capabilities of an open-source visualization software called ParaView and made it compatible with hardware platforms such as zSpace and HTC Vive. Researchers can sit down and apply 2-D visualization techniques on 3D data with zSpace or use handheld HTC Vive Controller devices to interact with a 3D- rendered view of their data within a virtual environment.

For those who wish to interact with a 3-D virtual scene but maintain face-to-face communication with other people, Su and his colleagues constructed a mixed reality system that synchronized the view of multiple Microsoft Hololens devices. This way, multiple users can observe the same 3D holograms and make eye contact with each other like in the real world.

The team also developed hybrid 2-D and 3-D visualization tools, such as SyncVis, to streamline the data exploration process.

“We are leveraging the same technologies developed for the gaming industry to support the analysis and exploration of the Army’s data,” Su said. “Instead of looking at large, complex 3-D data on a 2-D platform and making guesses, researchers can immerse themselves in a 3-D environment and interact with the data directly.”
Army researchers created a Unity application that can illustrate a 3-D simulation over time from an omniscient view.
Army researchers created a Unity application that can illustrate a 3-D simulation over time from an omniscient view. (Photo Credit: Brian Simmonds) VIEW ORIGINAL

With numerous AR and VR systems at their disposal, Su and his colleagues helped researchers across the laboratory examine complex 3-D data in new and exciting ways.

So far, the team used their VR- and AR-enabled workflows to illuminate Army research on computational fluids dynamics, network modeling and human sciences.

According to Su, VR and AR technology can illuminate an endless variety of research endeavors. They can visualize high-speed spray structures necessary for understanding the fuel-air mixture processes in aviation engines; they can simulate mission scenarios with holograms to study team behavior; they can help researchers uncover new facets of information hidden within complex airflow data and so much more.

“The 30 percent of scientists who use VR and AR for data analysis have such a technical advantage that the other 70 percent just cannot afford to not use it,” Su said. “If we can make the software and hardware set up as easy as turning on a laptop, then it may become just as universal one day.”

The team published their research paper, Virtual and Augmented Reality Applications to Support Data Analysis and Assessment of Science and Engineering, in the peer-reviewed journal Computing in Science & Engineering.

(Photo Credit: U.S. Army) VIEW ORIGINAL

CCDC Army Research Laboratory is an element of the U.S. Army Combat Capabilities Development Command. As the Army’s corporate research laboratory, ARL discovers, innovates and transitions science and technology to ensure dominant strategic land power. Through collaboration across the command’s core technical competencies, CCDC leads in the discovery, development and delivery of the technology-based capabilities required to make Soldiers more lethal to win the nation’s wars and come home safely. CCDC is a major subordinate command of the U.S. Army Futures Command.