Touch & go: CERDEC and Microsoft share research for multi-touch technology

By EDRIC THOMPSON, CERDECJuly 23, 2009

Title
Captain Ramon W. Almodovar, commander, Company A, EXFOR, First Battalion, 29th Infantry Regiment, Fort Benning, Ga., uses a COMET prototype to plan a mission during C4ISR OTM E08. The COMET team is participating in E09 as well to evaluate the technol... (Photo Credit: U.S. Army) VIEW ORIGINAL

The U.S. Army Communications-Electronics Research, Development and Engineering Center recently signed a Cooperative Research and Development Agreement, or CRADA, with Microsoft to share research in support of developing multi-touch technology for the Warfighter.

Although the U.S. government represents Microsoft's single biggest customer, the CRADA with the CERDEC Command and Control Directorate represents only the second joint research project Microsoft has throughout the federal government and the Department of Defense.

"We believe this to be the most important CRADA that we have ever executed with DoD," said Brian Roach, general manager of U.S. Federal and Department of Defense for Microsoft. "The uniqueness of this agreement is that it was conceived and driven from the grass-roots level, and it will empower Army and Microsoft's best innovators to do great things unachievable without the partnership."

"Not only does this partnership help us to quickly adopt or adapt new commercial technologies to meet Army needs, it also helps us to be in front of the implementation edge by better understanding what is coming over the commercial sector horizon," said Dr. Gerardo J. Melendez, director of C2D.

"I see this document, not only in terms of the work that we are doing there in terms of surface technology, but adding to the padding which we've been going toward, expanding to information-knowledge management," Melendez said.

The multi-touch portion of the CRADA's tasks will be executed by the Command and Control Multi-touch Enabled Technology team. COMET is an effort to research the applicability of multi-touch technologies to command and control systems, said Cyndi Carpenter, systems engineer.

"We became interested in researching the problems associated with developing a face-to-face, digitally-assisted analog collaboration tool," Carpenter said. "In essence, we were looking to explore what an 'electronic sand table' might look like or how we could build electronic maps that would behave like normal maps, but have special capabilities."

Traditional collaboration is implemented with paper maps, grease pens, acetate layers and sand tables. While these tools are intuitive and can work exceptionally well to get a point across, they are ineffective at recording, saving and transmitting information and are incapable of providing automated assistance or analysis, said Nicholas Palmer, COMET technical lead.

"In the COMET vision, commanders and planners would gather around a multi-touch surface and interact with computer simulations of familiar analog planning tools, such as maps and acetate overlays, by touch: no keyboard or mouse would be necessary," Palmer said.

However, the electronic maps can respond in ways that paper cannot, Palmer said. For example, commanders could zoom in for additional detail, or they could change from raster [closely spaced rows of dots] to vector maps. Moreover, since the environment is multi-user, multiple staff positions could simultaneously contribute to group activities such as war gaming, rehearsal or after-action reviews.

"Because multi-touch technology lowers the barriers between the computer and its human users, these systems are approachable, easy to use and powerful," Carpenter said.

COMET is part of the overarching Collaborative Battlespace Reasoning and Awareness Army Technology Objective between CERDEC and the U.S. Army Geospatial Center's Engineering Research and Development Center, said Ron Szymanski, C2D technical lead for COBRA.

Driven by Unified Battle Command requirements to apply a consistent user interface across battle command systems, COBRA seeks to develop capabilities for greater collaboration and interoperability within all levels of Command and Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance - particularly the intelligence, operations, and geospatial communities.

COBRA seeks to support the integration and synchronization of intelligence, operations and geospatial functions by defining, developing and integrating multi-platform software services that will feature cross-community applications.

Moreover, COBRA is working to develop common infrastructure and tools that will enable the flow of information across boundaries that traditionally separate these communities, Szymanski said.

COBRA's collaboration services, decision-support software products, commercial joint mapping toolkit enhancements and unified net-centric data strategies will allow multiple Warfighter echelons to dynamically access, process and share relevant data despite using different systems or operating in a distributed environment.

C2D believes this will reduce training costs and result in faster, higher-quality decision cycles and increased Battle Command Unification, Szymanski said.

"COBRA is interested in exploring how OPS and INTEL collaboration can be more effective. The use of multi-touch technology seemed like a natural augmentation of existing modes of OPS and INTEL interoperation," he said.

COMET began as an afternoon side project that COBRA team members worked on after finishing their "day jobs."

"As part of C2D's new strategic vision, individual employees are encouraged to spend part of their time pursuing relevant research interests," Carpenter said. "Ron had become interested in multi-touch technology, so he put a small team together to explore the technology and how it might be applied to C2 problems."

Due to the lack of commercially available equipment, C2D initially built the required hardware and software for the multi-touch research. After the proof-of-concept model was successful, the project gained traction.

Knowing its research and commercial offering in multi-touch technology could aid in development, C2D demonstrated COMET to Microsoft. After seeing the prototype, Microsoft agreed to enter into a CRADA.

"This is something that we're very proud of and believe in," Roach said. "Hopefully, it will be a strongly collaborative role and sharing much of what we're developing in our own research centers. There are a number of things that were identified in this CRADA relative to multi-touch technologies and how that can relate directly to command and control applications."

"Getting our perspective as users will be beneficial to Microsoft and getting their commercial perspective in our environment is beneficial for us," said Gary W. Blohm, director of CERDEC.

"We know that we need to be adaptive and continually improve our abilities to improve our capabilities and our software products. You put your head down for five minutes and the world has changed - it really does happen that fast. Our intent is to stay at the forefront," Blohm said.

The COBRA team has purchased three Microsoft Surface units and has placed them at Aberdeen Proving Ground, Fort Monmouth, N.J., and Fort Leavenworth, Kan.

COMET now has its own budget and a full-time development staff. COBRA's relationship with the U.S. Army Research Laboratory has resulted in ARL's purchase of a Microsoft Surface unit for study as well.

C2D has also built COMET software for C4ISR On-The-Move Event 09, the largest integrated technology demonstration of its kind.

E09, which began June 1 at Fort Dix, N.J., incorporates various live communications, sensor and battle command systems augmented with virtual and constructive simulation using organic, high-performance computing assets and distributed connectivity.

The brigade-sized element of live and simulated systems is hosted in a battlefield-like environment, providing realistic conditions that are relevant to Soldiers.

"Participating in E09 will give us Soldier feedback and comments early in the development so that changes can be made. This will result in a better developed tool for the Soldiers when they are finally fielded," Szymanski said.

"It will also provide a real-world, relevant environment to help us evaluate the prototypes: communications systems don't work all the time, and bandwidth is at a premium," he said.

So far, testing has been limited to demonstrations and Warfighter responses to the COMET prototype at E08, Palmer said. However, the results were very encouraging, leading several organizations, including Battle Command Battle Laboratories at Fort Leavenworth, Kan., to express interest in working with the COMET team to further research multi-touch technology.

C2D has supported and developed COMET based on needs expressed by Project Manager Battle Command. PM BC will be targeted in the future for transition, but no transition dates have been set, Carpenter said.

"This is long-range research that will ultimately require additional hardware innovation. C2D is working with Microsoft to provide input for Microsoft's next revisions of the Surface equipment," she said.

"The bottom line is that our user, the Soldier who's out at the tip of the spear, needs to have certain information presented in a simple, understandable format with the ability to coordinate that data without a lot of mouse clicks. This kind of partnership helps us do that and be more responsive to the Warfighter's requirements," Blohm said.