AI Task Force taking giant leaps forward

By Gary SheftickOctober 7, 2019

Army AI Task Force
1 / 2 Show Caption + Hide Caption – Col. Doug Matty (left), deputy director of the Army's Artificial Intelligence Task Force, discusses an AI app for talent management being developed by Maj. Kevin Goulding (right). Behind them in the lobby of the National Robotics Engineering Center i... (Photo Credit: U.S. Army) VIEW ORIGINAL
2 / 2 Show Caption + Hide Caption – Josh Cauvel, program manager for the Artificial Intelligence Hub at Carnegie Mellon University, points out characteristics of CHIMP, a humanoid robot at the National Robotics Engineering Center that won third place in a contest sponsored by DARPA in ... (Photo Credit: U.S. Army) VIEW ORIGINAL

*Leveraging AI applications to achieve Army goals will be one of the topics the Army's AI Task Force will be discussing during AUSA's annual meeting Oct. 14-16 (Warriors Corner Monday, Oct. 14). Experts will be on hand to answer questions and provide resources and information to attendees. For more information on the event, visit

PITTSBURGH --- From their office overlooking a bay of autonomous vehicles and robotics, members of a small Army task force are collaborating with academic partners to develop artificial intelligence systems.

The Army's AI Task Force stood up less than a year ago through a partnership between Army Futures Command and Carnegie Mellon University, and members are already working on a number of cutting-edge applications to assist with tasks ranging from aircraft maintenance to talent management.

While Task Force Director Brig. Gen. Matthew Easley has an administrative section near the Pentagon, the operational element of the task force is located at CMU's National Robotics Engineering Center in Pittsburgh, headed up by Col. Doug Matty, Army AI TF deputy director.

"We were able to leverage existing relationships" between Carnegie Mellon and DOD through Army Research Lab, Matty said, to create an Army task force that could tap into the artificial intelligence "ecosystem."

CMU has a long history of working with the Defense Advanced Research Projects Agency, known as DARPA, he explained, and NASA was one of the first agencies to fund projects at the university's National Robotics Engineering Center.


Located along the Allegheny River about 2 miles from the main CMU campus, NREC is an integral part of the university's Robotics Institute that boasts over 1,000 faculty and researchers. Five university staffers at NREC have formed an AI Hub to work directly with the Army task force and reach out to other universities and industry partners to link a network of researchers.

The offices of five Army officers and two Army employees at NREC look down into a sprawling interior bay that includes some of the first fully autonomous cars and robots designed to clean up nuclear power-plant disasters.

"While the Army AI Task Force didn't necessarily sponsor that work, we're benefiting from it," Matty said. He explained that access to the sensors, different types of electro-mechanical devices and computing capabilities there enable the task force to quickly develop artificial intelligence for other applications.

"We're not starting from zero," he said.


Automated recognition --- the capability for a computer to identify military vehicles and systems in images --- was one of the first projects that the task force tackled.

NREC teams already were using electro-optical 360-degree camera sensors on robots, LiDAR light-detecting and ranging lasers, and other types of sensors.

"That's what's allowing us to go so fast when it comes time to build out a new sensor package for automated recognition," Matty said. "We're able to put those systems together, because they've already solved those problems."

Since NREC built the first fully autonomous prototype in 2004, "you can imagine the advances in sensors, the advances in computation, the reduction in power usage, all of those types of things --- you can see how much more improved and enhanced those capabilities are. I think that's really the key thing that we're trying to leverage," Matty said.

Maintenance for helicopters is another project that the task force is working on. Data scientists are developing an application that will recognize and alert flight crews when it's time for different types of maintenance.


The latest project for which the task force is developing an AI application for is talent management. Maj. Kevin Goulding, who has a master's degree from CMU, is heading up the project.

Goulding said he is "plugged into" the Army's Talent Management Task Force in Arlington, Virginia, to develop an app that will assist branch managers when they make assignment decisions.

He is creating an "optimization algorithm" that will "give a branch manager a better way to create a base assignment scheme," using data about the education and experience Soldiers have acquired, Goulding said. He added that it's definitely not going to replace branch managers.

The goal is to "maximize happiness of both officer and organization," he said.

To that end, Goulding is building machine-learning models and typing computer code.

He's writing code, "not making new math," Goulding pointed out, because he's often tweaking algorithms that have already been developed.

Matty says the task force is looking to develop capabilities through a "scrum methodology" which he defines as "just a big old mess of pushing and pulling, but it's really about moving the ball forward." In his case, it's about moving the frontier of technology forward.


The task force has been working closely with cross functional teams across Army Futures Command.

"Even though we're here in Pittsburgh and Futures Command is located in Austin, we proactively work to stay tight with our teammates," Matty said.

All eight of the command's cross-functional teams have visited Matty's task force in Pittsburgh to identify technology gaps that artificial intelligence might be able to fill.

From the Long Range Precision Fires CFT to the Synthetic Training Environment CFT, the task force has assessed their modernization initiatives for ways AI might help.

The task force is always looking for "the next big question" it can help address, Matty said.

Matty's team also supports DOD's Joint AI Center, and he said that's stipulated in its charter. One of its goals is to "push algorithms forward to the tactical edge," Matty said, to directly support operations.

And the task force is not just creating artificial intelligence apps, but an entire "AI stack," to include doctrine, organizations and training, he said.


"We kind of have a unique blend of technical expertise in the task force," Matty said, adding that some the Army's premiere data scientists and best data engineers are assigned.

The task force also benefits from temporary expertise that Matty can find. Currently a civilian employee from the Army's G-6 staff section, Taylor Cloyd, is with the task force in Pittsburgh for a few months. West Point faculty member Col. David Barnes is also in the middle of what he calls a "sort of sabbatical" with the task force. He teaches philosophy and English at the academy but is currently looking at ethical issues concerning artificial intelligence.

In addition, the task force has a contracting specialist and acquisition officer who provide what Matty calls the "more traditional institutional expertise" of coordination and synchronization. They allow the task force to "rapidly engage" the academic and industrial community, he said.

Ruben Cruz, contracts portfolio manager, draws up collaborative agreements with other universities. Nine universities currently have formal agreements with the task force, but Cruz said even more are collaborating informally.

He also reaches out to small businesses.

It used to be that acquisition and development was focused primarily on large "prime" contractors, Matty said. Now Futures Command is looking to engage with startups, small businesses and mid-level companies.

"One of the things I do is I reach out to the 'technology incubators' out there," Cruz said.

"If there are certain businesses or people who we feel have innovative technology … we talk to them … and if it's something that we're interested in, then it's my job to figure out how to bring them into the fold."

The key partner for this mechanism is AFC's Army Applications Lab.


The university's AI Hub is essentially a network of partner universities, as well as potential industry partners, Matty said, that the Army will look to leverage.

One of the jobs of the AI Hub staff is to facilitate smooth communications between universities and the Army, said Josh Cauvel, the hub's program manager. "Army language doesn't always translate into academic," he said.

Rob Toth, executive director of the AI Hub, said that although his direct staff is small, the entire faculty and staff of Carnegie Mellon is behind the program.

"It's not only the academic research you see with professors and graduate students," Matty said, but many of the academic partners also have federally-funded research and development centers, such as the MIT Lincoln Labs, Georgia Tech Research Institute and the University of Southern California's Institute for Creative Technology.

Researchers at these organizations are working on projects that will make a difference, Cauvel said, and providing "synergy to leapfrog fields forward."

"There's a real excitement and energy that's coming with these projects," Cauvel said, adding researchers are "pushing out beyond the bleeding edge."

Toth compared the AI Hub to the Apollo space program in the 1960s: "We're bringing together the best and the brightest from around the country to put together something that doesn't exist today."

He said the task force is looking for solutions to "hard problems" that nobody today has an answer to yet, and the results will ultimately be a "game-changer."

Related Links:

Army News Service

ARNEWS archives

Soldiers magazine features

2019 AUSA annual meeting and live webcast schedule

Army Futures Command

STAND-TO! Army AI and Machine Learning Initiatives