ADELPHI, Md. -- An Army scientist led an effort to release a new report on what foundational research is needed to create speech-capable robots that communicate with humans easily and effectively.
Dr. Matthew Marge, from the U.S. Army Combat Capabilities Development Command, now known as DEVCOM, Army Research Laboratory, led an international panel of artificial intelligence experts from industry, academia and government in releasing a first-of-its-kind report, Spoken Language Interaction with Robots: Research Issues and Recommendations.
The report stemmed from a National Science Foundation and University of Maryland sponsored workshop in 2019, which brought the researchers together for the first time.
The NSF sponsored the document, which identifies and highlights research areas, such as shareable resources and software infrastructure for speech and robotics, AI components that are robust to uncertainty and designs for multi-party team interaction.
“The technology roadmap and research recommendations within this seminal report are to direct the Department of Defense and other funding agencies on foundational research needed to create speech-capable robots that can naturally talk with humans,” Marge said. “Many Army programs that have a critical need for autonomous systems to interact through speech with Soldiers, like Next Generation Combat Vehicle, will benefit from the guidelines in the report.”
This effort supports Army priority research areas for Artificial Intelligence and Autonomy, he said, by reducing Soldier burden when teaming with autonomous systems and by allowing verbal command and control of systems.
The achievements of the workshop and the report that followed were threefold:
- Connecting the speech/dialogue and robotics disciplines
- Making specific recommendations for research
- Developing a long-term technology roadmap
The researchers identified key scientific and engineering advances needed to enable effective spoken language interaction with robots, Marge said.
They also made 31 recommendations involving eight general themes: putting human needs first, better modeling the social and interactive aspects of language, improving robustness, creating new methods for rapid adaptation, better integrating speech and language with other communication modalities, giving speech and language components access to rich representations of the robot’s knowledge and state, making all components operate in real time, and improving research infrastructure and resources.
Research and development that prioritizes these topics will, the researchers believe, accelerate progress in building speech-capable robots, Marge said.
The capability to address a robot with speech in a natural way with everyday language represents a transformational capability for the Army, Marge said. Not only can a Soldier keep their eyes on their surroundings with speech in comparison to a joystick as is done today, but they can issue tasks to more than one robot, scaling up the number of robots that a single Soldier can team with, he said.
“The NSF-sponsored workshop that led to this report served as a catalyst to bring the speech and robotics communities together to help shape future directions of research,” Marge said. “The recommendations laid out by the AI experts in the report will help the Army focus investments in this research.”
The laboratory helped organize and had a major role in the workshop, including helping to prepare the list of attendees, creating breakout sessions and leading discussion outbriefs during the workshop.
Army-relevant problems, such as remote exploration and disaster relief, were well-received and were included as critical use cases in the final report, Marge said.
Projects such as the Artificial Intelligence for Maneuver and Mobility Essential Research Program’s JUDI capability will benefit from more investment in this area, Marge said. At the same time, researchers can explore ways that this research could translate to use in supporting interaction with other software agents, he added.
“The meeting of the minds that contributed to both the workshop and report will fundamentally change how research in the space of spoken language interaction with robots will be done for years to come,” Marge said. “Technology innovations in this space will help Soldiers transition to interacting with autonomous systems in hands-free ways using speech.”
The report is to be distributed across the DOD and government enterprise to inform how future research involving speech and robotics is to be prioritized.
Visit the laboratory's Media Center to discover more Army science and technology stories
DEVCOM Army Research Laboratory is an element of the U.S. Army Combat Capabilities Development Command. As the Army’s corporate research laboratory, ARL is operationalizing science to achieve transformational overmatch. Through collaboration across the command’s core technical competencies, DEVCOM leads in the discovery, development and delivery of the technology-based capabilities required to make Soldiers more successful at winning the nation’s wars and come home safely. DEVCOM is a major subordinate command of the Army Futures Command.
Social Sharing