A pair of artificial intelligence projects from U.S. Army researchers are easing communication barriers that limit the relationship between AI systems and soldiers.
The artificial intelligence projects are designed to support ongoing efforts for the Army’s next-generation combat vehicle modernization priority, which includes a focus on autonomous vehicles and AI-enabled platforms.
The first project, named the Joint Understanding and Dialogue Interface, or JUDI, is an AI system that can understand the intent of a soldier when that individual gives a robot verbal instructions. The second project, Transparent Multi-Modal Crew Interface Designs, is meant to give soldiers a better understanding of why AI systems make decisions.
“We’re attacking a similar problem from opposite ends,” said Brandon Perelman, a research psychologist at the Army Research Laboratory who worked on Transparent Multi-Modal Crew Interface Designs.
The JUDI system will improve soldiers’ situational awareness when working with robots because it will transform that relationship from a “heads down, hands full” to a “heads up, hands free” interaction, according to Matthew Marge, a computer scientist at the lab. Simply put, “this means that soldiers will be more aware of their surroundings,” he said.
Natural language AI systems are available on the commercial market, but the JUDI system requires a level of awareness in a physical environment that isn’t matched by commercial products. Commercial systems can understand what a person is saying and take instructions, but they don’t know what is going on in the surrounding area. For the Army, the autonomous system needs to know that.
“You want a robot to be able to process what you’re saying and ground it to the immediate physical context,” Marge said. “So that means the robot has to not only interpret the speech, but also have a good idea of where it is [in] the world, the mapping of its surroundings, and how it represents those surroundings in a way that can relate to what the soldier is saying.”
Researchers looked into how soldiers speak to robots, and how robots talk back. In prior research, Marge found that humans speak to technology in much simpler, direct language; but when talking to other people, they usually talk about a course of action and the steps involved. However, those studies were done in a safe environment, and not a stressful one similar to combat, during which a soldier’s language could be different. That’s an area where Marge knows the Army must perform more research.