Cornell team finds automated bot helps improve non-native participation in multilingual meetings

Research out of Cornell University finds that during multilingual online meetings the use of an automated bot that interrupts the conversation can help allow non-native speakers to participate on a level playing field with native speakers.

According to a report by the Cornell Chronicle, doctoral student Xiaoyan Li used the multilingual groups to test out the bot, which is called a “conversational agent.” It was programmed to interrupt a conversation after a native speaker took six straight turns. This reportedly helped increase non-native speaker participation from 12% to 17%.

“Non-native speakers appreciated having a gap to reflect on the conversation and the opportunity to ask questions,” Li said in the Cornell Chronicle report “Also, being invited to speak, they felt like their communication partners were valuing their perspectives.”

The study is called “Improving non-native speakers’ participation with an automatic agent in multilingual groups.” It was presented at the Association for Computing Machinery International Conference on Supporting Group Work. 

The research paper was published in Proceedings of the ACM on human-computer interaction. It was conducted with 48 volunteers who were put into groups of three, including two native English speakers and a native Japanese speaker.

The task was for the groups to undergo three survival exercises that included disaster scenarios and ranking items that would be used to survive. The agent was built by co-author Naomi Yamashita, a distinguished researcher at the Nippon Telegraph and Telephone Corporation (NTT). 

They used IBM Watson automatic speech recognition software to track who was speaking. It would blink and wave to signal when there would be an interruption. Non-native speakers found the agent helpful, but native speakers did say that the interruptions were distracting.

“Non-native speakers appreciated having a gap to reflect on the conversation and the opportunity to ask questions,” Li said in the report. “Also, being invited to speak, they felt like their communication partners were valuing their perspectives.”

Previous efforts have sought to help overcome language barriers, including meeting transcripts, automatic language translation and graphics that show participation. However, these all ultimately failed. The Cornell agent was successful, however, showing participation from non-native speakers increased 40%.

Li was joined by co-author Susan Fussell, a professor in the Department of Information Science in the Cornell Ann S. Bowers College of Computing and Information Science and in the Department of Communication in the College of Agriculture and Life Sciences. They recently developed their own agent and plan to test various proposed improvements. 

Fussell and Li plan to use more subtle signals, such as private messages to native speakers, or use AI or biosensors.