From left, Rangan Majumder, Yi-Min Wang and Jianfeng Gao on Microsoft’s Redmond, Washington, campus. Photo by Dan DeLong.
Microsoft researchers have already created technology that can do two difficult tasks about as well as a person: identify images and recognize words in a conversation.
Now, the company’s leading AI experts are working on systems that can do something even more complex: Read passages of text and answer questions about them.
“We’re trying to develop what we call a literate machine: A machine that can read text, understand text and then learn how to communicate, whether it’s written or orally,” said Kaheer Suleman, the co-founder of Maluuba, a Quebec-based deep learning startup that Microsoft acquired earlier this year.
The Maluuba team is one of several groups at Microsoft that are tackling the challenge of machine reading. Two other research teams, one at the company’s Redmond, Washington, headquarters and the other in its Beijing, China, research lab, are currently leading a competition run by Stanford University that uses information from Wikipedia to test how well AI systems can answer questions about text passages.
The so-called SQuAD dataset is the core benchmark for the emerging field of machine reading, and many leading academic and industry teams are using it to test their systems. It’s similar to the ImageNetcompetition that spurred advances in computer vision.
Microsoft researchers and other industry and academic experts also are competing for the best results using another dataset, called MS MARCO, that uses real, anonymized data from Bing search queries to test a system’s ability to answer a question.
The teams say that’s an added challenge because it’s based on people’s real-world questions. Testing on that kind of data helps ensure the systems they are building will eventually be useful for real customers.
“We’re not just going to build a bunch of algorithms to solve theoretical problems. We’re using them to solve real problems and testing them on real data,” said Rangan Majumder, a partner group program manager within Microsoft’s Bing division. He’s working closely with the Redmond machine reading research team and led development of the MS MARCO dataset.
A group photo of Microsoft Research Asia’s natural language processing team, led by Ming Zhou, center in light blue.
Cognition versus perception
In general, AI experts say machine reading is more difficult than other AI tasks, like image recognition, because there is so much more ambiguity involved.
Ming Zhou, assistant managing director of Microsoft Research Asia in Beijing, who leads the Natural Language Research Group, said skills like image recognition are perception tasks: The system uses a machine learning algorithm to recognize an image based on all the images it has seen before.
Machine reading is more of a cognitive task: It requires the system to also take a big-picture view, looking for the context of the words it is reading and perhaps even bringing in some background knowledge it already has on the subject.
“Some words might mean different things, and the same things might be mentioned in different ways,” Zhou said.
Another complication: The answer may not contain all – or even any – of the words in the question.
For example, let’s say someone asks the question, “What is John Smith’s citizenship?” The answer could be “John Smith was born in the United States” or “He has a U.S. passport.” In either case, the system needs to look for, and use, information that relates to a question about citizenship but may not explicitly say that word.
“It has to generate an answer – it’s not like the answer is already there,” said Jianfeng Gao, a partner research manager in Microsoft’s Deep Learning Technology Center.
Suleman, the Maluuba co-founder, noted that this is exactly how people test whether other people have learned something: They ask questions, starting when humans are just babies and continuing through most of a person’s education.
It was a deeper look at how people learn that prompted his team to take the machine reading task one step further: They are working on a system that can read a passage and formulate a question about it, rather than an answer. The work was inspired by research in the early 1980s showing that students who were asked to write questions about a topic generally did better on question-and-answer tests.
CLICK HERE to read the full article to learn more about artificial intelligence and how Microsoft is leading the way in machine reading cognitive tasks!
To learn more about AI and how Tallan can help you maximize customer engagement and support with Chatbot Technology, CLICK HERE.