Intelligent Question Answering Using the Wisdom of the Crowd
In recent years, community Question Answering forums such as StackOverflow, Quora, and BG-Mamma have gained a lot of popularity as a source of knowledge. These forums typically organize their content in the form of multiple topic-oriented question–answer threads, where a question posted by a user in the past is followed by a possibly very long list of other users' comments intended to answer the question. Many such on-line forums are not moderated, which often results in noisy and redundant content, as users tend to deviate from the question and start asking new questions or engage in conversations, fights, etc. Yet, they represent a rich source of information, which can help answer a number of new questions, as people often ask similar things again and again.
I will explore three general problems related to such forums: (i) deciding which answers are good, (ii) finding related/duplicated questions, and (iii) finding good answers to a new question. This will involve models based on deep learning and semantic/syntactic kernels. Part of this work was integrated in a production system, e.g., as thumbs up in a forum for (i), or as part of a smart search for (iii).
I will also introduce some promising extensions of this work in directions such as application to Arabic (can we apply these models to Arabic medical forums) and Bulgarian (how about the BG-Mamma forum?), cross-language question answering (can we answer a question in Arabic using a forum that is in English), fact checking (many answers in the forum look superficially good, but which of them are actually factual), trollness detection (can we find the forum trolls and their answers), answer justification (can we find and group contradictory answers), and interactive cQA (can we turn an entire forum into a chatbot, e.g., one that can be integrated in Alexa).
This research was performed by the Arabic Language Technologies (ALT) group at the Qatar Computing Research Institute, BHKU. It is part of the Interactive sYstems for Answer Search (IYAS) project, which is developed in collaboration with MIT-CSAIL.
In recent years, community Question Answering forums such as StackOverflow, Quora, and BG-Mamma have gained a lot of popularity as a source of knowledge. These forums typically organize their content in the form of multiple topic-oriented question–answer threads, where a question posted by a user in the past is followed by a possibly very long list of other users' comments intended to answer the question. Many such on-line forums are not moderated, which often results in noisy and redundant content, as users tend to deviate from the question and start asking new questions or engage in conversations, fights, etc. Yet, they represent a rich source of information, which can help answer a number of new questions, as people often ask similar things again and again.
I will explore three general problems related to such forums: (i) deciding which answers are good, (ii) finding related/duplicated questions, and (iii) finding good answers to a new question. This will involve models based on deep learning and semantic/syntactic kernels. Part of this work was integrated in a production system, e.g., as thumbs up in a forum for (i), or as part of a smart search for (iii).
I will also introduce some promising extensions of this work in directions such as application to Arabic (can we apply these models to Arabic medical forums) and Bulgarian (how about the BG-Mamma forum?), cross-language question answering (can we answer a question in Arabic using a forum that is in English), fact checking (many answers in the forum look superficially good, but which of them are actually factual), trollness detection (can we find the forum trolls and their answers), answer justification (can we find and group contradictory answers), and interactive cQA (can we turn an entire forum into a chatbot, e.g., one that can be integrated in Alexa).
This research was performed by the Arabic Language Technologies (ALT) group at the Qatar Computing Research Institute, BHKU. It is part of the Interactive sYstems for Answer Search (IYAS) project, which is developed in collaboration with MIT-CSAIL.