An adaptive retrieval framework for multi-turn retrieval-based chatbots
Date
2021
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
Chatbots have been playing an important role in many applications such as customer support and tutoring systems. Previous chatbot systems were mostly built on manually defined rules, but they could only respond if there was a matching query in the database. With the development of deep learning techniques, neural networks have been proved to be a new paradigm of chatbot models.They can learn patterns of conversations and continuously improve as more data comes in. In this dissertation,an adaptive response retrieval framework for multi-turn chatbot models is proposed based on the research results of chatbot models and the main advantages of neural networks. Specifically, the proposed framework can be adaptive to:(1) semantic matching types, (2) length, and (3) question responses. ☐ In the first part of this dissertation, an adaptive response matching networks (ARM) which adapts to semantic matching types is proposed. Although various models have been proposed for response matching, few of them studied how to adapt the matching mechanism to utterance types and domain knowledge. To address this limitation, we proposed an adaptive response matching network (ARM) to better model the matching relationship in multi-turn conversations. It also has a knowledge embedding component to inject domain-specific knowledge in the matching process. ☐ Then this dissertation presents our proposed framework to perform length adaptive regularization on candidate responses based on predicted length expectation. Intuitively, based on the nature of the conversations, some responses are expected to be long and informative while others need to be more concise. Unfortunately, none of the existing retrieval-based chatbot models have considered the effect of response length. Therefore, we proposed a length adaptive regularization method for retrieval-based chatbot models. The proposed length adaptive regularization method is general enough to be applied to all existing retrieval-based chatbot models. ☐ This thesis research concludes with a proposed framework that is adaptive to question responses. During the conversation, chatbots are expected to (1) provide a direct assistant when the user request is clear or (2) ask clarifying questions to gather more information to better understand the user's need. Despite its importance, few studies have focused on when to ask questions and how to retrieve relevant questions accordingly. As a result, existing retrieval-based chatbot models poorly perform when the correct response is a question. To overcome this limitation, we proposed an adaptive response retrieval model. Experiments show the proposed adaptive framework can significantly and consistently improve the retrieval performance, particularly the responses to questions. ☐ This framework improves the performance of answer retrieval at each stage. Moreover, the proposed framework doesn't contain any domain-specific tuning, and it consistently outperforms the state of the art methods across different domains. The better retrieval capabilities provided in this dissertation are a solid step towards the next generation of intelligent chatbot systems.
Description
Keywords
Chatbots, Deep learning, Neural networks