Sentiment detection through deep image learning in limited data
Date
2022
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
In this dissertation, I describe several advances to deep learning applied to sentiment detection on both and video with insufficient data. A deep learning system built using the ideas presented herein finished third in the 2019 and first in the 2020 Emotion in the Wild Grand Challenges. ☐ The first advance is the use of multi-cue transfer learning as a way of solving tasks that lack data. I develop an efficient transfer learning approach that utilizes multiple deep models pre-trained on several high-resource datasets. The models, learning from multiple related source tasks, apply the transferred multi-domain based knowledge to the target task that improves the performance and alleviates overfitting caused by insufficient data. ☐ The second advance is a multi-cue attention-based Gated Recurrent Unit (GRU) deep network that solves the video detection task. In a video, all frames convey some information, but not all frames are equally meaningful. A specially designed attention layer aims to select the useful information across the time series data that is adapted to our multi-cue hybrid model which incorporates a different modality of features. ☐ The third advance is a hybrid network with multi-modal contrastive distillation, which enables supervised transfer across multiple modalities, resulting in improved visual representations, a more robust classifier, and faster inference. We commonly wish to move knowledge from a big to a small model to speed up inference. The concept of knowledge distillation is frequently used to transmit knowledge from a teacher to a student model. The student model, on the other hand, loses important structural knowledge present in the teacher model, resulting in subpar performance. To address this issue, we develop a novel hybrid network structure that combines a compositional contrastive loss for learning visual representations across several modalities with a cross entropy loss for learning a classifier, based on the concept that richer features produce better classifiers. ☐ In addition to the advances mentioned above, this dissertation focuses on the simplicity of the techniques dealing with practical tasks or applications. Also, I study transfer learning, multi-cues, and attention mechanisms, contrastive learning, and how the techniques address specific problems regarding low quantity of data. This dissertation is organized as follows. Chapter one and Chapter two introduce background knowledge and related work. Chapters three, four, five and six explore our proposed methods on predicting group level sentimental tasks in image and engagement intensity of students watching educational videos. Chapter 7 provides conclusion and suggestion for future work.
Description
Keywords
Sentiment detection, Limited data, Deep learning, Static image, Contrastive learning, Educational videos