Classifying Tweets using Convolutional Neural Networks with Multi-Channel Distributed Representation

This paper is focused on a state-of-art classification method for short text messages. With the increasing interest in social media, people are posting many short text messages, not only to communicate with other people, but also to share information. This trend has led to a new research area, microblog mining. This type of data mining can extract realworld topics and events from microblogs. However, because text messages on a micro-blogging site are short, their classification is a challenging task. In particular, Twitter is one of the most well-known micro-blogging services, where tweets frequently users’ reactions to real-world topics and events occurring in their surroundings. In our previous paper, we proposed a realtime topic monitoring system that implements a naive Bayes classifier to classify tweets into two classes: “relevant” and “irrelevant” to a monitored topic. The classification performance has limitations, because the naive Bayes’ classification is based on word probabilities. To address this problem, we propose a deep-learning-based classification method that features a new distributed representation for words, multichannel distributed representation. Distributed representation indicates word vectors representing the latent features of words. To enhance the capability of a distributed representation, each of its items has several channel values in a multi-channel distributed representation. In our experiments, we evaluated our model’s performance in comparison with that of other convolutional neural network (CNN) models and a long shortterm memory model. The results showed that the classification performance of the deep learning models was superior to that of the naive Bayes classifier. Moreover, a CNN with multi-channel distributed representation can classify tweets better than a CNN without multi-channel distributed representation.