Spam filtering using classification algorithms

This thesis proposes an innovative adaptive multi-classifier spam filtering model, with a grey-list analyser and a dynamic feature selection method, to overcome false-positive problems in email classification. It also presents additional techniques to minimize the added complexity. Empirical evidence indicates the success of this model over existing approaches.

FEATURE SELECTION AND COMPARISON OF TWO NAÏVE BAYES CLASSIFICATION METHODS IN THE CONTEXT OF SPAM FILTERING

Treating e-mail filtering as a binary text classification problem, researchers have applied several statistical learning algorithms to email corpora with promising results. This paper examines the performance of a Naive Bayes classifier using different approaches to feature selection and tokenization on different email corpora

Spam filtering using ML algorithms

Spam is commonly defined as unsolicited email messages, and the goal of spam categorization is to distinguish between spam and legitimate email messages. Spam used to be considered a mere nuisance, but due to the abundant amounts of spam being sent today, it has progressed from being a nuisance to becoming a major problem. Spam filtering is able to control the problem in a variety of ways. Many researches in spam filtering has been centred on the more sophisticated classifier-related issues. Currently,  machine learning for spam classification is an important research issue at present. Support Vector Machines (SVMs) are a new learning method and achieve substantial improvements over the currently preferred methods, and behave robustly whilst tackling a variety of different learning tasks. Due to its high dimensional input, fewer irrelevant features and high accuracy, the  SVMs are more important to researchers for categorizing spam. This paper explores and identifies the use of different learning algorithms for classifying spam and legitimate messages from e-mail. A comparative analysis among the filtering techniques has also been presented in this paper.

Spam filtering based on preference ranking

When the average number of spam messages received is continually increasing exponentially, both the Internet service provider and the end user suffer. The lack of an efficient solution may threaten the usability of the email as a communication means. In this paper we present a filtering mechanism applying the idea of preference ranking. This filtering mechanism will distinguish spam emails from other email on the Internet. The preference ranking gives the similarity values for nominated emails and spam emails specified by users, so that the ISP/end users can deal with spam emails at filtering points. We designed three filtering points to classify nominated emails into spam email, unsure email and legitimate email. This filtering mechanism can be applied on both middleware and at the client-side. The experiments show that high precision, recall and TCR (total cost ratio) of spam emails can be predicted for the preference based filtering mechanisms.

Dynamic feature selection for spam filtering using support vector machine

Spam is commonly defined as unsolicited email messages and the goal of spam filtering is to differentiate spam from legitimate email. Much work have been done to filter spam from legitimate emails using machine learning algorithm and substantial performance has been achieved with some amount of false positive (FP) tradeoffs. In this paper, architecture of spam filtering has been proposed based on support vector machine (SVM,) which will get better accuracy by reducing FP problems. In this architecture an innovative technique for feature selection called dynamic feature selection (DFS) has been proposed which is enhanced the overall performance of the architecture with reduction of FP problems. The experimental result shows that the proposed technique gives better performance compare to similar existing techniques.

Spam filtering for network traffic security on a multi-core environment

This paper presents an innovative fusion-based multi-classifier e-mail classification on a ubiquitous multicore architecture. Many previous approaches used text-based single classifiers to identify spam messages from a large e-mail corpus with some amount of false positive tradeoffs. Researchers are trying to prevent false positive in their filtering methods, but so far none of the current research has claimed zero false positive results. In e-mail classification false positive can potentially cause serious problems for the user. In this paper, we use fusion-based multi-classifier classification technique in a multi-core framework. By running each classifier process in parallel within their dedicated core, we greatly improve the performance of our multi-classifier-based filtering system in terms of running time, false positive rate, and filtering accuracy. Our proposed architecture also provides a safeguard of user mailbox from different malicious attacks. Our experimental results show that we achieved an average of 30% speedup at an average cost of 1.4 ms. We also reduced the instances of false positives, which are one of the key challenges in a spam filtering system, and increases e-mail classification accuracy substantially compared with single classification techniques.

Spamcooling : a parallel heterogeneous ensemble spam filtering system based on active learning techniques

Anti-spam technology is developing rapidly in recent years. With the emerging applications of machine learning in diverse fields, researchers as well as manufacturers around the world have attempted a large number of related algorithms to prevent spam. In this paper, we designed an effective anti-spam protection system, SpamCooling, based on the mechanism of active learning and parallel heterogeneous ensemble learning techniques. The system adopts a batch method to filter spam and can be easily incorporated with existing mail clients (MUA). It can actively obtain user feedbacks for providing users with personalized spam filtering experiences. The parallel heterogeneous ensemble method can help system achieve high spam detection rate as well as low ham misclassification rate.

Collaborative knowledge creation in electronic mail

With organisational work increasingly performed by the collaboration of distributed groups, an improved understanding is needed of the co-creation of knowledge in emerging virtual structures. We explore the potential of the ubiquitous organisational tool, electronic mail (e-mail), for supporting collaborative knowledge creation in such settings. This research draws on a case study of knowledge creation occurring in e-mail conversations in a large Australian university and adopts a discourse analysis research approach. We describe a model of collaborative knowledge creation derived from the study and identify a preliminary set of key factors for organisational knowledge tools and their use by groups to support collaborative knowledge creation. The paper also provides insights into the role of e-mail in collaborative knowledge creation, not only in facilitating this process, but in shaping a participatory, multi-perspective, team-based approach to knowledge building. Organisational implications arising from this type of knowledge creation are also discussed in the paper.

An innovative spam filtering model based on support vector machine

Spam is commonly defined as unsolicited email messages and the goal of spam categorization is to distinguish between spam and legitimate email messages. Many researchers have been trying to separate spam from legitimate emails using machine learning algorithms based on statistical learning methods. In this paper, an innovative and intelligent spam filtering model has been proposed based on support vector machine (SVM). This model combines both linear and nonlinear SVM techniques where linear SVM performs better for text based spam classification that share similar characteristics. The proposed model considers both text and image based email messages for classification by selecting an appropriate kernel function for information transformation.

Architecture of adaptive spam filtering based on machine learning algorithms

Spam is commonly defined as unsolicited email messages and the goal of spam filtering is to distinguish between spam and legitimate email messages. Much work has been done to filter spam from legitimate emails using machine learning algorithm and substantial performance has been achieved with some amount of false positive (FP) tradeoffs. In the case of spam detection FP problem is unacceptable sometimes. In this paper, an adaptive spam filtering model has been proposed based on Machine learning (ML) algorithms which will get better accuracy by reducing FP problems. This model consists of individual and combined filtering approach from existing well known ML algorithms. The proposed model considers both individual and collective output and analyzes them by an analyzer. A dynamic feature selection (DFS) technique also proposed in this paper for getting better accuracy.

Socio-ethical issues for expertise location from electronic mail

Knowledge workers frequently lack sufficient expertise to perform work effectively. This paper describes a recently developed expertise locator system based on automated key-phrase identification of experts from electronic mail (e-mail) messages. The paper provides an analysis of the key socio-ethical challenges involved in the implementation and use of the e-mail expertise locator system. Findings include a set of complex socio-ethical challenges, and their managerial and theoretical implications are discussed. The paper highlights possible sensitivities of employees with respect to their potential identification by the system as domain experts. It also highlights the potential for employee misuse of e-mail expertise locator systems, which must be carefully managed to reduce the risks involved.

Study on ensemble classification methods towards spam filtering

Recently, many scholars make use of fusion of filters to enhance the performance of spam filtering. In the past several years, a lot of effort has been devoted to different ensemble methods to achieve better performance. In reality, how to select appropriate ensemble methods towards spam filtering is an unsolved problem. In this paper, we investigate this problem through designing a framework to compare the performances among various ensemble methods. It is helpful for researchers to fight spam email more effectively in applied systems. The experimental results indicate that online based methods perform well on accuracy, while the off-line batch methods are evidently influenced by the size of data set. When a large data set is involved, the performance of off-line batch methods is not at par with online methods, and in the framework of online methods, the performance of parallel ensemble is better when using complex filters only.

Minimizing the drawbacks of grey-list analyser in synthesis based spam filtering

In the last decade, the rapid growth of the Internet and email, there has been a dramatic growth in spam. Spam is commonly defined as unsolicited email messages and protecting email from the infiltration of spam is an important research issue. Classifications algorithms have been successfully used to filter spam, but with a certain amount of false positive trade-offs, which is unacceptable to users sometimes. This paper presents an approach of email classification to overcome the burden of analyzing technique of GL (grey list) analyzer as further refinements of synthesis based email classification technique. In this approach, we introduce a “majority voting grey list (MVGL)” analyzing technique which will analyze the GL emails by using the majority voting (MV) algorithm. We have presented two different variations of the MV system, one is simple MV (SMV) and other is the Ranked MV (RMV). Our empirical evidence proofs the improvements of this approach compared to existing GL analyzer [7].