Enhancing inductive learning with feature selection and example selection

Abstract

While most of the stable learning algorithms perform well on domains with relevant information, they degrade in the presence of irrelevant or redundant information. Selective or focused learning presents a solution to this problem. Two components of selective learning are selective attention (feature selection) and selective utilization (example selection). In this thesis, we present novel algorithms for feature selection and example selection and present the benefits of these two approaches independently and as a combined scheme. We propose a sequential search filter approach called Subset selection using Case-based Relevance APproach (SCRAP) for identifying and eliminating irrelevant features. The SCRAP filter addresses the problem of finding a feature subset that provides a balance between defining consistent hypotheses and improving prediction accuracy. The SCRAP filter was compared with the RELIEF filter algorithm and was found to perform better on three families of learning algorithms. We also propose the Learning Algorithm using SEarch Ring (LASER) framework to perform example selection for learning algorithms. The LASER framework has two components, an example selection scheme and target learner. Naive Bayes was used as the target learner for our experiments. LASER provides significant improvement in prediction accuracy of the naive Bayes learner compared to the naive Bayes classifier without example selection. Application of both feature and example selection schemes to the naive Bayes learner resulted in better prediction accuracy.