Design of hierarchical classifiers

Decision trees provide a powerful method of pattern classification. At each node in a binary tree a decision is made based upon the value of one of many possible attributes or features. The leaves of the tree represent the various classes that can be recognized. Various techniques have been used to select the feature and threshold to use at each node based upon a set of training data. Information theoretic methods are the most popular techniques used for designing each node in the tree. An alternate method uses the Kolmogorov-Smirnov test to design classification trees involving two classes. This paper presents an extension of this method that can produce a single decision tree when there are multiple classes. The relationship between this generalized Kolmogorov-Smirnov method and entropy minimization methods will be described. Experiments comparing classification results using this decision tree with results of using a Bayesian classifier will be presented.