Statistical classification

You don't need to be Editor-In-Chief to add or edit content to WikiDoc. You can begin to add to or edit text on this WikiDoc page by clicking on the edit button at the top of this page. Next enter or edit the information that you would like to appear here. Once you are done editing, scroll down and click the Save page button at the bottom of the page.

Statistical classification is a procedure in which individual items are placed into groups based on quantitative information on one or more characteristics inherent in the items (referred to as traits, variables, characters, etc) and based on a training set of previously labeled items.

Formally, the problem can be stated as follows: given training data produce a classifier which maps an object to its classification label . For example, if the problem is filtering spam, then is some representation of an email and y is either "Spam" or "Non-Spam".

Statistical classification algorithms are typically used in pattern recognition systems.

Note: in community ecology, the term "classification" is synonymous with what is commonly known (in machine learning) as clustering. See that article for more information about purely unsupervised techniques.

Contents 1 Statistical classification techniques 2 Application domains 3 References 4 External links 5 See also

Statistical classification techniques

While there are many methods for classification, they are solving one of three related mathematical problems

• The first is to find a map of a feature space (which is typically a multi-dimensional vector space) to a set of labels. This is equivalent to partitioning the feature space into regions, then assigning a label to each region. Such algorithms (e.g., the nearest neighbour algorithm) typically do not yield confidence or class probabilities, unless post-processing is applied. Another set of algorithms to solve this problem first apply unsupervised clustering to the feature space, then attempt to label each of the clusters or regions.

• The second problem is to consider classification as an estimation problem, where the goal is to estimate a function of the form

where the feature vector input is , and the function f is typically parameterized by some parameters . In the Bayesian approach to this problem, instead of choosing a single parameter vector , the result is integrated over all possible thetas, with the thetas weighted by how likely they are given the training data D:

• The third problem is related to the second, but the problem is to estimate the class-conditional probabilities and then use Bayes' rule to produce the class probability as in the second problem.

Examples of classification algorithms include:

• Linear classifiers
  • Fisher's linear discriminant
  • Logistic regression
  • Naive Bayes classifier
  • Perceptron
• Quadratic classifiers
• k-nearest neighbor
• Boosting
• Decision trees
• Neural networks
• Bayesian networks
• Support vector machines
• Hidden Markov models

An intriguing problem in pattern recognition yet to be solved is the relationship between the problem to be solved (data to be classified) and the performance of various pattern recognition algorithms (classifiers). Van der Walt and Barnard (see reference section) investigated very specific artificial data sets to determine conditions under which certain classifiers perform better and worse than others.

Classifier performance depend greatly on the characteristics of the data to be classified. There is no single classifier that works best on all given problems (a phenomenon that may be explained by the No-free-lunch theorem). Various empirical tests have been performed to compare classifier performance and to find the characteristics of data that determine classifier performance. Determining a suitable classifier for a given problem is however still more an art than a science.

The most widely used classifiers are the Neural Network (Multi-layer Perception), Support Vector Machines, k-Nearest Neighbours, Gaussian Mixture Model, Gaussian, Naive Bayes, Decision Tree and RBF classifiers.

Application domains

• Computer vision
  • Medical Imaging and Medical Image Analysis
  • Optical character recognition
• Geostatistics
• Speech recognition
• Handwriting recognition
• Biometric identification
• Natural language processing
• Document classification
• Internet search engines
• Credit scoring

References

• C.M. van der Walt and E. Barnard,“Data characteristics that determine classifier performance”, in Proceedings of the Sixteenth Annual Symposium of the Pattern Recognition Association of South Africa, pp.160-165, 2006.

External links

• Classifier showdown A practical comparison of classification algorithms.
• Statistical Pattern Recognition Toolbox for Matlab.

See also

• Data mining
• Fuzzy logic
• Information retrievalar:تصنيف إحصائي

de:Klassifikationsverfahren lt:Klasifikavimo algoritmaith:การแบ่งประเภทข้อมูล