Feature Extraction for the k-Nearest Neighbour
Classifier with Genetic Programming
M. Bot. Genetic Programming, Proceedings of EuroGP'2001, volume 2038 of LNCS, page 256--267. Lake Como, Italy, Springer-Verlag, (18-20 April 2001)
Abstract
In pattern recognition the curse of dimensionality can
be handled either by reducing the number of features,
e.g. with decision trees or by extraction of new
features.
We propose a genetic programming (GP) framework for
automatic extraction of features with the express aim
of dimension reduction and the additional aim of
improving accuracy of the k-nearest neighbour (k-NN)
classifier. We will show that our system is capable of
reducing most datasets to one or two features while
k-NN accuracy improves or stays the same. Such a small
number of features has the great advantage of allowing
visual inspection of the dataset in a two-dimensional
plot.
Since k-NN is a non-linear classification algorithm, we
compare several linear fitness measures. We will show
the a very simple one, the accuracy of the minimal
distance to means (mdm) classifier outperforms all
other fitness measures.
We introduce a stopping criterion gleaned from numeric
mathematics. New features are only added if the
relative increase in training accuracy is more than a
constant d, for the mdm classifier estimated to be
3.3%.
%0 Conference Paper
%1 bot:2001:EuroGP
%A Bot, Martijn C. J.
%B Genetic Programming, Proceedings of EuroGP'2001
%C Lake Como, Italy
%D 2001
%E Miller, Julian F.
%E Tomassini, Marco
%E Lanzi, Pier Luca
%E Ryan, Conor
%E Tettamanzi, Andrea G. B.
%E Langdon, William B.
%I Springer-Verlag
%K Extraction, Feature Learning Machine algorithms, genetic programming,
%P 256--267
%T Feature Extraction for the k-Nearest Neighbour
Classifier with Genetic Programming
%U http://www.springerlink.com/openurl.asp?genre=article&issn=0302-9743&volume=2038&spage=256
%V 2038
%X In pattern recognition the curse of dimensionality can
be handled either by reducing the number of features,
e.g. with decision trees or by extraction of new
features.
We propose a genetic programming (GP) framework for
automatic extraction of features with the express aim
of dimension reduction and the additional aim of
improving accuracy of the k-nearest neighbour (k-NN)
classifier. We will show that our system is capable of
reducing most datasets to one or two features while
k-NN accuracy improves or stays the same. Such a small
number of features has the great advantage of allowing
visual inspection of the dataset in a two-dimensional
plot.
Since k-NN is a non-linear classification algorithm, we
compare several linear fitness measures. We will show
the a very simple one, the accuracy of the minimal
distance to means (mdm) classifier outperforms all
other fitness measures.
We introduce a stopping criterion gleaned from numeric
mathematics. New features are only added if the
relative increase in training accuracy is more than a
constant d, for the mdm classifier estimated to be
3.3%.
%@ 3-540-41899-7
@inproceedings{bot:2001:EuroGP,
abstract = {In pattern recognition the curse of dimensionality can
be handled either by reducing the number of features,
e.g. with decision trees or by extraction of new
features.
We propose a genetic programming (GP) framework for
automatic extraction of features with the express aim
of dimension reduction and the additional aim of
improving accuracy of the k-nearest neighbour (k-NN)
classifier. We will show that our system is capable of
reducing most datasets to one or two features while
k-NN accuracy improves or stays the same. Such a small
number of features has the great advantage of allowing
visual inspection of the dataset in a two-dimensional
plot.
Since k-NN is a non-linear classification algorithm, we
compare several linear fitness measures. We will show
the a very simple one, the accuracy of the minimal
distance to means (mdm) classifier outperforms all
other fitness measures.
We introduce a stopping criterion gleaned from numeric
mathematics. New features are only added if the
relative increase in training accuracy is more than a
constant d, for the mdm classifier estimated to be
3.3%.},
added-at = {2008-06-19T17:35:00.000+0200},
address = {Lake Como, Italy},
author = {Bot, Martijn C. J.},
biburl = {https://www.bibsonomy.org/bibtex/2f25ed7ae886ad11c66fe5b358776bbab/brazovayeye},
booktitle = {Genetic Programming, Proceedings of EuroGP'2001},
editor = {Miller, Julian F. and Tomassini, Marco and Lanzi, Pier Luca and Ryan, Conor and Tettamanzi, Andrea G. B. and Langdon, William B.},
interhash = {4776459684e88ae327c9ab586904d761},
intrahash = {f25ed7ae886ad11c66fe5b358776bbab},
isbn = {3-540-41899-7},
keywords = {Extraction, Feature Learning Machine algorithms, genetic programming,},
month = {18-20 April},
notes = {EuroGP'2001, part of \cite{miller:2001:gp}},
organisation = {EvoNET},
pages = {256--267},
publisher = {Springer-Verlag},
publisher_address = {Berlin},
series = {LNCS},
size = {12 pages},
timestamp = {2008-06-19T17:36:50.000+0200},
title = {Feature Extraction for the k-Nearest Neighbour
Classifier with Genetic Programming},
url = {http://www.springerlink.com/openurl.asp?genre=article&issn=0302-9743&volume=2038&spage=256},
volume = 2038,
year = 2001
}