Ensemble-Based Classifiers

This article discusses ensemble-based classifiers in machine learning, including their components, proposed AdaBoost approaches to huge database challenges, methods of combining the base-classifiers output, diversity for ensembles of classifiers, and similarities and differences between the diversities. The article is relevant to DSC540: Advanced Machine Learning and provides insights into the subject matter.

ENSEMBE-BASE CLASSFIERS
DSC540: ADVANCED MACHINE LEARNING
Name
Instructor
Date
ENSEMBE-BASED CLASSFIERS
Ensemble learning is a machine learning approach which is designed to solve problems by
training several learners. It tends to build a set of hypothesis and combine them to come up with
a new hypothesis.1
1. Components of ensemble machine learning
1 Tsai et al. "Intrusion detection by machine learning: A review." 11994.

2
I. Representation: This component of machine learning describes how knowledge can be
represented though graphs and other models.2
II. Evaluation: This component of machine learning which examines how machine
learning evaluates candidates or learners hypothesis with accuracy, predictions,
likelihoods, posterior probability and others.
III. Optimization: This component deals with how hypothesis are generated though a
searching process.
2. Proposed AdaBoost approaches to huge database challenges
AdaBoost criteria is adaptive in that classifiers that come following each other for an execution
are adjusted according to those cases that were classified wrongly by the previous classifiers.
Some of these adoptive measures are;3
 Adoption of new storage technologies. Big data complexities require adoption of efficient
data management techniques. New storage systems which do not actually have the same
spread performance is one of the major approaches to huge database management
challenges.
2 Dietterich, "An experimental comparison of three methods for constructing ensembles
of decision trees: Bagging, boosting, and randomization." 139.
3 Kodovský et al. "Ensemble classifiers for steganalysis of digital media." 439.

3
 Keeping track to the type of process adopted in processing and the conditions in which
the processes were performed. This is because data processing requires reproduction for
example in scientific applications
 Filtering and cleaning of data. Huge data requires filtering, cleansing as well as arranging
the data into different structure which are simply understandable by the computer storage
and analytic programs.
3. Methods of combining the base-classifiers output
a) Operating combination: Methods of combining data are sometimes classified
according to the level at which they operate. These include feature level and score
level.4
b) Output combination: This type of combination combines classifiers according to the
outputs of the classifiers in the combination. They include; abstract level, rank level
and measurement level.
c) Complex classifier combination: This method of classifiers combination classifies
data according to the structure of the combination functions. They are low, medium,
medium II and high complexity combinations.
d) Fixed classifiers combination: This type of combination categorizes classifiers on
the basis of whether the combinations are fixed or in large pool.
4 Dietterich, "Ensemble methods in machine learning." 15.

4
e) Score combination: This combination technique operates on the outputs of the
individual classifiers.
4. Diversity for ensembles of classifiers
Ensemble diversity in machine learning allows learners to achieve accurate results from
their data analysis which are not achievable in other models.5 For ensemble to be more
accurate, it must have a high accuracy base classifier with the highest level of diversity in
predictions. This includes feature selection and integration.6
5. Similarities and differences between the diversities
Similarities
One of the similarities is that both diversities are in position to deal with huge data in order to
provide a solution to a given problem at a particular period of time.
Differences
Ensemble classification does not require huge storage systems while nearest neighbor
classification requires large storage systems.
5 Tsai et al. "Intrusion detection by machine learning: A review." 11997.
6Kodovský et al. "Ensemble classifiers for steganalysis of digital media." 444.

5
Secondly, while ensemble classification diversity is static, nearest neighbor classification is
dynamic.
Bibliography
Dietterich Thomas. "An experimental comparison of three methods for constructing ensembles
of decision trees: Bagging, boosting, and randomization." Machine learning 40, no. 2,
2000: 139-157.

6
Dietterich Thomas. "Ensemble methods in machine learning." In International workshop on
multiple classifier systems, pp. 1-15. Springer, Berlin, Heidelberg, 2000.
Kodovský Jan, Jessica Fridrich, and Vojtech Holub. "Ensemble classifiers for steganalysis of
digital media." IEEE Trans. Information Forensics and Security 7, no. 2,2012: 432-444.
Tsai Chih-Fong, Yu-Feng Hsu, Chia-Ying Lin, and Wei-Yang Lin. "Intrusion detection by
machine learning: A review." Expert Systems with Applications 36, no. 10, 2009: 11994-
12000.