1) Decision tree
Decision tree methodology
is a usually used data mining method for founding classification systems based
on multiple covariates or for evolving prediction algorithms for a target
The basic concept of the
are three types of nodes. (Lu and Song, 2017)
A root hub, likewise called a choice hub, speaks
to a decision that will bring about the subdivision of all records into at
least two totally unrelated subsets.
Internal hubs, likewise called possibility hubs,
speak to one of the conceivable decisions accessible by then in the tree
structure, the best edge of the hub is associated with its parent hub and the
base edge is associated with its kid hubs or leaf hubs.
Leaf hubs, additionally called end hubs, speak to
the last aftereffect of a blend of choices or occasions.
2. Branches. (Lu and Song, 2017)
represent chance outcomes or occurrences that emanate from root nodes and
tree model is formed using a hierarchy of branches. Each path from the root
node through internal nodes to a leaf node represents a classification decision
decision tree pathways can also be represented as ‘if-then’ rules.
Splitting. (Lu and Song, 2017)
variables related to the target variable are used to split parent nodes into
purer child nodes of the target variable.
Both discrete input
variables and continuous input variables which are collapsed into two or more
categories can be used.
When building the
model one must first identify the most important input variables, and then
split records at the root node and at subsequent internal nodes into two or
more categories or ‘bins’ based on the status of these variables.
The type of the decision tree
Classification tree analysis is when the
predicted outcome is the class to which the data belongs.
Regression tree analysis is when the
predicted outcome can be considered a real number (e.g. the price of a house,
or a patient’s length of stay in a hospital).
2. Logistic Regression
Logistic regression is used to find the
probability of event=Success and event=Failure. We should use logistic
regression when the dependent variable is binary (0/ 1, True/ False, Yes/ No)
logistic model is charity to estimate the probability of a binary response based
on one or more predictor (or independent) variables (features).
one to say that the presence of a risk factor increases the odds of a given
outcome by a specific factor.
Logistic regression doesn’t require
linear relationship between dependent and independent variables. It can handle various types of relationships
because it applies a non-linear log transformation to the predicted odds ratio.
The type of logistic regression
1. Binary logistic regression (Wiley,2011)
used when the dependent variable is
dichotomous and the independent variables are either continuous or categorical.
dependent variable is not dichotomous and is comprised of more than two
categories, a multinomial logistic regression.
2. Multinomial Logistic Regression (Wiley,2011)
regression analysis to conduct when the dependent variable is nominal with more
than two levels. Thus it is an extension of logistic regression, which analyses
dichotomous (binary) dependents.
regression is used to describe data and to explain the relationship between one
dependent nominal variable and one or more continuous-level (interval or ratio
scale) independent variables.
The logistic regression does not assume a linear relationship between
the independent variable and dependent variable and it may handle nonlinear
effects. The dependent variable need not be normally distributed. It does not
require that the independents be interval and unbounded. Logistic regression
come at a cost, it requires much more data to achieve stable, meaningful
results. logistic regression come at a cost: it requires much more data to
achieve stable, meaningful results. With standard regression, and dependent
variable, typically 20 data points per predictor is considered the lower bound.
For logistic regression, at least 50 data points per predictor is necessary to
achieve stable results (Wiley,2011)
3) Neural Network
Neural network is a method of the computing,
based on the interaction of multiple connected processing elements. Ability to
deal with incomplete information. When an element of the neural network fails,
it can continue without any problem by their parallel nature.
(Liu, Yang and Ramsay, 2011)
Basic concept of the
neural network (Liu, Yang and Ramsay, 2011)
understanding and modelling operations of
single neurons or small neuronal circuits, e.g. minicolumns.
Modelling information processing in actual
brain systems, e.g. auditory tract.
Modelling human perception and cognition.
Artificial Neural Networks
Used in Pattern recognition, adaptive
control, time series prediction and etc.
areas contributing to Artificial neural networks are Statistical Pattern
recognition, Computational Learning Theory, Computational Neuroscience,
Dynamical systems theory and Nonlinear optimisation.
The type of neural
Feed-Forward neural network
There is the commonest type of neural
network in practical application. The first layer is the input and the last
layer is output.
If the is more than one hidden layer, we
call them ‘deep’ neural networks. They compute a series of transformation that
change the similarities between cases.
These have directed cycles in their
connection graph. That means you can sometimes get back to where you started by
following the arrows.
They can have complicated dynamic and this can
make them very difficult to train.
A neural network can perform tasks that a linear program cannot. A
neural network learns and does not need to be reprogrammed. It can be
implemented in any application. It can be implemented without any problem. Neural
networks requiring less formal statistical training, ability to implicitly
detect complex nonlinear relationships between dependent and independent
variables, ability to detect all possible interactions between predictor
variables, and the availability of multiple training algorithms. (JV,1996)
Nodes: Root node, Internal node, Leaf nodes
output can take
only two values, “0” and “1”, which represent outcomes
such as pass/fail and win/lose
Classification tree analysis
Regression tree analysis
Can quickly express complex alternatives
Can easily modify
Standard decision tree notation is easy to adopt.
It does not assume a linear relationship between the
independent variable and dependent variable
It may handle nonlinear effects
Requires much more data to achieve stable
Can perform tasks that a linear program cannot.
Requiring less formal statistical training
Ability to implicitly detect complex nonlinear
Ability to detect all