There are two approaches to do Dimensionality REDUCTION:

1. Feature Selection
2. Feature EXTRACTION

Following are the DIFFERENT ways by which we can perform 'Feature Selection':

1. Filter Method
2. Wrapper Method
3. Embedded Method

In 'Feature Extraction' we reduce the data from a 'high dimensional space' to a lesser number of dimensions or 'lower-dimensional space'. The PROCESS of 'Dimensionality Reduction' can be linear or nonlinear. Several methods are used with  Dimensionality Reduction.

Some of these are:

1. PCA (Principal Component Analysis)
2. LCA (Linear Discriminant Analysis)
3. DCA (Generalized Discriminant Analysis)

When we are using the 'Principal Component Analysis', there is a requirement that the variance of the data which is in the 'lower-dimensional space' should be 'maximum'.  When it is being mapped to a 'lower-dimensional space' from a 'higher dimensional space'. The following steps are followed in the process of Principal Component

Analysis:

1. Constructing the Covariance Matrix of the given data.
2. Computing the EigenVectors of the computed matrix.
3. Reconstructing the variance of the original data by using the Eigen Vectors corresponding to the largest eigenvalue.

By using 'Linear Discriminant Analysis' we try to find such a linear combination of features that can SEPARATE the two or more classes of objects/events.

The 'Generalized  Discriminant Analysis' method is used to provide a mapping of the given 'input vectors' into a 'high dimensional feature space’.