Overview

Feature selection is the processing of selecting a subset of features for use in training a model.

Related Concepts

Feature Engineering/Extraction: Creating new features from existing ones based on domain knowledge

• Produces more features, should be performed before feature selection
• e.x. We use feature extraction to make many potentially useful features, and then feature selection to pick the best subset to improve a model’s performance.

Dimensionality Reduction: Reduces number of features by transforming data from a higher dimensional space into a lower dimensional space

• Similar purpose to feature selection
• Could be run after feature selection, but in practice, only one method is chosen usually.

Supervised Methods