What is Support Vector Machine?
Also known as: SVM
A Support Vector Machine (SVM) is a supervised machine learning algorithm mainly used for classification (and sometimes regression). It finds the optimal boundary, called a hyperplane, that separates data points of different classes with the maximum margin.
SVM works by identifying the hyperplane that best divides the classes in the feature space. The goal is to maximize the distance (margin) between the hyperplane and the nearest data points from each class, known as support vectors.
For data that is not linearly separable, SVM uses the kernel trick to map inputs into higher-dimensional spaces where a separating hyperplane can be found. Common kernels include linear, polynomial, and radial basis function (RBF).
Training involves solving an optimization problem that balances maximizing the margin while minimizing classification errors, often using techniques like soft margins for noisy data.
Example
Imagine sorting apples and oranges based on weight and color. An SVM would plot the fruits and draw the widest possible line separating the two groups, using only the apples and oranges closest to that line (the support vectors) to define it.
Why it matters
SVMs remain relevant for high-dimensional data tasks like text classification and bioinformatics due to their effectiveness with smaller datasets and strong theoretical foundations. They also influenced later kernel-based methods in modern AI.
Frequently asked questions
Support vectors are the data points closest to the decision boundary; they alone determine the position and orientation of the separating hyperplane.
Related terms
Logistic Regression is a supervised machine learning algorithm used for binary classification that estimates the probability an input belongs to a particular class.
A decision tree is a supervised machine learning model that predicts outcomes by recursively splitting data into branches based on feature values, forming a tree-like structure with decisions at internal nodes and final predictions at the leaves.
A Random Forest is an ensemble machine learning algorithm that builds many decision trees during training and combines their outputs to produce a more accurate and stable prediction.
K-Nearest Neighbors (KNN) is a simple supervised machine learning algorithm used for classification and regression that predicts the label or value of a new data point based on the majority vote or average of its K closest training examples.
Active learning is a machine learning technique where the model itself selects the most informative unlabeled data points to be labeled by a human, rather than labeling data randomly or all at once.
Adam (Adaptive Moment Estimation) is a popular optimization algorithm used to train machine learning models by iteratively updating parameters based on gradients.