What is Decision Tree?
A decision tree is a supervised machine learning model that predicts outcomes by recursively splitting data into branches based on feature values, forming a tree-like structure with decisions at internal nodes and final predictions at the leaves.
It works by selecting the best feature to split on at each step, using metrics like information gain or Gini impurity to maximize separation between classes or reduce variance in regression.
The tree grows until stopping criteria are met, such as maximum depth or minimum samples per leaf, and can be pruned afterward to reduce overfitting.
Decision trees support both classification (predicting categories) and regression (predicting numbers) and are valued for their transparency since the path from root to leaf shows exactly how a prediction is made.
Example
To predict whether a customer will buy a phone, a decision tree might first split on 'age > 30', then on 'income > $50k' for one branch, eventually reaching a leaf that says 'yes' or 'no' based on the path taken.
Why it matters
Decision trees are foundational in modern AI because they are highly interpretable and serve as building blocks for powerful ensemble methods like random forests and gradient boosting that dominate many real-world applications.
Frequently asked questions
It evaluates each feature using criteria like information gain or Gini impurity and picks the one that best separates the data at that node.
Related terms
A Random Forest is an ensemble machine learning algorithm that builds many decision trees during training and combines their outputs to produce a more accurate and stable prediction.
Gradient Boosting is an ensemble machine learning technique that builds a strong predictive model by sequentially adding weak learners, typically decision trees, that correct the errors of the previous models.
Classification is a supervised machine learning task that assigns input data to one of several predefined categories or classes based on patterns learned from labeled training examples.
Regression is a supervised machine learning method that predicts continuous numerical values from input features.
Overfitting happens when a machine learning model learns the training data too closely, including its noise and quirks, so it fails to perform well on new, unseen data.
Active learning is a machine learning technique where the model itself selects the most informative unlabeled data points to be labeled by a human, rather than labeling data randomly or all at once.