When I first started training neural networks, I assumed that more training always meant a better model. Then I ran into a wall — my model’s accuracy plateaued at a disappointingly low number, and no amount of extra epochs helped. That was my first real encounter with underfitting. It’s one of the two classic failure modes in machine learning (the other being overfitting), and understanding it deeply is essential for anyone building deep learning systems.
In this article, I’ll walk through what underfitting actually is, why it happens, the mathematics behind it, how to detect it, and — most importantly — how to fix it. I’ll use plain-language explanations first, then build up to the technical depth needed for real-world model debugging.
What Is Underfitting?
Underfitting occurs when a machine learning model is too simple to capture the underlying pattern in the data. The model performs poorly not just on unseen data but also on the training data itself. In other words, an underfit model hasn’t even learned the training set properly, let alone generalized to new examples.
Think of it like trying to draw a straight line through a set of points that clearly follow a curve. No matter how you rotate or shift that line, it will never fit the curve well. That’s underfitting in a nutshell: the model’s capacity (its “hypothesis space”) is too limited to represent the true relationship between inputs and outputs.
A Simple Analogy
Imagine a student preparing for an exam by only skimming the textbook’s table of contents. They walk into the exam with almost no understanding of the material, so they perform poorly on both practice questions (analogous to the training set) and the real exam (the test set). That’s an underfit “model” — under-trained, under-capacity, and unable to capture the necessary complexity of the subject.
The Bias-Variance Tradeoff
Underfitting is best understood through the lens of the bias-variance tradeoff, a foundational concept in statistical learning theory.
- Bias refers to the error introduced by approximating a real-world problem, which may be complex, with a simplified model. High bias means the model makes strong assumptions about the data (e.g., assuming a linear relationship when the true relationship is nonlinear).
- Variance refers to how much the model’s predictions change if trained on a different subset of data. High variance means the model is overly sensitive to the training data.
Underfitting is associated with high bias and low variance. The model is consistently wrong in a predictable way because it lacks the flexibility to learn the true underlying function.
Mathematically, the expected generalization error of a model can be decomposed as:
$$ \mathbb{E}[(y – \hat{f}(x))^2] = \text{Bias}[\hat{f}(x)]^2 + \text{Var}[\hat{f}(x)] + \sigma^2 $$
where $\sigma^2$ is the irreducible noise in the data. In an underfitting scenario, the $\text{Bias}[\hat{f}(x)]^2$ term dominates the total error.
Mathematical Formulation
Consider a neural network trying to approximate a true function $f(x)$ using a hypothesis $\hat{f}(x; \theta)$, parameterized by weights $\theta$. The training objective is to minimize a loss function, typically mean squared error for regression:
$$ \mathcal{L}(\theta) = \frac{1}{N} \sum_{i=1}^{N} \left( y_i – \hat{f}(x_i; \theta) \right)^2 $$
If the network’s architecture (number of layers, neurons, or the choice of activation function) restricts the hypothesis space such that no parameter setting $\theta$ can approximate $f(x)$ well, then $\mathcal{L}(\theta)$ remains high even at the global minimum achievable by that architecture. This is a capacity problem, not an optimization problem — even a perfect optimizer can’t fix underfitting caused by insufficient model capacity.
Contrast this with overfitting, where $\mathcal{L}(\theta)$ can be driven very low on the training set but the model fails to generalize.
Symptoms of Underfitting
You can usually diagnose underfitting by observing training curves:
| Symptom | Underfitting Indicator |
|---|---|
| Training loss | High and plateaus early |
| Validation loss | High, close to training loss |
| Training accuracy | Low |
| Validation accuracy | Low, similar to training accuracy |
| Gap between train/val performance | Small (both are bad) |
This last point is the key differentiator from overfitting. In overfitting, training performance is excellent while validation performance is poor — there’s a large gap. In underfitting, both are poor, and the gap is small because the model simply hasn’t learned enough to distinguish between the two sets.
Common Causes of Underfitting
1. Insufficient Model Capacity
A neural network with too few layers or too few neurons per layer may lack the representational power to model complex patterns. For example, using a single-layer perceptron to classify data that is not linearly separable (like the classic XOR problem) will always underfit, no matter how it’s trained.
2. Excessive Regularization
Regularization techniques like L1/L2 penalties, dropout, or early stopping are designed to prevent overfitting. But applied too aggressively, they can suppress the model’s ability to learn even the genuine signal in the data.
The L2 regularization term added to the loss looks like:
$$ \mathcal{L}{\text{reg}}(\theta) = \mathcal{L}(\theta) + \lambda \sum{j} \theta_j^2 $$
If $\lambda$ (the regularization strength) is set too high, the penalty term dominates the loss, forcing weights toward zero and crippling the model’s expressiveness.
3. Too Few Training Epochs
If training is stopped before the model converges, it may not have had enough time to learn the patterns present in the data — especially with complex architectures that need many iterations to find good weight configurations.
4. Overly Aggressive Learning Rate
A learning rate that’s too high can cause the optimizer to overshoot minima repeatedly, preventing convergence, which can look like underfitting even though the model has sufficient capacity.
5. Poor Feature Representation
If the input features don’t contain enough information to predict the target, no model — regardless of size — can fit the data well. This is a data problem rather than a model problem.
6. Inappropriate Architecture Choices
Using a fully connected network for image data, instead of a convolutional network that can exploit spatial structure, often leads to underfitting on vision tasks because the architecture isn’t suited to the data’s structure.
Visualizing Underfitting
Below is a simple diagram illustrating where underfitting falls relative to a good fit and overfitting, in terms of model complexity versus error.
graph LR
A[Low Model Complexity] -->|Underfitting: High Bias| B[Optimal Complexity]
B -->|Well-Fit Model: Balanced Bias/Variance| C[High Model Complexity]
C -->|Overfitting: High Variance| D[Poor Generalization]
A -.High Training & Validation Error.-> A
D -.Low Training Error, High Validation Error.-> D
How to Detect Underfitting in Practice
- Plot learning curves. Track training and validation loss/accuracy across epochs. If both curves plateau at a poor performance level, that’s a strong sign of underfitting.
- Compare to a baseline. If a simple baseline model (e.g., logistic regression) performs similarly to your neural network, your network likely isn’t leveraging its capacity effectively.
- Check training accuracy specifically. If your model can’t even memorize the training set, that’s a clear underfitting signal, since a sufficiently large network should be able to overfit small datasets.
Fixing Underfitting: Practical Strategies
Increase Model Capacity
Add more layers or more neurons per layer. In convolutional networks, this might mean adding more filters or increasing depth. In transformers, this could mean increasing the number of attention heads or the embedding dimension.
Reduce Regularization
If dropout rate, L2 penalty, or weight decay is too aggressive, dial it back. For example, reducing dropout from 0.5 to 0.2 can allow the network to retain more of its learned representations during training.
Train Longer
Increase the number of epochs, provided you’re monitoring validation performance to catch the point where overfitting might begin.
Use a Better Optimizer or Tune the Learning Rate
Adaptive optimizers like Adam or RMSprop often converge faster and more reliably than vanilla SGD, especially for tricky loss landscapes. Learning rate schedules (like cosine annealing or warm restarts) can also help the model escape poor local regions.
Engineer Better Features
If the raw inputs don’t carry enough signal, consider feature engineering, using pretrained embeddings, or applying data augmentation techniques to expose the model to more informative variations of the input.
Choose an Architecture Suited to the Data
Use convolutional networks for image data, recurrent networks or transformers for sequential data, and graph neural networks for graph-structured data. Matching architecture to data structure is often more impactful than simply adding parameters.
Code Example: Diagnosing and Fixing Underfitting in PyTorch
import torch
import torch.nn as nn
import torch.optim as optim
# An underfit model: too small for a complex dataset
class UnderfitModel(nn.Module):
def __init__(self):
super().__init__()
self.fc = nn.Linear(20, 1) # Single linear layer, no nonlinearity
def forward(self, x):
return self.fc(x)
# A fixed model: added capacity and nonlinearity
class ImprovedModel(nn.Module):
def __init__(self):
super().__init__()
self.net = nn.Sequential(
nn.Linear(20, 128),
nn.ReLU(),
nn.Linear(128, 64),
nn.ReLU(),
nn.Linear(64, 1)
)
def forward(self, x):
return self.net(x)
def train(model, X_train, y_train, epochs=200, lr=1e-3):
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=lr)
for epoch in range(epochs):
optimizer.zero_grad()
preds = model(X_train)
loss = criterion(preds, y_train)
loss.backward()
optimizer.step()
if epoch % 50 == 0:
print(f"Epoch {epoch}, Loss: {loss.item():.4f}")
return model
Running the UnderfitModel on a nonlinear dataset will show the loss plateauing quickly at a high value. Switching to ImprovedModel, which has added depth and nonlinearity via ReLU activations, typically results in a substantially lower and still-decreasing loss.
Underfitting vs. Overfitting: Quick Comparison
| Aspect | Underfitting | Overfitting |
|---|---|---|
| Model complexity | Too low | Too high |
| Training error | High | Low |
| Validation error | High | High |
| Bias | High | Low |
| Variance | Low | High |
| Fix | Increase capacity, reduce regularization | Add regularization, get more data |
Advantages of Understanding Underfitting
Recognizing and addressing underfitting isn’t just about fixing a broken model — it builds a deeper intuition for model capacity, the bias-variance tradeoff, and the importance of matching architecture to problem complexity. This intuition transfers across nearly every machine learning project.
Limitations and Nuances
- It’s not always a fix-by-adding-capacity problem. Sometimes underfitting stems from bad data quality or mislabeled examples, and no architecture change will help.
- Increasing capacity has costs. Bigger models take longer to train, require more compute and memory, and are more prone to overfitting if not managed carefully — so fixing underfitting can inadvertently introduce a new problem if taken too far.
- Underfitting can be task-dependent. A model that underfits one dataset might be perfectly appropriate — or even overfit — on another, smaller or simpler dataset.
Real-World Use Cases and Examples
- Medical imaging: A shallow CNN trained on X-ray images to detect pneumonia may underfit if it doesn’t have enough convolutional layers to capture the fine-grained texture differences between healthy and diseased lung tissue.
- Financial forecasting: Using a simple linear regression to predict stock prices, which are governed by highly nonlinear and noisy dynamics, is a textbook underfitting scenario.
- Natural language processing: An overly small embedding dimension in a text classification model may fail to capture the semantic nuance needed to distinguish between similar categories, leading to underfitting.
Best Practices Checklist
- Start with a slightly over-parameterized model, then apply regularization as needed — it’s often easier to fix overfitting than underfitting.
- Always plot learning curves before drawing conclusions about model performance.
- Use learning rate finders or schedules rather than guessing a fixed learning rate.
- Validate that your model can overfit a tiny subset of the data (a sanity check that confirms your architecture and training pipeline are functioning correctly).
- Iterate: adjust one variable at a time (capacity, regularization, learning rate) so you can isolate what’s helping or hurting.
Summary
Underfitting happens when a model is too simple, too constrained, or too under-trained to capture the true patterns in data, resulting in poor performance on both training and validation sets. It’s characterized by high bias and low variance, and it can be diagnosed by examining learning curves and comparing training versus validation performance. Fixing it generally involves increasing model capacity, reducing excessive regularization, training longer, tuning the learning rate, or choosing an architecture better suited to the data. Understanding underfitting alongside its counterpart, overfitting, is essential to building neural networks that generalize well to real-world data.
References
- Goodfellow, I., Bengio, Y., & Courville, A. Deep Learning. MIT Press. https://www.deeplearningbook.org/
- Hastie, T., Tibshirani, R., & Friedman, J. The Elements of Statistical Learning. https://hastie.su.domains/ElemStatLearn/
- PyTorch Official Documentation. https://pytorch.org/docs/stable/index.html
- Bishop, C. M. Pattern Recognition and Machine Learning. Springer.