Offensive Security Researcher & Quantum Cryptography Analyst

What Is a Convolutional Neural Network (CNN) and How Does It Work?

Abdul Wahab Junaid
What is a convolutional neural network (CNN)?

If you’ve ever used Face ID to unlock your phone, had Google Photos automatically tag a picture of your dog, or watched a self-driving car demo spot a pedestrian, you’ve seen a Convolutional Neural Network (CNN) at work. CNNs are the backbone of modern computer vision, and understanding how they work is one of the most valuable things you can learn in deep learning.

In this article, I’ll break down what a CNN actually is, why it was invented, the math behind convolution, and how to build one in code — going from beginner intuition all the way to architectural details used in production systems.

Table of Contents

  1. Why Regular Neural Networks Struggle with Images
  2. What Is a CNN?
  3. The Convolution Operation Explained
  4. Key Components of a CNN Architecture
  5. The Math Behind Convolution
  6. Pooling Layers
  7. Putting It All Together: A Full CNN Architecture
  8. Popular CNN Architectures
  9. CNN Implementation in Python (Keras/TensorFlow)
  10. Advantages and Disadvantages
  11. Real-World Applications
  12. Best Practices
  13. Summary

1. Why Regular Neural Networks Struggle with Images

A standard feed-forward neural network treats every input pixel as an independent feature, connected to every neuron in the next layer. For a small 100×100 grayscale image, that’s already 10,000 input values. Connect that to a modest hidden layer of 1,000 neurons, and you have 10 million weights — for one tiny image, in one layer.

This approach has two major flaws:

CNNs were designed specifically to solve both problems.

2. What Is a CNN?

A Convolutional Neural Network is a type of deep learning architecture designed to process data with a grid-like topology — most commonly images. Instead of connecting every neuron to every pixel, a CNN uses small, learnable filters (kernels) that slide across the image, detecting local patterns like edges, textures, and shapes. As data moves through deeper layers, these simple patterns combine into more complex ones — edges become shapes, shapes become object parts, and object parts become full objects.

This idea was inspired by how the visual cortex in animals processes information — small groups of neurons respond to specific regions of the visual field, a concept formalized in deep learning by Yann LeCun’s LeNet architecture in the late 1990s.

3. The Convolution Operation Explained

At the heart of a CNN is the convolution operation. Imagine sliding a small magnifying glass (a filter, typically 3×3 or 5×5 pixels) across an image, one step at a time. At each position, the filter multiplies its values with the pixel values underneath it, sums the result, and produces a single number. That number becomes one pixel in the output, called a feature map.

flowchart TD
    A[Input Image
e.g. 32x32x3] --> B[Convolution Layer
Apply Filters/Kernels]
    B --> C[Activation Function
ReLU]
    C --> D[Pooling Layer
Downsample]
    D --> E{More Conv Blocks?}
    E -->|Yes| B
    E -->|No| F[Flatten]
    F --> G[Fully Connected Layer]
    G --> H[Output: Class Probabilities]

Different filters detect different features — one might activate strongly on vertical edges, another on horizontal edges, another on color blobs. Crucially, the CNN learns these filter values during training rather than having them hand-designed.

4. Key Components of a CNN Architecture

ComponentPurpose
Convolutional LayerExtracts local features using learnable filters
Activation Function (ReLU)Introduces non-linearity
Pooling LayerReduces spatial dimensions, adds translation invariance
Fully Connected LayerCombines features for final classification
Softmax/Output LayerConverts outputs into class probabilities

5. The Math Behind Convolution

For a 2D input image $I$ and a filter (kernel) $K$, the convolution operation at position $(i, j)$ is defined as:

$$ S(i,j) = (I * K)(i,j) = \sum_{m} \sum_{n} I(i+m, j+n) \cdot K(m,n) $$

Where $m$ and $n$ range over the height and width of the kernel.

The output feature map’s spatial size depends on the input size, filter size, stride, and padding:

$$ O = \frac{W – F + 2P}{S} + 1 $$

Where:

For example, a 32×32 input with a 5×5 filter, no padding ($P=0$), and stride 1 produces an output of:

$$ O = \frac{32 – 5 + 0}{1} + 1 = 28 $$

So the output feature map would be 28×28.

Stride and Padding

6. Pooling Layers

Pooling layers reduce the spatial size of feature maps, which cuts computation and helps the network become more robust to small shifts or distortions in the input.

Max Pooling takes the maximum value in each region:

$$ P(i,j) = \max_{(m,n) \in R_{ij}} S(m,n) $$

Average Pooling takes the mean value instead:

$$ P(i,j) = \frac{1}{|R_{ij}|}\sum_{(m,n) \in R_{ij}} S(m,n) $$

Max pooling is more common in practice because it tends to preserve the strongest, most distinctive features.

7. Putting It All Together: A Full CNN Architecture

A typical CNN for image classification looks like this:

  1. Input — raw image (e.g., 224x224x3 for RGB).
  2. Conv Block 1 — convolution + ReLU + pooling, extracting low-level features (edges, colors).
  3. Conv Block 2, 3, … — deeper blocks extract increasingly abstract features (textures, shapes, object parts).
  4. Flatten — convert the final feature maps into a 1D vector.
  5. Fully Connected Layers — combine features to make a final decision.
  6. Output Layer — softmax activation producing class probabilities.

7b. Receptive Fields: What Each Neuron “Sees”

A crucial concept for understanding CNN depth is the receptive field — the region of the original input image that influences a given neuron’s activation. A neuron in the first convolutional layer might only “see” a 3×3 patch of pixels. But a neuron in the fifth layer, after several rounds of convolution and pooling, effectively “sees” a much larger region of the original image, because it’s built from neurons that were each built from smaller local regions in the layer before.

This is why stacking small filters (like 3×3) is generally preferred over using a few large filters (like 11×11): two stacked 3×3 convolutional layers achieve the same receptive field as a single 5×5 layer, but with fewer parameters and an extra non-linearity (ReLU) in between, which gives the network more expressive power for the same computational budget. This insight, popularized by the VGGNet architecture, is one of the most influential design principles in modern CNN design.

7c. Feature Map Visualization: What Filters Actually Learn

One of the most illuminating things you can do when learning about CNNs is visualize what the filters in different layers actually detect. Research using techniques like activation maximization and Grad-CAM has consistently shown a clear pattern:

This hierarchical structure is precisely why transfer learning works so well with CNNs — the early and middle layers, trained on a large generic dataset like ImageNet, learn features general enough to be reused for almost any other vision task, so only the later, task-specific layers typically need retraining.

8. Popular CNN Architectures

ArchitectureYearKey Innovation
LeNet-51998First practical CNN, used for digit recognition
AlexNet2012Deeper network, ReLU, dropout, GPU training
VGGNet2014Very deep, uniform 3×3 filters
GoogLeNet (Inception)2014Multi-scale filters within a single layer
ResNet2015Residual (skip) connections enabling very deep networks
EfficientNet2019Scaling depth, width, and resolution together efficiently

9. CNN Implementation in Python (TensorFlow/Keras)

import tensorflow as tf
from tensorflow.keras import layers, models

model = models.Sequential([
    layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)),
    layers.MaxPooling2D((2, 2)),
    layers.Conv2D(64, (3, 3), activation='relu'),
    layers.MaxPooling2D((2, 2)),
    layers.Conv2D(64, (3, 3), activation='relu'),
    layers.Flatten(),
    layers.Dense(64, activation='relu'),
    layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.summary()
history = model.fit(train_images, train_labels, epochs=10,
                     validation_data=(test_images, test_labels))

This small network — three convolutional blocks followed by two dense layers — is enough to reach strong accuracy on datasets like CIFAR-10.

9b. Transfer Learning: Reusing Pretrained CNNs

Given how expensive it is to train a large CNN from scratch, most practical applications today start from a pretrained model — one already trained on a massive dataset like ImageNet (1.2 million images, 1,000 classes) — and adapt it to a new, smaller dataset. This is called transfer learning, and it typically follows one of two strategies:

import tensorflow as tf
from tensorflow.keras import layers, models

base_model = tf.keras.applications.ResNet50(
    weights='imagenet', include_top=False, input_shape=(224, 224, 3)
)
base_model.trainable = False  # freeze pretrained layers for feature extraction

model = models.Sequential([
    base_model,
    layers.GlobalAveragePooling2D(),
    layers.Dense(128, activation='relu'),
    layers.Dense(5, activation='softmax')  # 5 new classes
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

This approach routinely achieves strong accuracy with just a few hundred to a few thousand labeled images per class, compared to the millions typically needed to train a comparable CNN entirely from scratch.

10. Advantages and Disadvantages

Advantages:

Disadvantages:

11. Real-World Applications

12. Best Practices

13. Summary

CNNs revolutionized computer vision by replacing brute-force, fully connected layers with local, shared filters that respect the spatial structure of images. Through the combination of convolution, activation, and pooling layers, a CNN progressively builds up an understanding of an image — from raw pixels, to edges, to shapes, to full objects. Whether you’re building a simple digit classifier or fine-tuning a state-of-the-art ResNet, the core principles covered here — convolution math, pooling, and layered feature extraction — remain the same.

12b. Data Augmentation: Getting More Out of Limited Data

Because CNNs generally need substantial data to reach their full potential, data augmentation — artificially expanding a training set by applying realistic transformations to existing images — is a standard part of almost every CNN training pipeline. Common augmentations include random rotations, horizontal flips, crops, brightness/contrast adjustments, and slight zooms.

from tensorflow.keras.preprocessing.image import ImageDataGenerator

datagen = ImageDataGenerator(
    rotation_range=20,
    width_shift_range=0.1,
    height_shift_range=0.1,
    horizontal_flip=True,
    zoom_range=0.15,
    brightness_range=[0.8, 1.2]
)

train_generator = datagen.flow(train_images, train_labels, batch_size=32)
model.fit(train_generator, epochs=20)

The intuition is straightforward: a CNN trained only on upright, centered photos of cats might fail on a photo of a cat lying sideways, even though the underlying object is identical. By exposing the network to many transformed versions of the same image during training, it learns to be robust to exactly the kind of variation it will encounter at inference time, effectively multiplying the value of a limited dataset without collecting a single new image.

13b. Beyond Classification: Other CNN Tasks

While this article has focused mainly on image classification, CNNs form the backbone of several other major computer vision tasks, each adapting the basic convolutional architecture in a different way. Object detection models (like YOLO and Faster R-CNN) extend a CNN backbone with additional layers that predict both class labels and bounding box coordinates for multiple objects in a single image. Semantic segmentation models (like U-Net) use an encoder-decoder CNN structure to classify every individual pixel in an image rather than the image as a whole, which is essential in applications like medical imaging where the exact boundary of a tumor matters, not just its presence. Image generation models, including GANs and diffusion models, often use CNN-based architectures in reverse — starting from noise and progressively building up a full image through learned convolutional transformations. Understanding the core convolution and pooling operations covered here is the foundation for all of these more specialized architectures.

13c. A Closing Thought on Intuition vs. Implementation

It’s easy to get lost in filter sizes, stride formulas, and framework syntax when learning CNNs, so it’s worth stepping back to the core intuition one more time: a CNN succeeds because it builds understanding the same way a person scanning a photo does — first noticing edges and colors, then shapes, then recognizable parts, then the whole object. Every architectural choice covered in this article — shared filters, pooling, stacked layers, skip connections in ResNet — exists to make that hierarchical, local-to-global pattern of understanding both computationally efficient and mathematically trainable at scale. If a particular design choice ever seems confusing, it’s often useful to ask which part of that human-like visual hierarchy it’s trying to preserve or improve.

References

Exit mobile version