Browsing Tag
neural network
57 posts
What is the Purpose of the Kullback-Leibler (KL) Divergence Loss Function
I still remember the first time I ran into Kullback-Leibler divergence. I was debugging a variational autoencoder that…
Explaining the Concept of Binary Cross-Entropy Loss
Binary cross-entropy was the very first loss function I truly understood on a gut level, mostly because I…
What is Categorical Cross-Entropy Loss and When Is It Used
The first time I trained a multi-class image classifier — ten digits, zero through nine, on the MNIST…
How Is the Mean Squared Error (MSE) Loss Function Defined
Mean squared error was the first loss function I ever encountered — long before I even knew what…
Common Types of Loss Functions Used in Neural Networks
When I first started building neural networks, I treated the loss function as a box I didn’t need…
What Is the Hyperbolic Tangent (Tanh) Activation Function Used For
Before ReLU took over the deep learning world, tanh was one of my go-to activation functions, especially for…
Explaining the Concept of the Softmax Activation Function
Softmax was one of those functions I used correctly for a long time before I actually understood why…
What Is the Purpose of the Sigmoid Activation Function
Sigmoid was the very first activation function I learned, back when I was building a simple logistic regression…
How Does the Rectified Linear Unit (ReLU) Activation Function Work
I remember reading a paper years ago that credited ReLU with helping make training genuinely deep networks practical,…
Popular Activation Functions Used in Neural Networks
For a long time, I thought of activation functions as an afterthought — a small detail you plug…