Offensive Security Researcher & Quantum Cryptography Analyst

Particle Swarm Optimization (PSO) Algorithm: Working and Applications

Abdul Wahab Junaid
Particle Swarm Optimization algorithm and working of this algorithm.

Particle Swarm Optimization algorithm and working of this algorithm.

I find Particle Swarm Optimization (PSO) to be one of the most intuitive optimization algorithms I have worked with, because its inspiration is something I have actually watched with my own eyes: flocks of birds or schools of fish moving together, adjusting their direction based on their own experience and the experience of their neighbors. PSO is a population-based, gradient-free optimization technique where a swarm of candidate solutions, called particles, moves through the search space, guided by their own best-known position and the best-known position of the swarm as a whole. I use it for continuous, nonlinear, and often non-differentiable optimization problems where I don’t have or don’t want to rely on gradient information.

History and Background

I trace PSO back to 1995, when James Kennedy, a social psychologist, and Russell Eberhart, an electrical engineer, published “Particle Swarm Optimization” at the IEEE International Conference on Neural Networks. Kennedy’s original interest was in modeling social behavior and how ideas or norms spread through a population, and Eberhart’s background in engineering pushed the model toward a practical optimization tool. What began as an attempt to simulate the social behavior of flocks of birds searching for food evolved into a simplified mathematical model that turned out to be an effective optimizer in its own right. Over the following years, researchers introduced refinements like inertia weight (Shi and Eberhart, 1998) to better balance exploration and exploitation, and constriction factors (Clerc and Kennedy, 2002) to guarantee convergence behavior, both of which I consider standard parts of PSO today.

Problem Statement

I use PSO to solve optimization problems of the form: find $x^*$ that minimizes (or maximizes) an objective function $f(x)$ over a continuous search space, without requiring the gradient of $f$. Classical gradient-based methods can fail or get stuck when $f$ is non-convex, noisy, discontinuous, or expensive to differentiate. The problem PSO solves is finding a good approximate global optimum through a decentralized, cooperative search process, where many candidate solutions explore the space simultaneously and share information about what has worked well so far.

Core Concepts

How It Works

I run PSO through these steps:

  1. I initialize a swarm of particles with random positions and velocities within the search space bounds.
  2. I evaluate the objective function at each particle’s position.
  3. I update each particle’s personal best if its current position is better than its recorded personal best.
  4. I update the global best if any particle’s position is better than the current global best.
  5. I update each particle’s velocity using a combination of its previous velocity, the pull toward its personal best, and the pull toward the global best.
  6. I update each particle’s position by adding its new velocity.
  7. I repeat steps 2 through 6 for a fixed number of iterations or until convergence criteria are met.
  8. I return the global best position as the solution.

Working Principle

The internal mechanism I find most elegant about PSO is how it blends three forces acting on every particle: momentum (the tendency to keep moving in its current direction), personal memory (the pull back toward the best place it has individually found), and social influence (the pull toward the best place the whole swarm has found). No single particle needs to “know” where the optimum is — the swarm collectively discovers it through this constant negotiation between individual exploration and shared knowledge. Because there’s a stochastic component in how strongly each pull is applied each iteration (through random coefficients), particles don’t all collapse onto the same point immediately; they explore somewhat different paths, which helps the swarm avoid getting trapped too early in a poor region of the search space.

Mathematical Foundation

The velocity update rule for particle $i$, dimension $d$, at iteration $t+1$ is:

$$ v_{i,d}(t+1) = w \cdot v_{i,d}(t) + c_1 r_1 \left(pbest_{i,d} – x_{i,d}(t)\right) + c_2 r_2 \left(gbest_d – x_{i,d}(t)\right) $$

The position update rule is then:

$$ x_{i,d}(t+1) = x_{i,d}(t) + v_{i,d}(t+1) $$

where:

Clerc and Kennedy’s constriction factor variant reformulates this to guarantee convergence by introducing a constriction coefficient $\chi$:

$$ \chi = \frac{2}{\left|2 – \phi – \sqrt{\phi^2 – 4\phi}\right|}, \quad \phi = c_1 + c_2, \ \phi > 4 $$

$$ v_{i,d}(t+1) = \chi \left[v_{i,d}(t) + c_1 r_1 (pbest_{i,d} – x_{i,d}(t)) + c_2 r_2 (gbest_d – x_{i,d}(t))\right] $$

This constriction factor mathematically dampens velocity growth over time, ensuring the swarm does not diverge and instead converges to a stable point.

Diagrams

flowchart TD
    A["Initialize particles"] --> B["Evaluate fitness"]
    B --> C["Update personal best"]
    C --> D["Update global best"]
    D --> E["Update velocity"]
    E --> F["Update particle positions"]
    F --> G{"Stopping condition met?"}
    G -- No --> B
    G -- Yes --> H["Return best solution"]

Pseudocode

function PSO(f, num_particles, dimensions, max_iterations, w, c1, c2):
    for each particle i:
        position[i] = random position in search space
        velocity[i] = random small velocity
        pbest[i] = position[i]
        pbest_fitness[i] = f(position[i])

    gbest = position of particle with best pbest_fitness
    gbest_fitness = min(pbest_fitness)

    for iteration in 1 to max_iterations:
        for each particle i:
            fitness = f(position[i])
            if fitness

Step-by-Step Example

I will illustrate PSO minimizing $f(x) = x^2$ over a single dimension, with a tiny swarm of 3 particles, $w=0.5$, $c_1=c_2=1.5$.

Initial positions: $x_1=4, x_2=-3, x_3=6$; initial velocities all 0.

Iteration 1 (using illustrative $r_1=r_2=0.6$ for simplicity):

I can already see all particles pulling strongly toward the region near $x=-3$, and as iterations continue, fitness values shrink and gbest moves closer to the true minimum at $x=0$. Over more iterations, the swarm oscillates around zero with decreasing amplitude, eventually converging very close to $x=0$, $f(x)=0$.

Time Complexity

Each iteration requires evaluating the objective function for every particle, costing $O(P \cdot C)$ where $P$ is the number of particles and $C$ is the cost of one fitness evaluation. Updating velocity and position for every particle across all dimensions costs $O(P \cdot D)$ where $D$ is the number of dimensions. Across $T$ iterations, the total time is $O(T \cdot P \cdot (C + D))$. Since PSO typically needs relatively few particles (tens to low hundreds) compared to other population-based methods, and the update equations are cheap, most of the cost usually comes from the objective evaluation itself.

Space Complexity

I need to store, for every particle: its current position ($D$ values), its current velocity ($D$ values), and its personal best position ($D$ values), giving $O(P \cdot D)$ space overall. I also store the single global best position, which is $O(D)$ and negligible compared to the per-particle storage.

Correctness Analysis

I look at PSO’s correctness through the lens of convergence analysis rather than a guarantee of finding the global optimum. Clerc and Kennedy’s constriction factor analysis shows that under proper parameter settings ($\phi = c_1+c_2 > 4$, with $\chi$ computed as above), each particle’s trajectory converges to a stable point — specifically, a weighted average of its personal best and the global best — meaning the swarm will settle rather than diverge or oscillate indefinitely. However, this proves stability of convergence, not convergence to the true global optimum; the swarm can converge to a local optimum if the parameters or topology don’t maintain enough diversity. In practice, I rely on empirical benchmarking across standard test functions, and on techniques like random restarts or diversity-preserving neighborhood topologies, to increase confidence that PSO finds solutions near the true global optimum for a given problem class.

Advantages

Disadvantages

Applications

Implementation in C

#include 
#include 
#include 
#include 

#define NUM_PARTICLES 30
#define DIMENSIONS 2
#define MAX_ITER 100
#define W 0.7
#define C1 1.5
#define C2 1.5
#define LOWER_BOUND -10.0
#define UPPER_BOUND 10.0

double objective_function(double *x) {
    // Sphere function: f(x) = sum(x_i^2), minimum at origin
    double sum = 0.0;
    for (int i = 0; i  UPPER_BOUND) position[i][d] = UPPER_BOUND;
            }
        }

        printf("Iteration %d: gbest fitness = %f\n", iter, gbest_fitness);
    }

    printf("Best position found: (");
    for (int d = 0; d

Sample Input and Output

For the sphere function $f(x) = x_1^2 + x_2^2$ over $[-10,10]^2$, a typical run produces:

Iteration 0: gbest fitness = 3.214560
Iteration 20: gbest fitness = 0.048213
Iteration 60: gbest fitness = 0.000032
Iteration 99: gbest fitness = 0.000001
Best position found: (0.000452, -0.000317)

The swarm converges very close to the true global minimum at the origin, with fitness approaching zero.

Optimization Techniques

Common Mistakes

Further Reading

Exit mobile version