Total Parameters
134.3M
~124M in classifier FC layers
Conv Layers
13
All 3×3 kernels, stride 1, pad 1
Baseline FLOPs
~15.5G
At 224×224 input resolution
Max Pooling
5 layers
2×2 kernel, stride 2 → halves spatial dims
Classifier
3 FC layers
4096 → 4096 → 10 (adapted from 1000)
Pruning Ratios
30 / 50 / 70%
Filter removal per Conv2d layer
A conv layer, up close
A conv layer takes a 3-D volume in (channels × H × W) and produces another 3-D volume out. The "weights" of the layer live in a collection of small filters (also called kernels). Each filter slides over the input volume and produces one 2-D feature map. Stack the feature maps from all filters together, and that's the output.
INPUT VOLUME
C_in × H × W
e.g. 128 × 56 × 56
⊛
256 FILTERS
each is C_in × 3 × 3
= 128 × 3 × 3
→
OUTPUT VOLUME
C_out × H × W
= 256 × 56 × 56
The weights of this conv layer are exactly the 256 filters × (128 × 3 × 3) numbers = 256 × 128 × 3 × 3 = 294,912 weights. Each filter is one learned pattern detector. The same filter is reused at every spatial position when it slides over the input — that's "weight sharing", and it's why conv layers have orders of magnitude fewer weights than FC layers.
An FC layer, up close
An FC ("fully connected") layer is just a matrix multiplication. The 3-D volume from the last conv block has been flattened into one long vector, and the first FC layer turns that vector into a smaller vector by multiplying it with a weight matrix. Every output neuron is a weighted sum of every input neuron — no spatial sliding, no kernels, no sharing.
x₁
x₂
x₃
x₄
x₅
x₆
⋮
INPUT VECTOR x
25,088 numbers
(= 512 × 7 × 7 flattened)
MATRIX W
4096 × 25,088
~100 million weights!
y₁
y₂
y₃
y₄
y₅
y₆
⋮
OUTPUT VECTOR y
4096 numbers
y = W · x
Each output neuron yᵢ is computed as yᵢ = W[i, :] · x — that is, row i of the matrix dot-producted with the full input vector. So neuron yᵢ "looks at" every single input value, weighted by its row of W.
"Filters" vs "neurons" — same idea, different shape
People throw around both words and it gets confusing. Here's the simple distinction:
FILTER conv layer
- A small 3-D kernel (C_in × 3 × 3) of learned weights.
- Slides across the input, applying the same weights at every spatial position.
- Produces one 2-D feature map as output.
- A conv layer with 256 filters has 256 feature maps in its output → those 256 output channels = 256 "filters". The words are interchangeable.
- When we "prune a filter", we remove that whole kernel + the whole feature map it produces. The next layer gets one fewer input channel.
NEURON FC layer
- A single output unit: one row of the weight matrix W.
- No spatial sharing — it sees the entire flattened input vector at once.
- Produces one scalar number as output.
- The second FC layer (the FRL for NISP) has 4096 neurons → 4096 rows in W → 4096 scalar outputs.
- "Pruning a neuron" would mean removing one row of W (we don't prune FC neurons in this benchmark — only conv filters).
Why a conv "filter" is sometimes called a "neuron": in a strict mathematical sense, every output number in a feature map is one neuron's output. A 56×56 feature map technically has 56·56 = 3,136 neurons that all share the same weights (the filter). So conv filters are weight-shared neurons. When the literature says "filter", it means the shared weight kernel; when it says "channel" or "feature map", it means the output that kernel produces. They refer to the same indexed thing.
Where the 134 M weights actually live
Most people are surprised by this: over 100 million of VGG-16's 134 M weights sit in the first FC layer alone, because flattening a 512×7×7 volume into a 25,088-long vector and feeding it into 4096 neurons needs a 4096 × 25,088 matrix. Conv filters look big but each only needs C_in × 9 numbers.
Bars scaled to fraction of 134.3 M total parameters. Conv block totals lump 2–3 layers together for readability.
Important nuance for pruning: the benchmark only prunes conv filters, not FC neurons. So even at 70% pruning the FC layers stay full-size. But removing conv filters also shrinks the input dimension of FC1 (because the last conv block produces fewer feature maps to flatten), which is where the real parameter savings come from in practice.