Image Filter Playground

About

A digital image is a grid of numbers. A kernel is a much smaller grid, usually 3 × 3 or 5 × 5. To filter the image, slide the kernel over every pixel, multiply each number under it by the matching weight, and add the products up. That sum becomes the new pixel.

That one operation, repeated with different weights, gives you blur, sharpening, embossing and edge detection. It is also the core operation of a convolutional neural network. The difference is only where the weights come from: here you choose them; a CNN learns them from data.

The pixel inspector shows the arithmetic for a single pixel, using exactly the numbers the lab computes. Hover over the image (or tap on a phone) and watch the neighbourhood, the kernel and the sum change together.

Things to try

1. Find the edgesOn the Shapes sample, compare Sobel x and Sobel y. Each one misses the edges the other finds. Edge magnitude combines them.
2. Blur, then detectUse Street with Edge magnitude, then press A and set stage 1 to Gaussian blur, stage 2 to Edge magnitude. Fine texture disappears and the strong outlines stay.
3. FrequenciesOn the Zone plate, Gaussian blur wipes out the fine rings at the edge but keeps the broad centre. The Laplacian does the opposite.
4. Break brightnessMake a custom kernel whose weights sum to 2, then to 0. Sum 1 keeps brightness, sum 0 keeps only change.
5. Inspect an edgePark the inspector on the boundary of the yellow circle with Sobel x. One side of the window is large, the other small, and the weights turn that difference into a big number.

How it works

For an input image I and a k × k kernel K with radius r = (k − 1)/2, the output at column x, row y is:

Every output pixel is a weighted sum of the k² input pixels around it. The same weights are used at every position.

Here s is an optional scale such as 1/9 for a box blur, which keeps the weights readable as whole numbers. A 3 × 3 kernel costs 9 multiply-adds per pixel per channel; the stats bar counts them for the whole chain.

Convolution or correlation?

Strictly, mathematical convolution flips the kernel before sliding it (I(x − i, y − j) instead of I(x + i, y + j)). The formula above, without the flip, is cross-correlation. Deep learning libraries implement cross-correlation and call it convolution; PyTorch's Conv2d documentation says so explicitly. For symmetric kernels such as blurs and the Laplacian the two are identical. For Sobel, flipping only swaps the sign, which the absolute value hides.

Three practical details

Borders. At the edge of the image part of the window falls outside. This lab repeats the nearest edge pixel (clamp-to-edge). CNNs usually pad with zeros instead, which is why their feature maps often show a faint frame.

Output range. Weighted sums can be negative or larger than 255. Edge kernels produce both signs, so the lab shows the absolute value. Emboss and sharpen are clamped. A custom kernel can add 128 instead, making zero mid-grey so you can see both signs. The stats bar reports the raw range before mapping.

Colour. In colour mode each channel is filtered separately with the same kernel. Greyscale first converts to luma with the Rec. 601 weights 0.299 R + 0.587 G + 0.114 B, which is how most edge detectors are run in practice.

Kernel field guide

Identity

Copies each pixel unchanged. The baseline every other kernel departs from.

Box blur

Replaces each pixel with the plain average of its 3×3 neighbourhood. Weights are scaled by 1/9.

Gaussian blur

A weighted average that favours the centre. Smooth, without the boxy artefacts of a box blur. Weights are scaled by 1/256.

Sharpen

Identity plus a negative Laplacian: pushes each pixel away from its neighbours.

Sobel x

Right minus left. Bright where brightness changes horizontally, so it finds vertical edges.

Sobel y

Bottom minus top. Finds horizontal edges.

Edge magnitude

Runs Sobel x and Sobel y, then combines them as √(gx² + gy²): edges in every direction. Shown: the Sobel y half.

Laplacian

A second derivative. Zero on flat areas and on smooth ramps, strong at edges and fine detail.

Emboss

A diagonal difference plus the pixel itself, so edges look raised, as if lit from one side.

From filters to CNNs

In 1959 Hubel and Wiesel found neurons in the cat visual cortex that fire for an edge at one particular orientation in one small patch of the visual field. Each behaves much like a Sobel kernel parked at one position. Hand-designed edge detectors followed that intuition for decades: Sobel's 3 × 3 operator (1968), Marr and Hildreth's Laplacian of Gaussian (1980), Canny's detector (1986).

A convolutional network stacks many kernels and learns their weights by gradient descent. LeCun and colleagues trained such a network to read handwritten digits in 1998. When Krizhevsky, Sutskever and Hinton trained AlexNet on ImageNet in 2012, the 96 kernels in its first layer came out as oriented edge detectors and colour blobs, discovered from data rather than designed. Deeper layers combine them into textures, parts and eventually whole objects; Olah and colleagues traced this progression unit by unit in InceptionV1.

The filter chain in this lab is a very small version of the same idea. Stage 1 blurs, stage 2 finds edges on the blurred result. A CNN layer does the same thing with dozens of kernels at once, adds a non-linearity between layers, and learns every weight.

Continue with the Computer Vision lesson, see learned features at work in the Object Detection Lab, or read about pixel-level labelling in Image Segmentation.

Sources and further reading

1. 01Receptive fields of single neurones in the cat's striate cortex. Hubel and Wiesel, Journal of Physiology, 1959
2. 02Theory of edge detection. Marr and Hildreth, Proc. Royal Society B, 1980
3. 03A computational approach to edge detection. Canny, IEEE TPAMI, 1986
4. 04Gradient-based learning applied to document recognition. LeCun, Bottou, Bengio and Haffner, Proc. IEEE, 1998
5. 05ImageNet classification with deep convolutional neural networks. Krizhevsky, Sutskever and Hinton, NeurIPS 2012
6. 06Visualizing and understanding convolutional networks. Zeiler and Fergus, ECCV 2014
7. 07An overview of early vision in InceptionV1. Olah et al., Distill, 2020
8. 08A guide to convolution arithmetic for deep learning. Dumoulin and Visin, 2016
9. 09torch.nn.Conv2d. PyTorch documentation (notes that it computes cross-correlation)