How modern self-supervised learning moved from negative pairs and augmentation invariance toward explicit representation regularization, latent prediction, and eventually predictive world models.

Self-supervised learning is built around a simple but powerful idea: instead of relying on human-provided labels, we can construct a learning objective directly from the structure of the data itself.

For computer vision, this means learning an encoder

\[f_\theta(x) \rightarrow h\]

that maps an image or video into a representation that can later be reused for classification, detection, retrieval, segmentation, or other downstream tasks.

The difficult part is not merely minimizing a self-supervised loss.

A model can sometimes minimize the objective while learning an entirely useless representation.

The canonical failure mode is representation collapse:

\[f_\theta(x_1) = f_\theta(x_2) = \cdots = f_\theta(x_n) = c\]

where every input is mapped to the same constant vector.

The loss may look good, but the representation contains essentially no information about the input.

A useful way to read the evolution of visual self-supervised learning is therefore to ask:

What prevents collapse, and what kind of information is the model actually being asked to learn?

This perspective provides a coherent path through SimCLR, MoCo, BYOL, DINO, MAE, VICReg, VISReg, I-JEPA, and V-JEPA 2.