This page collects papers related to contrastive learning to help understand contrastive learning.

related project

Not yet

What is Contrastive Learning?

Contrastive Learning, It’s a type of SSL(self-supervised learning). See at wikipedia. Contrastive Learning use both positive and negative sammples to learn.

Which paper recommand to read? (wrote as ISO 690 format)

Use Data augmentation is important in this paper. Non-linear projection head enhances the quality of expression of previous layers.

• CHEN, Ting, et al. A simple framework for contrastive learning of visual representations. In: International conference on machine learning. PMLR, 2020. p. 1597-1607.

Improve CPC(Contrastive Predictive Coding), CPC enables data-efficient learning. The CPC method presented in the paper also improved performance on the imagenet scale.

• HENAFF, Olivier. Data-efficient image recognition with contrastive predictive coding. In: International conference on machine learning. PMLR, 2020. p. 4182-4192.

Suggest AMDIM(Augmented Multiscale DIM) that improve DIM(Deep InfoMax). Original DIM using a method for learining representations by maximizing the mutual information between the input and the output of a deep neural network encoder. But, AMDIM propose an approach to self-supervised representation learning by maximizing the mutual information between features extracted from multiple views of a shared context.

• BACHMAN, Philip; HJELM, R. Devon; BUCHWALTER, William. Learning representations by maximizing mutual information across views. Advances in neural information processing systems, 2019, 32.

review paper

not read yet

• none