In recent years, there has been impressive progress in learning with noisy labels, particularly in leveraging a small set of clean data. Meta-learning-based label correction techniques have further advanced performance by correcting noisy labels during training. However, these methods require multiple back-propagation steps, which considerably slows down the training process. Alternatively, some researchers have attempted to estimate the label transition matrix on-the-fly to address the issue of noisy labels. These approaches are more robust and faster than meta-learning-based techniques. The use of the transition matrix makes the classifier skeptical about all corrected samples, thereby mitigating the problem of label noise. We propose a novel three-head architecture that can efficiently estimate the label transition matrix and two new label smoothing matrices at each iteration. Our approach enables the estimated matrices to closely follow the shifting noise and reduce over-confidence on classes during classifier model training. We report extensive experiments on synthetic and real world noisy datasets, achieving state of the art performance on synthetic variants of CIFAR-10/100 and on the challenging Clothing1M datasets. Code at https://github.com/z3n0e/STM.

Smoothing and Transition Matrices Estimation to Learn with Noisy Labels

Uricchio T.;
2023-01-01

Abstract

In recent years, there has been impressive progress in learning with noisy labels, particularly in leveraging a small set of clean data. Meta-learning-based label correction techniques have further advanced performance by correcting noisy labels during training. However, these methods require multiple back-propagation steps, which considerably slows down the training process. Alternatively, some researchers have attempted to estimate the label transition matrix on-the-fly to address the issue of noisy labels. These approaches are more robust and faster than meta-learning-based techniques. The use of the transition matrix makes the classifier skeptical about all corrected samples, thereby mitigating the problem of label noise. We propose a novel three-head architecture that can efficiently estimate the label transition matrix and two new label smoothing matrices at each iteration. Our approach enables the estimated matrices to closely follow the shifting noise and reduce over-confidence on classes during classifier model training. We report extensive experiments on synthetic and real world noisy datasets, achieving state of the art performance on synthetic variants of CIFAR-10/100 and on the challenging Clothing1M datasets. Code at https://github.com/z3n0e/STM.
2023
978-3-031-43147-0
978-3-031-43148-7
File in questo prodotto:
File Dimensione Formato  
samplepaper.pdf

solo utenti autorizzati

Tipologia: Documento in post-print (versione successiva alla peer review e accettata per la pubblicazione)
Licenza: Tutti i diritti riservati
Dimensione 556.38 kB
Formato Adobe PDF
556.38 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11393/326452
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
social impact