eprintid: 10199672
rev_number: 7
eprint_status: archive
userid: 699
dir: disk0/10/19/96/72
datestamp: 2024-11-07 16:03:41
lastmod: 2024-11-07 16:03:41
status_changed: 2024-11-07 16:03:41
type: proceedings_section
metadata_visibility: show
sword_depositor: 699
creators_name: Feng, Chen
creators_name: Tzimiropoulos, Georgios
creators_name: Patras, Ioannis
title: CLIPCleaner: Cleaning Noisy Labels with CLIP
ispublished: pub
divisions: UCL
divisions: B04
divisions: F46
keywords: Sample selection, Noisy Labels, CLIP
note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
abstract: Learning with Noisy labels (LNL) poses a significant challenge for the Machine Learning community. Some of the most widely used approaches that select as clean samples for which the model itself (the in-training model) has high confidence, e.g., 'small loss', can suffer from the so called 'self-confirmation' bias. This bias arises because the in-training model, is at least partially trained on the noisy labels. Furthermore, in the classification case, an additional challenge arises because some of the label noise is between classes that are visually very similar (`hard noise'). This paper addresses these challenges by proposing a method (CLIPCleaner) that leverages CLIP, a powerful Vision-Language (VL) model for constructing a zero-shot classifier for efficient, offline, clean sample selection. This has the advantage that the sample selection is decoupled from the in-training model and that the sample selection is aware of the semantic and visual similarities between the classes due to the way that CLIP is trained. We provide theoretical justifications and empirical evidence to demonstrate the advantages of CLIP for LNL compared to conventional pre-trained models. Compared to current methods that combine iterative sample selection with various techniques, CLIPCleaner offers a simple, single-step approach that achieves competitive or superior performance on benchmark datasets. To the best of our knowledge, this is the first time a VL model has been used for sample selection to address the problem of Learning with Noisy Labels (LNL), highlighting their potential in the domain.
date: 2024-10-28
date_type: published
publisher: ACM
official_url: https://doi.org/10.1145/3664647.3680664
oa_status: green
full_text_type: other
language: eng
primo: open
primo_central: open_green
verified: verified_manual
elements_id: 2332666
doi: 10.1145/3664647.3680664
lyricists_name: Feng, Chen
lyricists_id: CFENA90
actors_name: Flynn, Bernadette
actors_id: BFFLY94
actors_role: owner
full_text_status: public
pres_type: paper
publication: Proceedings of the 32nd ACM International Conference on Multimedia
pagerange: 876-885
event_title: MM '24: The 32nd ACM International Conference on Multimedia
book_title: Proceedings of the 32nd ACM International Conference on Multimedia
citation:        Feng, Chen;    Tzimiropoulos, Georgios;    Patras, Ioannis;      (2024)    CLIPCleaner: Cleaning Noisy Labels with CLIP.                     In:  Proceedings of the 32nd ACM International Conference on Multimedia.  (pp. pp. 876-885).  ACM       Green open access   
 
document_url: https://discovery.ucl.ac.uk/id/eprint/10199672/1/ACMMM2024_CLIPCleaner_camera_ready.pdf