Saeed, SU;
Huang, S;
Ramalhinho, J;
Gayo, IJMB;
Montaña-Brown, N;
Bonmati, E;
Pereira, SS;
... Hu, Y; + view all
(2024)
Competing for pixels: a self-play algorithm for weakly-supervised semantic segmentation.
IEEE Transactions on Pattern Analysis and Machine Intelligence
(In press).
Preview |
Text
selfplay_seg_manuscript-3.pdf - Accepted Version Download (6MB) | Preview |
Abstract
Weakly-supervised semantic segmentation (WSSS) methods, reliant on image-level labels indicating object presence, lack explicit correspondence between labels and regions of interest (ROIs), posing a significant challenge. Despite this, WSSS methods have attracted attention due to their much lower annotation costs compared to fully-supervised segmentation. Leveraging reinforcement learning (RL) self-play, we propose a novel WSSS method that gamifies image segmentation of a ROI. We formulate segmentation as a competition between two agents that compete to select ROI-containing patches until exhaustion of all such patches. The score at each time-step, used to compute the reward for agent training, represents likelihood of object presence within the selection, determined by an object presence detector pre-trained using only image-level binary classification labels of object presence. Additionally, we propose a game termination condition that can be called by either side upon exhaustion of all ROI-containing patches, followed by the selection of a final patch from each. Upon termination, the agent is incentivised if ROIcontaining patches are exhausted or disincentivised if a ROIcontaining patch is found by the competitor. This competitive setup ensures minimisation of over- or under-segmentation, a common problem with WSSS methods. Extensive experimentation across four datasets demonstrates significant performance improvements over recent state-of-the-art methods. Code: https://github.com/s-sd/spurl/tree/main/wss
| Type: | Article |
|---|---|
| Title: | Competing for pixels: a self-play algorithm for weakly-supervised semantic segmentation |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?pu... |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Self-Play, Weak Supervision, Segmentation |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Med Phys and Biomedical Eng |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10197984 |
Archive Staff Only
 |
View Item |
