Pu, Wei;
Sober, Barak;
Daly, Nathan;
Zhou, Chao;
Sabetsarvestani, Zahra;
Higgitt, Catherine;
Daubechies, Ingrid;
(2023)
Image Separation with Side Information: A Connected Auto-Encoders Based Approach.
IEEE Transactions on Image Processing
10.1109/tip.2023.3275872.
(In press).
Preview |
Text
TIP-main1.pdf - Accepted Version Download (988kB) | Preview |
Abstract
X-radiography (X-ray imaging) is a widely used imaging technique in art investigation. It can provide information about the condition of a painting as well as insights into an artist’s techniques and working methods, often revealing hidden information invisible to the naked eye. X-radiograpy of double-sided paintings results in a mixed X-ray image and this paper deals with the problem of separating this mixed image. Using the visible color images (RGB images) from each side of the painting, we propose a new Neural Network architecture, based upon ’connected’ auto-encoders, designed to separate the mixed X-ray image into two simulated X-ray images corresponding to each side. This connected auto-encoders architecture is such that the encoders are based on convolutional learned iterative shrinkage thresholding algorithms (CLISTA) designed using algorithm unrolling techniques, whereas the decoders consist of simple linear convolutional layers; the encoders extract sparse codes from the visible image of the front and rear paintings and mixed X-ray image, whereas the decoders reproduce both the original RGB images and the mixed X-ray image. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The methodology was tested on images from the double-sided wing panels of the Ghent Altarpiece , painted in 1432 by the brothers Hubert and Jan van Eyck. These tests show that the proposed approach outperforms other state-of-the-art X-ray image separation methods for art investigation applications.
| Type: | Article |
|---|---|
| Title: | Image Separation with Side Information: A Connected Auto-Encoders Based Approach |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/tip.2023.3275872 |
| Publisher version: | https://doi.org/10.1109/TIP.2023.3275872 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | X-ray imaging, Painting, Paints, Gray-scale, Deep learning, Art, Task analysis |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Electronic and Electrical Eng |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10170474 |
Archive Staff Only
 |
View Item |
