Bourached, Anthony;
Bourached, Anthony;
(2022)
Extracting associations and meanings of objects depicted in artworks through bi-modal deep networks.
In:
IS&T International Symposium on Electronic Imaging 2023 / Computer Vision and Image Analysis of Art 2023.
(pp. 170-1-170-14).
Society for Imaging Science and Technology
Preview |
Text
EI_2022_CVAA-170_Kell-Gregory-.pdf - Published Version Download (15MB) | Preview |
Abstract
We present a novel bi-modal system based on deep networks to address the problem of learning associations and simple meanings of objects depicted in "authored" images, such as ne art paintings and drawings. Our overall system processes both the images and associated texts in order to learn associations between images of individual objects, their identities and the abstract meanings they signify. Unlike past deep net that describe depicted objects and infer predicates, our system identies meaning-bearing objects ("signifiers") and their associations ("signifieds") as well as basic overall meanings for target artworks. Our system had precision of 48% and recall of 78% with an F1 metric of 0.6 on a curated set of Dutch vanitas paintings, a genre celebrated for its concentration on conveying a meaning of great import at the time of their execution. We developed and tested our system on ne art paintings but our general methods can be applied to other authored images.
| Type: | Proceedings paper |
|---|---|
| Title: | Extracting associations and meanings of objects depicted in artworks through bi-modal deep networks |
| Event: | Computer Vision and Analysis of Art |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.2352/EI.2022.34.13.CVAA-170 |
| Publisher version: | https://doi.org/10.2352/EI.2022.34.13.CVAA-170 |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Education UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Culture, Communication and Media |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10209689 |
Archive Staff Only
 |
View Item |
