eprintid: 10185309 rev_number: 6 eprint_status: archive userid: 699 dir: disk0/10/18/53/09 datestamp: 2024-01-11 15:17:27 lastmod: 2024-01-11 15:17:27 status_changed: 2024-01-11 15:17:27 type: article metadata_visibility: show sword_depositor: 699 creators_name: Zhang, Juncheng creators_name: Liao, Qingmin creators_name: Ma, Haoyu creators_name: Xue, Jing-Hao creators_name: Yang, Wenming creators_name: Liu, Shaojun title: Exploit the Best of Both End-to-End and Map-Based Methods for Multi-Focus Image Fusion ispublished: inpress divisions: UCL divisions: B04 divisions: C06 divisions: F61 keywords: Deep learning , Transforms , Feature extraction , Image fusion , Image segmentation , Fuses , Visualization note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. abstract: Multi-focus image fusion is a technique to fuse the images focused on different depth ranges to generate an all-in-focus image. Existing deep learning approaches to multi-focus image fusion can be categorized as end-to-end methods and decision map based methods. End-to-end methods can generate natural fusion near the focus-defocus boundaries (FDB), but the output is often inconsistent with the input in the areas far from the boundaries (FFB). On the contrary, decision map based methods can preserve original images in the FFB areas, but often generate artifacts near the FDB. In this paper, we propose a dual-branch network for multi-focus image fusion (DB-MFIF) to exploit the best of both worlds, achieving better results in both FDB and FFB areas, i.e. with naturally sharper FDB areas and more consistent FFB areas with the inputs. In our DB-MFIF, an end-to-end branch and a decision map based branch are proposed to mutually assist each other. In addition, to this end, two map-based loss functions are also proposed. Experiments show that our method surpasses existing algorithms on multiple datasets, both qualitatively and quantitatively, and achieves the state-of-the-art performance. The code and model is available on GitHub: https://github.com/Zancelot/DB-MFIF . date: 2024-01-08 date_type: published publisher: Institute of Electrical and Electronics Engineers (IEEE) official_url: http://dx.doi.org/10.1109/tmm.2024.3350924 oa_status: green full_text_type: other language: eng primo: open primo_central: open_green verified: verified_manual elements_id: 2139144 doi: 10.1109/tmm.2024.3350924 lyricists_name: Xue, Jinghao lyricists_id: JXUEX60 actors_name: Xue, Jinghao actors_id: JXUEX60 actors_role: owner full_text_status: public publication: IEEE Transactions on Multimedia citation: Zhang, Juncheng; Liao, Qingmin; Ma, Haoyu; Xue, Jing-Hao; Yang, Wenming; Liu, Shaojun; (2024) Exploit the Best of Both End-to-End and Map-Based Methods for Multi-Focus Image Fusion. IEEE Transactions on Multimedia 10.1109/tmm.2024.3350924 <https://doi.org/10.1109/tmm.2024.3350924>. (In press). Green open access document_url: https://discovery.ucl.ac.uk/id/eprint/10185309/1/JunchengZhang-TMM-accepted.pdf