eprintid: 10199675
rev_number: 8
eprint_status: archive
userid: 699
dir: disk0/10/19/96/75
datestamp: 2024-11-07 14:43:32
lastmod: 2024-11-07 14:43:32
status_changed: 2024-11-07 14:43:32
type: proceedings_section
metadata_visibility: show
sword_depositor: 699
creators_name: Yuan, H
creators_name: Xu, J
creators_name: Pan, H
creators_name: Bousseau, A
creators_name: Mitra, NJ
creators_name: Li, C
title: CADTalk: An Algorithm and Benchmark for Semantic Commenting of CAD Programs
ispublished: pub
divisions: UCL
divisions: B04
divisions: F48
note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
abstract: CAD programs are a popular way to compactly encode shapes as a sequence of operations that are easy to para-metrically modify. However, without sufficient semantic comments and structure, such programs can be challenging to understand, let alone modify. We introduce the problem of semantic commenting CAD programs, wherein the goal is to segment the input program into code blocks corresponding to semantically meaningful shape parts and assign a semantic label to each block. We solve the problem by combining program parsing with visual-semantic analysis afforded by recent advances in foundational language and vision models. Specifically, by executing the input programs, we create shapes, which we use to generate conditional photorealistic images to make use of semantic annotators for such images. We then distill the information across the images and link back to the original programs to semantically comment on them. Additionally, we collected and annotated a benchmark dataset, CADTalk, consisting of 5,288 machine-made programs and 45 human-made programs with ground truth semantic comments. We exten-sively evaluated our approach, compared it to a GPT-based baseline, and an open-set shape segmentation baseline, and reported an 83.24% accuracy on the new CADTalk dataset. Code and data: https://enigma-li.github.io/CADTalk/.
date: 2024-09-16
date_type: published
publisher: IEEE
official_url: https://doi.org/10.1109/CVPR52733.2024.00360
oa_status: green
full_text_type: other
language: eng
primo: open
primo_central: open_green
verified: verified_manual
elements_id: 2332605
doi: 10.1109/CVPR52733.2024.00360
lyricists_name: Mitra, Niloy
lyricists_id: NMITR19
actors_name: Flynn, Bernadette
actors_id: BFFLY94
actors_role: owner
full_text_status: public
pres_type: paper
publication: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
place_of_pub: Seattle, WA, USA
pagerange: 3753-3762
event_title: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
event_dates: 16 Jun 2024 - 22 Jun 2024
issn: 1063-6919
book_title: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
citation:        Yuan, H;    Xu, J;    Pan, H;    Bousseau, A;    Mitra, NJ;    Li, C;      (2024)    CADTalk: An Algorithm and Benchmark for Semantic Commenting of CAD Programs.                     In:  Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.  (pp. pp. 3753-3762).  IEEE: Seattle, WA, USA.       Green open access   
 
document_url: https://discovery.ucl.ac.uk/id/eprint/10199675/1/Yuan_CADTalk_An_Algorithm_and_Benchmark_for_Semantic_Commenting_of_CAD_CVPR_2024_paper.pdf
document_url: https://discovery.ucl.ac.uk/id/eprint/10199675/7/Yuan_CADTalk_An_Algorithm_CVPR_2024_supplemental.pdf