?url_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Adc&rft.title=LLM-Assisted+Multi-Teacher+Continual+Learning+for+Visual+Question+Answering+in+Robotic+Surgery&rft.creator=Chen%2C+K&rft.creator=Du%2C+Y&rft.creator=You%2C+T&rft.creator=Islam%2C+M&rft.creator=Guo%2C+Z&rft.creator=Jin%2C+Y&rft.creator=Chen%2C+G&rft.creator=Heng%2C+PA&rft.description=Visual+question+answering+(VQA)+can+be+fundamentally+crucial+for+promoting+robotic-assisted+surgical+education.+In+practice%2C+the+needs+of+trainees+are+constantly+evolving%2C+such+as+learning+more+surgical+types+and+adapting+to+new+surgical+instruments%2Ftechniques.+Therefore%2C+continually+updating+the+VQA+system+by+a+sequential+data+stream+from+multiple+resources+is+demanded+in+robotic+surgery+to+address+new+tasks.+In+surgical+scenarios%2C+the+privacy+issue+of+patient+data+often+restricts+the+availability+of+old+data+when+updating+the+model%2C+necessitating+an+exemplar-free+continual+learning+(CL)+setup.+However%2C+prior+studies+overlooked+two+vital+problems+of+the+surgical+domain%3A+i)+large+domain+shifts+from+diverse+surgical+operations+collected+from+multiple+departments+or+clinical+centers%2C+and+ii)+severe+data+imbalance+arising+from+the+uneven+presence+of+surgical+instruments+or+activities+during+surgical+procedures.+This+paper+proposes+to+address+these+two+problems+with+a+multimodal+large+language+model+(LLM)+and+an+adaptive+weight+assignment+methodology.+We+first+develop+a+new+multi-teacher+CL+framework+that+leverages+a+multimodal+LLM+as+the+additional+teacher.+The+strong+generalization+ability+of+the+LLM+can+bridge+the+knowledge+gap+when+domain+shifts+and+data+imbalances+occur.+We+then+put+forth+a+novel+data+processing+method+that+transforms+complex+LLM+embeddings+into+logits+compatible+with+our+CL+framework.+We+also+design+an+adaptive+weight+assignment+approach+that+balances+the+generalization+ability+of+the+LLM+and+the+domain+expertise+of+the+old+CL+model.+Finally%2C+we+construct+a+new+dataset+for+surgical+VQA+tasks.+Extensive+experimental+results+demonstrate+the+superiority+of+our+method+to+other+advanced+CL+models.&rft.subject=Continuing+education%2C+%0D%0AAdaptation+models%2C+%0D%0AVisualization%2C+%0D%0AInstruments%2C+%0D%0ALarge+language+models%2C+%0D%0ASurgery%2C+%0D%0ATransforms&rft.publisher=IEEE&rft.date=2024-08-08&rft.type=Proceedings+paper&rft.language=eng&rft.source=+++++In%3A++Proceedings+-+IEEE+International+Conference+on+Robotics+and+Automation.++(pp.+pp.+10772-10778).++IEEE%3A+Yokohama%2C+Japan.+(2024)+++++&rft.format=text&rft.identifier=https%3A%2F%2Fdiscovery.ucl.ac.uk%2Fid%2Feprint%2F10197049%2F1%2FLLM-Assisted.pdf&rft.identifier=https%3A%2F%2Fdiscovery.ucl.ac.uk%2Fid%2Feprint%2F10197049%2F&rft.rights=open